Commit Graph

14369 Commits

Author SHA1 Message Date
yyforyongyu
9b61e09ca7
docs: update release notes for gossip fix
This commit also puts relevant docs closer to each other.
2023-02-17 14:16:31 +08:00
yyforyongyu
e34a088608
discovery: send both local and remote anns in the same goroutine
This commit changes the sending of anns from using separate goroutines
to always sending both local and remote announcements in the same
goroutine. In addition, the local announcements are always sent first.
This change is to fix the following case:

1. Alice and Bob have a channel
2. Alice receives Bob's NodeAnnouncement
3. Alice goes to broadcast the channel
4. The broadcast is split into a local and remote broadcast due to PR
   #7239. Bob's NodeAnnouncement is in the remote batch. Everything else
   (ChannelAnnouncement, ChannelUpdate x2, and Alice's NodeAnnouncement)
   is in the local batch.
5. The remote batch (containing Bob's NodeAnnouncement) runs before the
   local batch since they are spawned in separate goroutines. This means
   that Alice sends Carol the NodeAnnouncement before Carol knows of the
   channel.

In step 2), Bob's NodeAnnouncement (isRemote = true) replaces Bob's
NodeAnnouncement that Alice was going to relay (isRemote = false) after
processing the AnnouncementSignatures.
2023-02-17 14:10:19 +08:00
yyforyongyu
b73cfc5998
discovery: split sendBatch into local and remote
This commit refactors the method `sendBatch` into `sendLocalBatch` and
`sendRemoteBatch` for clarity. The batch size calculation is also moved
into `splitAnnouncementBatches`.
2023-02-17 14:10:10 +08:00
yyforyongyu
c3d1d3c4f1
multi: make SubBatchDelay configurable
This commit adds a new config option `--gossip.sub-batch-delay` so we
can speed up our itest.
2023-02-17 14:10:09 +08:00
yyforyongyu
c6c218f384
discovery: fix confusing loggings 2023-02-17 14:10:09 +08:00
Oliver Gugger
93b7821df3
Merge pull request #7408 from carlaKC/bugfix-updatenodeanncli
bugfix/trivial: fix update node announcement feature bit add /remove
2023-02-16 21:40:36 +01:00
Carla Kirk-Cohen
03a61617f8
release-notes: add UpdateNodeAnnouncement changes 2023-02-16 14:42:20 -05:00
Oliver Gugger
38dc67e1ef
Merge pull request #6815 from bitromortac/2205-bimodal
pathfinding: probability for bimodal distribution
2023-02-15 14:19:32 +01:00
Carla Kirk-Cohen
3cc8e81121
lncli/bugfix: use int64slice for update node announcement features
As is, we'll never get any add/remove features because the flag is
set as an int 64 slice but accessed as an int slice (which has no
value, and the cli doesn't fail if the flag doesn't exist).
2023-02-14 15:53:56 -05:00
bitromortac
d78e756406
routing: add probability benchmarks
$ go test -bench=PairProbability ./routing
goos: linux
goarch: amd64
pkg: github.com/lightningnetwork/lnd/routing
cpu: AMD Ryzen 5 3600 6-Core Processor
BenchmarkBimodalPairProbability-4   	 2050206	       599.8 ns/op
BenchmarkAprioriPairProbability-4   	 9524289	       126.7 ns/op

* math.Exp:
goos: linux
goarch: amd64
pkg: github.com/lightningnetwork/lnd/routing
cpu: AMD Ryzen 5 3600 6-Core Processor
BenchmarkExp-4   	143843622	         8.700 ns/op

* bimodal benchmark profile:
Showing nodes accounting for 2000ms, 76.05% of 2630ms total
Dropped 156 nodes (cum <= 13.15ms)
Showing top 10 nodes out of 141
      flat  flat%   sum%        cum   cum%
    1110ms 42.21% 42.21%     1110ms 42.21%  math.archExp
     230ms  8.75% 50.95%      230ms  8.75%  runtime.memclrNoHeapPointers
     180ms  6.84% 57.79%      930ms 35.36%  github.com/lightningnetwork/lnd/routing.(*BimodalEstimator).calculateProbability
     110ms  4.18% 61.98%      130ms  4.94%  runtime.scanobject
      90ms  3.42% 65.40%       90ms  3.42%  memeqbody
      60ms  2.28% 67.68%      940ms 35.74%  github.com/lightningnetwork/lnd/routing.(*BimodalEstimator).directProbability
      60ms  2.28% 69.96%      230ms  8.75%  github.com/lightningnetwork/lnd/routing.cannotSend
      60ms  2.28% 72.24%       60ms  2.28%  runtime.madvise
      50ms  1.90% 74.14%      480ms 18.25%  github.com/lightningnetwork/lnd/routing.(*BimodalEstimator).primitive
      50ms  1.90% 76.05%       60ms  2.28%  runtime.mapiternext

* apriori benchmark profile:
Showing nodes accounting for 1570ms, 77.34% of 2030ms total
Dropped 138 nodes (cum <= 10.15ms)
Showing top 10 nodes out of 151
      flat  flat%   sum%        cum   cum%
     340ms 16.75% 16.75%      340ms 16.75%  math.archExp
     290ms 14.29% 31.03%      980ms 48.28%  github.com/lightningnetwork/lnd/routing.(*AprioriEstimator).getNodeProbability
     190ms  9.36% 40.39%      260ms 12.81%  runtime.mapiternext
     190ms  9.36% 49.75%      190ms  9.36%  runtime.memclrNoHeapPointers
     160ms  7.88% 57.64%      340ms 16.75%  github.com/lightningnetwork/lnd/routing.(*AprioriEstimator).calculateProbability
     110ms  5.42% 63.05%      110ms  5.42%  aeshashbody
     110ms  5.42% 68.47%      130ms  6.40%  runtime.scanobject
      80ms  3.94% 72.41%      420ms 20.69%  github.com/lightningnetwork/lnd/routing.capacityFactor
      60ms  2.96% 75.37%       60ms  2.96%  runtime.madvise
      40ms  1.97% 77.34%       40ms  1.97%  runtime.isEmpty (inline)
2023-02-14 13:34:25 +01:00
bitromortac
2ccdfb1151
lncli+routerrpc: adapt mc api to dynamic estimator
The active probability estimator can be switched dynamically using the
`Set/GetMissionControl` API, maintaining backward compatibility. The
lncli commands `setmccfg` and `getmccfg` are updated around this
functionality. Note that deprecated configuration parameters were
removed from the commands.
2023-02-14 13:34:25 +01:00
bitromortac
58d5131e31
lnd+routerrpc: configure server with new estimator
We add new lnd.conf configuration options for both probability
estimators. Note that this is a breaking change for the existing apriori
parameters.
2023-02-14 13:34:25 +01:00
bitromortac
16986ee5c7
lnd+routing+rpc: switch mc to (external) estimator
We use a more general `Estimator` interface for probability estimation
in missioncontrol.

The estimator is created outside of `NewMissionControl`, passed in as a
`MissionControlConfig` field, to facilitate usage of externally supplied
estimators.
2023-02-14 13:34:25 +01:00
bitromortac
686816d784
routing: implement bimodal probability estimator
Implements a new probability estimator based on a probability theory
framework.

The computed probability consists of:
* the direct channel probability, which is estimated based on a
  depleted liquidity distribution model, formulas and broader concept derived
  after Pickhardt et al. https://arxiv.org/abs/2103.08576
* an extension of the probability model to incorporate knowledge decay
  after time for previous successes and failures
* a mixed node probability taking into account successes/failures on other
  channels of the node (similar to the apriori approach)
2023-02-14 13:34:25 +01:00
Oliver Gugger
f245591e4b
Merge pull request #7386 from guggero/remote-signer-p2tr-inputs
remote signer: fix channel funding with mixed funding input types
2023-02-14 12:34:11 +01:00
Oliver Gugger
c79b0799bf
Merge pull request #7402 from bottlepay/final-resolution-info
rpcserver: expose store final htlc resolutions flag on info call
2023-02-13 23:12:32 +01:00
Oliver Gugger
8ade1acba3
Merge pull request #7399 from prusnak/sample-conf-whitespace
docs: fix whitespace in sample-lnd.conf
2023-02-13 12:54:42 +01:00
Joost Jager
784c0631f3
rpcserver: expose store final htlc resolutions flag on info call
The storage of final htlc resolutions is a safety-critical feature. By
exposing it through the GetInfo call, connected applications can
ensure that the feature is turned on and avoid relying on the assumption
that lnd was configured correctly.
2023-02-13 11:53:27 +01:00
Joost Jager
db5dc4d360
rpcserver: fix line lengths 2023-02-13 11:53:10 +01:00
Oliver Gugger
0c5971ba9e
Merge pull request #7392 from adiabat/hltcinterceptfix
HTLC Interceptor: Fix nil pointer dereference
2023-02-13 11:43:29 +01:00
Pavol Rusnak
2441c2e016
docs: fix whitespace in sample-lnd.conf 2023-02-12 11:30:19 +01:00
adiabat
17f7b9f322 routerrpc: Check for nil IncomingCircuitKey
The grpc stream.Recv() will decode an ForwardHtlcInterceptResponse
without an IncomingCircuit, which will leave that pointer nil,
causing a nil pointer dereferece when components of in.IncomingCircuitKey
are used in the resolveFromClient() function.

This commit checks for the nil pointer and returns an error before
the rest of the parsing.
2023-02-10 13:40:40 -05:00
Oliver Gugger
939614b70d
Merge pull request #7395 from yyforyongyu/fix-unit-test-db
channeldb+invoices: fix unit test flakes
2023-02-10 14:27:17 +01:00
yyforyongyu
4a7d3bc9c2
invoices: fix flake in TestInvoiceExpiryWithRegistry 2023-02-10 20:52:36 +08:00
yyforyongyu
3d61d157c6
channeldb: add unique output index to unit test 2023-02-10 20:52:34 +08:00
bitromortac
b8c6227383
routing: add probability estimator interface
We introduce a probability `Estimator` interface which is implemented by
the current apriori probability estimator. A second implementation, the
bimodal probability estimator follows.
2023-02-10 09:58:40 +01:00
Oliver Gugger
f94500281f
Merge pull request #7348 from Torakushi/psbt
Add possibility to define a change scope in FundPsbt
2023-02-08 16:27:58 +01:00
Torakushi
9edc98f940
update release-notes-0.16.0 2023-02-08 12:35:48 +01:00
Torakushi
b3cbab4482
lncli: possibility to define an address type for changes in FundPsbt
With this commit, we can define an address type for default/imported
accounts in FundPsbt using '—change_type' flag. The value for this
flag can only be 'p2tr' at the moment. If not specified, we will use
P2WKH addresses for change outputs. Custom accounts don’t support
this flag as they have a unique scope.
2023-02-08 12:35:47 +01:00
Torakushi
cdb015223b
walletrpc+lnwallet: possibility to define an address type for changes in FundPsbt
By default, P2TR addresses are used for changes. However, some users
might encounter some problems with this change. We add the possibility
to define a custom address type in FundPsbt for default/imported accounts
(only P2TR for now). If no address type is specified for these accounts,
we will use P2WKH by default.
2023-02-08 12:35:44 +01:00
Torakushi
3bb68661e7
mod: update btcwallet dependency 2023-02-08 10:03:30 +01:00
Oliver Gugger
e7fb48ff73
docs: add release notes 2023-02-07 22:15:14 +01:00
Oliver Gugger
df1ea46d9e
rpcwallet: fix p2tr inputs in remote signing
The SendOutputs method isn't used very often in our code so the missing
Taproot sighash type wasn't detected before.
Also, a P2TR input will never have a sigScript, so we can explicitly set
that parameter to nil instead of relying on it being nil anyway.
2023-02-07 22:15:13 +01:00
Oliver Gugger
8bc16b4fb2
rpcwallet: fix np2wkh inputs in remote signing
There is a one byte difference between the sigScript and the
witnessScript in case of a np2wkh input. The remote signer actually
needs the witness script and not the sig script to produce a valid
signature.
2023-02-07 22:15:12 +01:00
Oliver Gugger
c6c52b8eb3
itest: add coverage for np2wkh in remote signing 2023-02-07 22:15:11 +01:00
Oliver Gugger
a2a96c7712
lnwallet: don't re-use sign descriptor
Fixes an issue where re-using a sign descriptor in a loop carried over
signing information from one call to the next, which caused the remote
signing issue.
2023-02-07 22:15:10 +01:00
Oliver Gugger
e4f3a35c0c
funding: fix peer key serialization in log message 2023-02-07 22:15:09 +01:00
Oliver Gugger
5d18fccd42
itest: reproduce remote signing issue
This commit attempts to reproduce the issue as described in #7276.
2023-02-07 22:15:08 +01:00
Oliver Gugger
a233df99c4
Merge pull request #7388 from bhandras/cert-fixup-lnd
lnd: bump lnd/cert version to v1.2.1
2023-02-07 17:23:25 +01:00
bitromortac
b0a998af8d
routing: rename apriori estimator
* we rename the current probability estimator to be the "apriori"
  probability estimator to distinguish from a different implementation
  later
* the AprioriEstimator is exported to later be able to type switch
* getLocalPairProbability -> LocalPairProbabiltiy (later part of an
  exported interface)
* getPairProbability -> getPairProbabiltiy (later part of an exported
  interface)
2023-02-07 16:18:38 +01:00
Andras Banki-Horvath
91af62a031
lnd: bump lnd/cert version
This commit bumps the lnd/cert version to 1.2.1 and fixes all call sites
where cert.GenCertPair was called.
2023-02-07 09:50:56 +01:00
Oliver Gugger
17d7e84b26
Merge pull request #7387 from bhandras/cert-fixup
cert: removed unused args from `cert.GenCertPair` [no-itest]
2023-02-06 22:52:42 +01:00
Andras Banki-Horvath
c3445e79d9
cert: removed unused args from cert.GenCertPair
Since the cert pair is written with a subsequent call to
cert.WriteCertPair we can safely remove the two file paths from the
argument list of cert.GenCertPair.
2023-02-06 17:11:56 +01:00
Oliver Gugger
3ba975b4db
Merge pull request #7384 from guggero/close-mac-db
walletunlocker: fix unit test flake by closing DB
2023-02-06 16:57:42 +01:00
Oliver Gugger
ad6fe6a78e
Merge pull request #7385 from guggero/github-label-trigger
GitHub: allow skipping itest and changelog check with labels
2023-02-06 15:50:16 +01:00
Oliver Gugger
6874582ed4
GitHub: allow skipping itest and changelog check with labels 2023-02-06 14:11:59 +01:00
Oliver Gugger
aa3647de82
docs: add release notes 2023-02-06 14:08:09 +01:00
Oliver Gugger
2086f09489
walletunlocker: fix unit test flake by closing DB
This fixes a unit test flake that occurred sometimes if the temporary
directory was attempted to be deleted but the wallet or macaroon DB
hasn't been closed yet.

````
--- FAIL: TestChangeWalletPasswordNewRootkey (1.63s)
    testing.go:1097: TempDir RemoveAll cleanup: unlinkat /tmp/TestChangeWalletPasswordNewRootkey3063283009/001/mainnet: directory not empty
FAIL
FAIL	github.com/lightningnetwork/lnd/walletunlocker	6.171s
FAIL
````
2023-02-06 14:08:08 +01:00
Oliver Gugger
0cf0a7dd3b
Merge pull request #7341 from bottlepay/final-settle-opt-in
channeldb: final htlc resolution storage opt-in
2023-02-06 13:21:40 +01:00
Oliver Gugger
75a39da01a
Merge pull request #7171 from guggero/musig2-versioned-rpc
signrpc: Upgrade to MuSig2 BIP draft v1.0.0rc2, add version flag to RPC
2023-02-06 12:35:49 +01:00