Commit Graph

14408 Commits

Author SHA1 Message Date
yyforyongyu
a080375b7d
itest: remove unnecessary shutdown 2023-02-23 21:56:09 +08:00
yyforyongyu
8b9ccfe310
itest: fix test async_bidirectional_payments 2023-02-23 21:56:09 +08:00
yyforyongyu
9d1d629001
itest+lntest: migrate lntemp to lntest
This commit performs the takeover that `lntemp` is now promoted to be
`lntest`, and the scaffolding is now removed as all the refactoring is
finished!
2023-02-23 21:56:09 +08:00
yyforyongyu
ee0790493c
itest+lntemp: move calculation functions into lntemp
Functions that can be useful to other tests are now moved into the
package `lntemp`.
2023-02-23 21:56:09 +08:00
yyforyongyu
4a9c3449a6
lntemp+itest: move utils function into lntemp 2023-02-23 21:56:09 +08:00
yyforyongyu
edba938996
multi: add new build tag integration
This commit adds a new build tag `integration` and removes the old tag
`rpctest` for clarity. Multiple unnecessary usages of `build !rpctest`
is also removed.
2023-02-23 21:56:09 +08:00
yyforyongyu
d730797880
itest: remove unused flag goroutineDump 2023-02-23 21:56:08 +08:00
yyforyongyu
94c64a886e
lntemp+itest: remove unused code to prepare lntemp's takeover
This commit removes the old code living in `lntest` to prepare
`lntemp`'s takeover.
2023-02-23 21:56:08 +08:00
yyforyongyu
106fbeae85
multi: move timeouts into package wait
This commit moves the definition of timeout values into package `wait`,
preparing the incoming merging of `lntemp` and `lntest`.
2023-02-23 21:56:08 +08:00
yyforyongyu
0bc86a3b4b
multi: move itest out of lntest
This commit moves all the test cases living in `itest` out of `lntest`,
further making `lntest` an independent package for general testing.
2023-02-23 21:56:08 +08:00
yyforyongyu
142b00711f
itest+github: remove flag temptest
This commit removes flag `temptest` and all its usage, marking the start
of promoting our `lntemp` to be `lntest`!
2023-02-23 21:56:08 +08:00
yyforyongyu
e500074017
itest: refactor testBidirectionalAsyncPayments 2023-02-23 21:56:08 +08:00
yyforyongyu
6618ab493a
multi: enhance loggings and fix logging format
Also adds TODO for a possible bug.
2023-02-23 21:56:08 +08:00
Olaoluwa Osuntokun
d793a53bba
Merge pull request #7406 from Roasbeef/gossip-cache-fix
multi: bump neutrino+btcwallet versions, use new generic lru cache
2023-02-22 15:49:49 -08:00
Oliver Gugger
b3e27f9ba7
Merge pull request #7415 from bitromortac/2302-max-htlc
routing+discovery: fail non-maxHTLC channel updates in validation
2023-02-22 08:45:56 +01:00
Olaoluwa Osuntokun
a15c45231e
multi: bump neutrino+btcwallet versions, use new generic lru cache 2023-02-21 17:28:48 -08:00
Olaoluwa Osuntokun
f4fdd82327
Merge pull request #7264 from yyforyongyu/gossip-resend-messages
discovery: fix message order in batch sending
2023-02-21 17:10:13 -08:00
bitromortac
6459385dfd
discovery: skip non-maxHTLC updates in gossip sync
We skip syncing channel updates that don't conform to the spec.
2023-02-21 14:03:51 +01:00
bitromortac
6aac2762b3
routing: stricter maxHTLC checks
We require channel updates to have the max HTLC message flag set.

Several flows need to pass that check before channel updates are
forwarded to peers:
* after channel funding: `addToRouterGraph`
* after receiving channel updates from a peer:
  `ProcessRemoteAnnouncement`
* after we update channel policies: `PropagateChanPolicyUpdate`
2023-02-21 14:03:51 +01:00
Oliver Gugger
d44823f6e8
Merge pull request #7431 from Crypt-iQ/remove_res_ids
lnwallet: remove from reservationIDs
2023-02-21 12:36:10 +01:00
bitromortac
dd5273c88c
multi: rename due to required maxHTLC bit
We rename `ChanUpdateOptionMaxHtlc` to `ChanUpdateRequiredMaxHtlc`
as with the latest changes it is now required.

Similarly, rename `validateOptionalFields` to
`ValidateChannelUpdateFields`, export it to use it in a later commit.
2023-02-21 11:10:39 +01:00
Olaoluwa Osuntokun
748461cba4
Merge pull request #7379 from ellemouton/addFieldsToRevocationLog
multi: add local and remote amount fields to revocation log
2023-02-20 17:57:08 -08:00
eugene
2c7f118d4e
lnwallet: remove from reservationIDs 2023-02-20 12:47:45 -05:00
Oliver Gugger
0f079729c1
Merge pull request #7418 from lightningnetwork/dependabot/go_modules/kvdb/golang.org/x/net-0.7.0
build(deps): bump golang.org/x/net from 0.4.0 to 0.7.0 in /kvdb
2023-02-20 08:35:45 +01:00
dependabot[bot]
ded9fb87d2
build(deps): bump golang.org/x/net from 0.4.0 to 0.7.0 in /kvdb
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.4.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.4.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-17 16:03:17 +00:00
Oliver Gugger
581e195bfe
Merge pull request #7414 from guggero/merge-queue-fix
github: fix merge group directive
2023-02-17 16:59:44 +01:00
Oliver Gugger
a257e6203d
github: fix merge group directive 2023-02-17 11:14:38 +01:00
Oliver Gugger
4d7ac82773
Merge pull request #7412 from Roasbeef/merge-queue-fix
build: use the new merge_group selector for the merge queue
2023-02-17 11:08:51 +01:00
Oliver Gugger
ccb00741df
Merge pull request #7324 from hieblmi/openchannel-fix-line-length
lncli: fix line length issues
2023-02-17 09:30:00 +01:00
yyforyongyu
9b61e09ca7
docs: update release notes for gossip fix
This commit also puts relevant docs closer to each other.
2023-02-17 14:16:31 +08:00
yyforyongyu
e34a088608
discovery: send both local and remote anns in the same goroutine
This commit changes the sending of anns from using separate goroutines
to always sending both local and remote announcements in the same
goroutine. In addition, the local announcements are always sent first.
This change is to fix the following case:

1. Alice and Bob have a channel
2. Alice receives Bob's NodeAnnouncement
3. Alice goes to broadcast the channel
4. The broadcast is split into a local and remote broadcast due to PR
   #7239. Bob's NodeAnnouncement is in the remote batch. Everything else
   (ChannelAnnouncement, ChannelUpdate x2, and Alice's NodeAnnouncement)
   is in the local batch.
5. The remote batch (containing Bob's NodeAnnouncement) runs before the
   local batch since they are spawned in separate goroutines. This means
   that Alice sends Carol the NodeAnnouncement before Carol knows of the
   channel.

In step 2), Bob's NodeAnnouncement (isRemote = true) replaces Bob's
NodeAnnouncement that Alice was going to relay (isRemote = false) after
processing the AnnouncementSignatures.
2023-02-17 14:10:19 +08:00
yyforyongyu
b73cfc5998
discovery: split sendBatch into local and remote
This commit refactors the method `sendBatch` into `sendLocalBatch` and
`sendRemoteBatch` for clarity. The batch size calculation is also moved
into `splitAnnouncementBatches`.
2023-02-17 14:10:10 +08:00
yyforyongyu
c3d1d3c4f1
multi: make SubBatchDelay configurable
This commit adds a new config option `--gossip.sub-batch-delay` so we
can speed up our itest.
2023-02-17 14:10:09 +08:00
yyforyongyu
c6c218f384
discovery: fix confusing loggings 2023-02-17 14:10:09 +08:00
Olaoluwa Osuntokun
aec10481c1
build: use the new merge_group selector for the merge queue
In this commit, we activate the merge queue by using the new merge_group
selector. Without this, the CI won't report back the progress of a CI
run to the merge queue, so things won't get auto merged.
2023-02-16 15:00:20 -08:00
Oliver Gugger
93b7821df3
Merge pull request #7408 from carlaKC/bugfix-updatenodeanncli
bugfix/trivial: fix update node announcement feature bit add /remove
2023-02-16 21:40:36 +01:00
Carla Kirk-Cohen
03a61617f8
release-notes: add UpdateNodeAnnouncement changes 2023-02-16 14:42:20 -05:00
Elle Mouton
884cb2047b
docs: update release notes 2023-02-16 20:47:35 +02:00
Elle Mouton
91a2cd45d8
lnwallet: make spendTx param in NewBreachRetribution optional
In this commit, the NewBreachRetribution function is adjusted so that a
caller can optionally set the spendTx parameter to nil. In this case,
the function will check the revocation log to see if the local and
remote amount fields are available there and use them if they are.
If the fields are not present, which they might not be given a previous
migration that removed the fields, then an error is returned.
2023-02-16 20:47:35 +02:00
Elle Mouton
2c786ec66f
channeldb: add local and remote balance to revocation log
This commit re-adds the LocalBalance and RemoteBalance fields to the
RevocationLog. The channeldb/migration30 is also adjusted so that anyone
who has not yet run the optional migration will not lose these fields if
they run the migration after this commit.

The reason for re-adding these fields is that they are needed if we want
to reconstruct all the info of the lnwallet.BreachRetribution without
having access to the breach spend transaction. In most cases we would
have access to the spend tx since we would see it on-chain at which time
we would want to reconstruct the retribution info. However, for the
watchtower subsystem, we sometimes want to construct the retribution
info withouth having access to the spend transaction.

A user can use the `--no-rev-log-amt-data` flag to opt-out of storing
these amount fields.
2023-02-16 20:47:35 +02:00
Elle Mouton
0730337cc7
multi: add new NoRevLogAmtData config option
In this commit, a new `--db.no-rev-log-amt-data` flag is added. The
config option is passed though to everywhere that it will be used. Note
that it is still a no-op in this commit. An upcoming commit will make
use of the flag.
2023-02-16 20:47:29 +02:00
Elle Mouton
9c01916bc0
channeldb: add NoAmountData field to MigrateRevLogConfigImpl
Add a NoAmountData field to the MigrateRevLogConfigImpl struct and set
it for tests. This field is still a no-op in the migration.
2023-02-16 20:46:05 +02:00
Elle Mouton
70e3f8f511
multi: pass in a MigrationConfig to all optional migrations
Define a MigrationConfig interface that should be used to pass the
config of an optional migration function. An implementation of this
interface is added for migration30 (the only current optional
migration).
2023-02-16 20:45:08 +02:00
Slyghtning
9e8ee618e3 lncli: fix line length issues 2023-02-16 19:41:30 +01:00
Elle Mouton
33b7b9221a
channeldb+wtdb: fix migration function name lint
Rename the ApplyMigrationWithDb function to ApplyMigrationWithDB to make
the linter happy.
2023-02-16 07:51:49 +02:00
Elle Mouton
ce8e7ecfa7
channeldb: prepare RevocationLog for optional fields
This commit is a pure refactor. It restructures the RevocationLog
serialize and deserialize methods so that optional TLV fields can be
easily added.
2023-02-16 07:51:48 +02:00
Oliver Gugger
38dc67e1ef
Merge pull request #6815 from bitromortac/2205-bimodal
pathfinding: probability for bimodal distribution
2023-02-15 14:19:32 +01:00
Carla Kirk-Cohen
3cc8e81121
lncli/bugfix: use int64slice for update node announcement features
As is, we'll never get any add/remove features because the flag is
set as an int 64 slice but accessed as an int slice (which has no
value, and the cli doesn't fail if the flag doesn't exist).
2023-02-14 15:53:56 -05:00
bitromortac
d78e756406
routing: add probability benchmarks
$ go test -bench=PairProbability ./routing
goos: linux
goarch: amd64
pkg: github.com/lightningnetwork/lnd/routing
cpu: AMD Ryzen 5 3600 6-Core Processor
BenchmarkBimodalPairProbability-4   	 2050206	       599.8 ns/op
BenchmarkAprioriPairProbability-4   	 9524289	       126.7 ns/op

* math.Exp:
goos: linux
goarch: amd64
pkg: github.com/lightningnetwork/lnd/routing
cpu: AMD Ryzen 5 3600 6-Core Processor
BenchmarkExp-4   	143843622	         8.700 ns/op

* bimodal benchmark profile:
Showing nodes accounting for 2000ms, 76.05% of 2630ms total
Dropped 156 nodes (cum <= 13.15ms)
Showing top 10 nodes out of 141
      flat  flat%   sum%        cum   cum%
    1110ms 42.21% 42.21%     1110ms 42.21%  math.archExp
     230ms  8.75% 50.95%      230ms  8.75%  runtime.memclrNoHeapPointers
     180ms  6.84% 57.79%      930ms 35.36%  github.com/lightningnetwork/lnd/routing.(*BimodalEstimator).calculateProbability
     110ms  4.18% 61.98%      130ms  4.94%  runtime.scanobject
      90ms  3.42% 65.40%       90ms  3.42%  memeqbody
      60ms  2.28% 67.68%      940ms 35.74%  github.com/lightningnetwork/lnd/routing.(*BimodalEstimator).directProbability
      60ms  2.28% 69.96%      230ms  8.75%  github.com/lightningnetwork/lnd/routing.cannotSend
      60ms  2.28% 72.24%       60ms  2.28%  runtime.madvise
      50ms  1.90% 74.14%      480ms 18.25%  github.com/lightningnetwork/lnd/routing.(*BimodalEstimator).primitive
      50ms  1.90% 76.05%       60ms  2.28%  runtime.mapiternext

* apriori benchmark profile:
Showing nodes accounting for 1570ms, 77.34% of 2030ms total
Dropped 138 nodes (cum <= 10.15ms)
Showing top 10 nodes out of 151
      flat  flat%   sum%        cum   cum%
     340ms 16.75% 16.75%      340ms 16.75%  math.archExp
     290ms 14.29% 31.03%      980ms 48.28%  github.com/lightningnetwork/lnd/routing.(*AprioriEstimator).getNodeProbability
     190ms  9.36% 40.39%      260ms 12.81%  runtime.mapiternext
     190ms  9.36% 49.75%      190ms  9.36%  runtime.memclrNoHeapPointers
     160ms  7.88% 57.64%      340ms 16.75%  github.com/lightningnetwork/lnd/routing.(*AprioriEstimator).calculateProbability
     110ms  5.42% 63.05%      110ms  5.42%  aeshashbody
     110ms  5.42% 68.47%      130ms  6.40%  runtime.scanobject
      80ms  3.94% 72.41%      420ms 20.69%  github.com/lightningnetwork/lnd/routing.capacityFactor
      60ms  2.96% 75.37%       60ms  2.96%  runtime.madvise
      40ms  1.97% 77.34%       40ms  1.97%  runtime.isEmpty (inline)
2023-02-14 13:34:25 +01:00
bitromortac
2ccdfb1151
lncli+routerrpc: adapt mc api to dynamic estimator
The active probability estimator can be switched dynamically using the
`Set/GetMissionControl` API, maintaining backward compatibility. The
lncli commands `setmccfg` and `getmccfg` are updated around this
functionality. Note that deprecated configuration parameters were
removed from the commands.
2023-02-14 13:34:25 +01:00