Commit Graph

780 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
67e18e6089
rpc: optimize DescribeGraph by caching the response
Unfortunately, we can't use the graph cache directly here as it doesn't
include all the information we need, since it only includes the minimal
amount of information needed for path finding.

Instead, we use a simple mutex guarded cache that evicts itself after a
certain interval. The default is set small enough that most users
shouldn't really notice. We also provide a way to disable the cache, and
also disable the cache in the itests.

Fixes https://github.com/lightningnetwork/lnd/issues/1232
2021-11-04 15:30:31 -07:00
Oliver Gugger
eb2be7c7c0
lntest: count unconfirmed balance as well
It looks like in some cases (apparently mostly on Windows) a previous
sub test sometimes leaves some unconfirmed balance in the wallet. That
balance is confirmed in the next test when new coins are sent to the
wallet.
2021-11-02 20:36:45 +01:00
Oliver Gugger
2a6f128a43
itest: wait for channel to be properly closed
It looks like in some cases a channel is still being closed while we
already try to create a sweep output. In that case the pending closed
channel is still counted as anchor channel and a reserve output is
created. To make sure that doesn't happen, we make sure there are no
pending or open channels of any kind around before we create the sweep
transaction.
2021-11-02 20:36:27 +01:00
Oliver Gugger
2ca3eea3f7
itest: fix REST WebSocket ping/pong itest
This commit fixes a flake in the REST itest that was caused by an
invoice being created before the actual subscription client was
registered.
2021-11-02 20:36:26 +01:00
Olaoluwa Osuntokun
66ca2a994b
Merge pull request #5901 from Roasbeef/dedup-interceptor
Dedup interceptor
2021-10-28 17:44:49 -07:00
Roei Erez
fe44aedb25
itest: dedup interceptor packets
Aded a test for the dedup intercepted packets of htlc interceptor.
2021-10-28 16:16:55 -07:00
Olaoluwa Osuntokun
a5ae7fbf9c
lntest: add new itest for repeated payments to the same AMP invoice 2021-10-28 15:50:35 -07:00
Olaoluwa Osuntokun
0a3bc3ee3d
Merge pull request #5840 from guggero/bolt-pathfinding-fallback
db: allow turning off in-memory graph cache for bbolt
2021-10-25 14:24:10 -07:00
Oliver Gugger
27632daa98
itest: test path finding without cache 2021-10-22 09:32:50 +02:00
Oliver Gugger
423de4d79a
itest: defer close of done channel
The testing framework uses runtime.Goexit() when a test fails which
still allows the defer calls to execute. That's why we should use defer
to close the done channel to allow all goroutines to exit properly.
2021-10-21 11:09:52 +02:00
Wilmer Paulino
e891dd0fd3
itest: test backup restore of script enforced lease channel type 2021-10-19 18:30:53 -07:00
Wilmer Paulino
974fc346cf
itest: update itests to use new script-enforced lease commitment type
We update several of our integration tests that exercise different
scenarios with regards to the broadcast of a channel commitment
transaction with HTLCs in-flight to use the new commitment type for
channel leases. We do this to ensure we have complete coverage of said
channel commitment type. This required changing several assumptions
throughout the tests based on when we should expect sweeps to happen.
2021-10-19 18:30:48 -07:00
Olaoluwa Osuntokun
94b7b71809
Merge pull request #5689 from guggero/remote-signer
Support remote signing over RPC
2021-10-18 18:39:17 -07:00
Oliver Gugger
a6ea019f56
lntest: add itest for remote signing 2021-10-14 16:14:49 +02:00
Oliver Gugger
36dc4b3abf
itest: refactor tests to be re-used 2021-10-14 15:42:59 +02:00
Oliver Gugger
a3addcc927
multi: forward SendCoins call over RPC 2021-10-14 15:42:52 +02:00
Oliver Gugger
6093393e2f
multi: refactor SignMessage to specify hashing 2021-10-14 15:42:44 +02:00
Oliver Gugger
5d7e814ea8
Merge pull request #5674 from torkelrogstad/2021-08-26-http-mux
lntest: avoid global ServeMux
2021-10-13 10:45:57 +02:00
Oliver Gugger
e79d59dd4c
multi: use key locator for lnwallet.MessageSigner
To simplify the message signing API even further, we refactor the
lnwallet.MessageSigner interface to use a key locator instead of the
public key to identify which key should be signed with.
2021-10-08 12:06:52 +02:00
Oliver Gugger
afa03f22cc
multi: use key locator only for signing messages
To simplify the API surface of a remote signer even more, we refactor
the SignMessage and SignMessageCompact calls to only accept a key
locator as we always know what key we're using for signing anyway.
2021-10-08 12:06:52 +02:00
Oliver Gugger
8b7c88537c
multi: refactor SignDigestCompact into SignMessageCompact
To make it possible to use a remote lnrpc server as a signer for our
wallet, we need to change our main interface to sign the message instead
of the message's digest. Otherwise we'd need to alter the
lnrpc.SignMessage RPC to accept a digest instead of only the message
which has security implications.
2021-10-08 12:06:52 +02:00
Oliver Gugger
02757f6735
multi: refactor SignDigest into SignMessage
To make it possible to use a remote signrpc server as a signer for our
wallet, we need to change our main interface to sign the message instead
of the message's digest. Otherwise we'd need to alter the
signrpc.SignMessage RPC to accept a digest instead of only the message
which has security implications.
2021-10-08 12:06:51 +02:00
Oliver Gugger
0fac6c400e
itest: wait for node to fully start up 2021-10-06 16:29:49 +02:00
Oliver Gugger
e9c9ec2e84
itest: exit goroutines on error 2021-10-06 16:29:44 +02:00
Oliver Gugger
c89637a4e2
itest: fix close of closed channel panic 2021-10-05 20:48:48 +02:00
Oliver Gugger
6bc0862fdf
itest: fix commitment_deadline context expired flake 2021-10-05 20:48:48 +02:00
Oliver Gugger
bedba02a2d
itest: enable slow mining by default
We now redirect the mineBlocks function to the mineBlocksSlow function
which waits after each mined block. To reduce the overall time impact of
using that function everywhere, we only wait 20 milliseconds instead of 50ms
after each mined block to give all nodes
some time to process the block. This will still slow down everything by a bit
but reduce flakes that are caused by different sub systems not being
up-to-date.
2021-10-05 20:48:47 +02:00
Oliver Gugger
98061dfd58
mod+lntest: disable stall handler in btcd mining node
The latest version of btcd allows its stall handler to be disabled. We
use that new config option to make sure the mining btcd node and the lnd
chain backend btcd node aren't disconnected if some test takes too long
and no new p2p messages are exchanged.
2021-10-05 20:48:47 +02:00
Elle Mouton
da64a24ad8
server: update persistent peer addrs from announcements
In this commit, a subscription is made to topology updates. For any
NodeAnnouncements received for our peristent peers, we store their newly
advertised addresses. If at the time of receiving these new addresses
there are any existing connection requests, these are updated to reflect
the newly advertised addresses.
2021-10-04 14:57:50 +02:00
Elle Mouton
6b5b53d517
lntest: show reconnection bug
In this commit we demonstrate a bug to show that if an inbound peer
changes their listening address to one not advertised in their original
NodeAnnouncement then we will not be able to reconnect to them. This bug
will be fixed in a follow-up commit.
2021-10-04 14:57:50 +02:00
Elle Mouton
d639a4d73f
server+lntest: use all addrs during reconnect
In this commit, all advertised addresses of a peer are used during
reconnection. This fixes a bug previously demonstrated in an itest.
2021-10-04 14:57:50 +02:00
Elle Mouton
18909e1a8e
lntest: show that multi addresses are not used
This commit adds an itest to demonstrate that if a peer advertises
multiplie external IP addresses, then they will not all be used to
reconnect to the peer during reconnection. This will be fixed in a
follow-up commit.
2021-10-04 14:57:49 +02:00
Elle Mouton
5177ec2f84
lntest: create persistent connection
This commit adds a ConnectNodesPerm function to the itest NetworkHarness
so that persistent connections between nodes can be mocked.
2021-10-04 14:57:49 +02:00
Elle Mouton
9a97577c00
lntest: refactor assertNumConnections
The assertNumConnection function currently takes in an 'expected' number
of connections argument and asserts that both alice and bob only each
only have that number of connections. So this fails to be useful if say
alice is also connected to charlie cause then if we call
assertNumConnections between alice and bob it will fail saying there are
2 connections between them since all it does is count alice's total
  number of connections. This commit replaces this function with 2 new
functions: assertConnected which asserts that at least one connection
exists between two peers and assertNotConnected which asserts that no
connections exists between the two peers.
2021-10-04 14:57:48 +02:00
Oliver Gugger
51d19dad87
Merge pull request #5363 from guggero/psbt-no-final-tx
Allow skipping `PsbtFinalize` step during channel funding to support external broadcast
2021-10-04 12:37:51 +02:00
Oliver Gugger
692ea25295
Merge pull request #5642 from guggero/in-memory-graph
In-memory graph cache for faster pathfinding
2021-10-04 11:20:23 +02:00
Oliver Gugger
57253c0c05
itest: add test for fully external PSBT funding 2021-10-04 11:17:08 +02:00
Olaoluwa Osuntokun
32fa48df7d
Merge pull request #5770 from Crypt-iQ/dust_threshold_0619
lnwallet+htlcswitch: make Switch dust-aware
2021-09-30 20:20:19 -07:00
eugene
702b3a3258
multi: introduce config-level DustThreshold for defining threshold 2021-09-30 13:44:35 -04:00
Oliver Gugger
0b4e03f5fc
multi: add golang 1.17 compatible build tags
With go 1.17 a change to the build flags was implemented:
https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md

The formatter now automatically adds the forward-compatible build tag
format and the linter checks for them, so we need to include them in our
code.
2021-09-29 17:31:37 -07:00
Oliver Gugger
7aa9661d42
lntest: use errgroup for node startup
This commit fixes a nil pointer issue when a node fails to start up.
Because require.NoErrorf() doesn't abort a test immediately if run
inside a goroutine, this lead to the test continuing with nil node
references which lead to a panic later on.
2021-09-29 17:31:32 -07:00
Olaoluwa Osuntokun
dd4cf268a0
Merge pull request #5804 from guggero/itest-setup-error
lntest: use errgroup for node startup [skip ci]
2021-09-29 17:31:17 -07:00
eugene
fdcd726f9a
multi: replace DefaultDustLimit with script-specific DustLimitForSize
This commit updates call-sites to use the proper dust limits for
various script types. This also updates the default dust limit used
in the funding flow to be 354 satoshis instead of 573 satoshis.
2021-09-29 13:33:10 -04:00
Oliver Gugger
493262e253
itest: fix flake in update_channel_status itest
This commit fixes a flake in the channel status update itest that
occurred if Carol got a channel edge update for a channel before it
heard of the channel in the first place. To avoid that, we wait for
Carol to sync her graph before sending out channel edge or policy
updates.

As always when we touch itest code, we bring the formatting and use of
the require library up to date.
2021-09-29 17:50:01 +02:00
Oliver Gugger
5aeb728f80
lntest: use errgroup for node startup
This commit fixes a nil pointer issue when a node fails to start up.
Because require.NoErrorf() doesn't abort a test immediately if run
inside a goroutine, this lead to the test continuing with nil node
references which lead to a panic later on.
2021-09-29 11:57:53 +02:00
yyforyongyu
c525c3d34e
itest: test wipe forwarind packages 2021-09-23 13:24:11 +08:00
Joost Jager
daeb96fe0a
postgres: add itest 2021-09-21 10:44:23 +02:00
Olaoluwa Osuntokun
9264185f5b
Merge pull request #5101 from guggero/macaroon-interceptor
Add macaroon based RPC middleware interceptor
2021-09-20 19:15:04 -07:00
Oliver Gugger
4b43e977b2
lntest: add RPC middleware itests 2021-09-20 17:04:39 +02:00
Oliver Gugger
0cc260bfc7
itest: add _test suffix to test file 2021-09-20 17:04:38 +02:00
Oliver Gugger
045765111a
multi: use safe copy for macaroons
Fixes #4383 by adding a new SafeCopyMacaroon function that correctly
clones all caveats and prevents modifications on the copy from affecting
the original.
2021-09-20 13:05:46 +02:00
Oliver Gugger
4b7452a35e
lnrpc+itest: fix write deadline issue with WS ping
Fixes #5680.
To make sure we're always reading from the WebSocket connection, we need
to always have an ongoing (but blocking) conn.ReadMessage() call going
on. To achieve this, we do the read in a separate goroutine and write to
a buffered channel. That way we can always read the next message while
the current one is being forwarded. This allows incoming ping messages
to be received and processed which then leads to the deadlines to be
extended correctly.
2021-09-20 12:21:03 +02:00
yyforyongyu
66dae6ecf7
itest: put node.CloseChannel inside wait 2021-09-17 07:50:44 +08:00
yyforyongyu
7038d0e5c8
itest: fix typo 2021-09-17 07:50:43 +08:00
yyforyongyu
cdec34c5f7
itest: remove the method waitForChannelUpdate
This commit removes the method waitForChannelUpdate, and uses
node.WaitForChannelPolicyUpdate instead.
2021-09-17 07:50:43 +08:00
yyforyongyu
06fa17513c
itest: move tests by their category 2021-09-17 07:50:43 +08:00
yyforyongyu
87c13d31b4
itest: watch channel policy updates in harness node 2021-09-17 07:50:43 +08:00
yyforyongyu
d2277ac915
itest: replace chanOpen bool with chanWatchType 2021-09-17 07:50:42 +08:00
yyforyongyu
a58543d1c7
itest: remove extra graph topology subscription 2021-09-17 07:50:42 +08:00
yyforyongyu
92cd6657c5
lntest: refactor handle close channel update 2021-09-17 07:50:42 +08:00
yyforyongyu
0701834a5d
lntest: refactor handle update open channel 2021-09-17 07:50:42 +08:00
yyforyongyu
a1024163fe
itest: add more verbose log and print node state 2021-09-17 07:50:42 +08:00
Turtle
0120c86687
itest: Test new RPC call for checking macaroon permissions 2021-09-14 16:27:02 -05:00
Turtle
ba17842e34
itest: Test new BakeMacaroonRequest for allowing external permissions 2021-09-14 16:27:02 -05:00
Joost Jager
3827d15ca6 lntest/itest: fix SendPaymentAMP test 2021-09-14 18:53:17 +02:00
yyforyongyu
c4913e6f4a
itest: require server being started when creating node
We now require the lnd to be fully started when creating a new node
using newNode.
2021-09-14 07:34:10 +08:00
Andras Banki-Horvath
9bf04f9870
itest+etcd: save etcd logs along the node logs 2021-09-10 14:40:57 +02:00
Andras Banki-Horvath
6c2d8bb176
etcd: enable optional log file for embedded etcd log output
In this commit we add an extra config for enabling logging to an
external file when using embedded etcd. This can be useful when running
integration tests to see more details about etcd related issues.
2021-09-10 14:40:54 +02:00
Oliver Gugger
7a9ee48cf1
itest: add batch funding test 2021-09-03 18:39:45 +02:00
Wilmer Paulino
523eef5cf4
lntest: use explicit channel commitment negotiation for multi-hop itests 2021-08-30 19:17:42 -07:00
Wilmer Paulino
449f207217
lntest: replace commitType type with rpc alternative 2021-08-30 19:17:39 -07:00
Torkel Rogstad
d6045a0fd5 lntest: avoid global ServeMux
Using the default, global ServeMux prevents the same process from
calling `lntest.NewNetworkHarness` multiple times, because we get a
panic when registering HTTP routes.

Instead, we use the ServeMux beloning to the fee service struct.
2021-08-27 11:30:01 +02:00
Oliver Gugger
23cd231483
Merge pull request #4717 from guggero/xprv-wallet-init
walletunlocker: Allow wallet to be created from extended master root key (xprv)
2021-08-24 13:01:41 +02:00
Oliver Gugger
c74c46147f
Merge pull request #5646 from yyforyongyu/fix-context-leak
itest: properly handle the creation of timeout context
2021-08-24 10:44:36 +02:00
Oliver Gugger
0822573743
itest: test wallet recovery from extended master root key
We add an additional test case to the on-chain fund recovery test that
tries restoring the same wallet from the extended master root key
instead of the seed.
2021-08-24 10:37:34 +02:00
Oliver Gugger
3fd944e7e4
lntest: allow node restore by extended root key
To allow testing restoring a node from an extended master root key, we
add an extra argument to the RestoreNodeWithSeed function.
2021-08-24 10:37:33 +02:00
yyforyongyu
104b7a09db
itest: fix inheritance when creating timeout ctxt
This commit fixes the issue where a wrong context being inherited to
create a timeout context. When a parent context timed out, all its
children contexts also timed out, even the children contexts had a
larger timeout value. This means it only makes sense to inherite from a
parent when its children have smaller timeout value. Given the setup of
the itest, all the timeout contexts need to be created from a context
background(hence no timeout on the parent) unless there's an explicit
timeout bound we want to set.
2021-08-24 12:28:40 +08:00
yyforyongyu
edffd65e92
itest: manage context timeout in utils.go
This commit finishes moving the context management into utils.go.
2021-08-24 12:28:40 +08:00
yyforyongyu
a6c5255e77
itest: manage context timeout inside completePaymentRequests 2021-08-24 12:28:39 +08:00
yyforyongyu
5a94919b7e
itest: manage context timeout inside network harness
This is the last commit to refactor the context management such that all
the timeout are managed inside the network harness.
2021-08-24 12:28:35 +08:00
yyforyongyu
d9e9238b0c
itest: make WaitForTxInMempool a private method 2021-08-24 12:16:46 +08:00
yyforyongyu
16403da91e
itest: manage context in CloseChannel and AssertChannelExists 2021-08-24 12:16:46 +08:00
yyforyongyu
3e26d77f91
itest: manage context inside WaitForChannelOpen and WaitForChannelClose 2021-08-24 12:16:46 +08:00
yyforyongyu
1629858a3d
itest: manage context inside assertions - II
This commit continues moving context management into assertions.
2021-08-24 12:16:40 +08:00
yyforyongyu
02e4c3ad4c
itest: manage context inside assertions - I
This commit changes the methods assertTxLabel, assertReports,
assertSweepFound, and sendAndAssertSuccess to manage their own context
with deadline.
2021-08-24 11:17:20 +08:00
yyforyongyu
74f8fe482d
itest: manage context inside EnsureConnected 2021-08-24 11:06:44 +08:00
yyforyongyu
d10d1e3e24
itest: manage context inside SendCoins 2021-08-24 11:06:44 +08:00
yyforyongyu
8de495b96b
itest: initialize context inside ConnectNodes 2021-08-24 11:06:43 +08:00
yyforyongyu
dca20d80a7
itest: fix context leak in openChannelAndAssert
In this commit, we put the context initialization inside
openChannelAndAssert, thus saving us a few lines and making sure the
context is always properly handled.
2021-08-24 11:06:43 +08:00
ErikEk
ca7192f8fd itest: include compressed btcd backend logs 2021-08-24 02:00:44 +02:00
de6df1re
29157fe02a itest: fix payment test 2021-08-20 10:01:45 -05:00
yyforyongyu
fda66f6a43
itest: add an extra utxo when using neutrino 2021-08-19 19:35:20 +08:00
yyforyongyu
24e654ffc4
contractcourt: more verbose logging for anchor sweeping 2021-08-19 19:33:36 +08:00
Joost Jager
09386213be
htlcswitch: log link message with channel point
In lnd, log messages about channels are generally logged with a
reference to their channel point rather than the short channel id.
Channel point is reorg-resistant and also easier to look up in for
example a block explorer.

In the link however, all log messages are accompanied by short channel
id. This makes it difficult to grep a log for all channel activity. The
PEER message for example which are often crucial to analyse, are logged
with channel points.

This commit modifies the link logging to also use channel points.
2021-08-16 08:24:35 +02:00
Stevie Zollo
00f7534e06
docs: fix inconsistent naming of channel.backup [skip ci] 2021-08-12 00:26:53 +02:00
yyforyongyu
8eddc50beb
itest: remove context creations outside close channel assertion 2021-08-11 12:44:39 +08:00
yyforyongyu
2e4417454b
itest: create context with timeout in closeChannelAndAssertType 2021-08-11 12:41:57 +08:00
Olaoluwa Osuntokun
be341bcd68
Merge pull request #5528 from guggero/keep-channel-backup-local-force-close
backups: Keep channel backup in case of local force close
2021-08-10 19:31:59 -07:00
Oliver Gugger
f7943448b9
itest: fix channel open and close ntfn test
Because we now get one more update per channel when closing it, we need
to update our test that looks at the close notifications sent by the
SubscribeChannelEvent RPC.
2021-08-09 16:06:56 +02:00
Oliver Gugger
bb4c754504
itest: test local force close restore scenario
We want to make sure we can recover funds from a channel that was force
closed by the local node just before the node needed to be restored. We
add a special test case with specific assertions for that scenario.
2021-08-09 13:55:37 +02:00