This commit deletes the function `BalancesAtHeight` since its only
usague is to find the push amount, which can be achieved by saving the
initial balances.
Another reason to remove it is to pave the way to incooperate our new
revocation log. If we ever need this function again, we can add it back
by visiting all the revocation logs to calculate the balances at a given
height.
Fixes#6396.
This commit fixes a panic that occurred when trying to sign for a
Taproot output without specifying the full UTXO information for each
input. Instead of panicking an error is now returned.
Because Taproot key spend only spends don't allow us to re-construct the
spent pkScript from the witness alone, we cannot support registering
spend notifications for v1 pkScripts only. We instead require the
outpoint to be specified. This commit makes it possible to only match by
outpoint and also adds an itest for it.
Fixes an issue with SignOutputRaw in remote signing mode where we
weren't able to sign on the remote signer if we only provided the public
key or only the family/index (and not both).
Fixes part of an issue detected in lightninglabs/loop#457.
We need to be able to query the watch-only wallet about a public key
when trying to sign with a key that we don't know the family or index
of. The easiest way to do that is to leverage the wallet's address index
to query the derivation path for a public key.
To give the RPC wallet access to that functionality, we need to expose
the method on the WalletController interface.
In this commit, we add a new integration tests to exercise the fix
introduced in the prior commit. In this test, we reconstruct a scenario
for a 3rd party to sweep an anchor spend after force closing, causing a
prior spend we had to be invalidated. Without the prior commit, this test
fails as the original anchor sweep is still found in the wallet.
Before this commit, we we were trying to sweep an anchor output, and
that output was spent by someone else (not the sweeper), then we would
report this back to the original resolver (allowing it to be cleaned
up), and also remove the set of inputs spent by that transaction from
the set we need to sweep.
However, it's possible that if a user is spending unconfirmed outputs,
then the wallet is holding onto an invalid transaction, as the outputs
that were used as inputs have been double spent elsewhere.
In this commit, we fix this issue by recursively removing all descendant
transactions of our past sweeps that have an intersecting input set as
the spending transaction. In cases where a user spent an unconfirmed
output to funding a channel, and that output was a descendant of the now
swept anchor output, the funds will now properly be marked as available.
Fixes#6241
This commit was previously split into the following parts to ease
review:
- 2d746f68: replace imports
- 4008f0fd: use ecdsa.Signature
- 849e33d1: remove btcec.S256()
- b8f6ebbd: use v2 library correctly
- fa80bca9: bump go modules
Update our test to assert that we have hop hints present when we
expect them, and fix the "alice is a private node" issue that was
previously preventing us from adding hop hints. Asserting that we
have hop hints present in this itest ensures that we'll fail our
itests if a change to SelectHopHints results in our no longer
having hints in this secnario.
If we only set the --datadir flag then lnd will create empty directories
(for letsencrypt and watchtower client) in the default user's home .lnd
directory and will also pick up a config file from there if one exists.
To instruct lnd to only act within the temp itest directory, we need to
use the --lnddir flag instead.
Using the --lnddir flag also overwrites the TLS cert/key path and the
log dir, so we don't need to explicitly set those anymore.
But we need to make sure the node's variables point to the correct
default files/directories.
This sets the `CustomCaveatCondition` value on rpc middleware requests
if one exists. Previously, this value was always blank even if the
macaroon had a value set for its custom caveat condition.
This fixeslightninglabs/loop#437 by adding all accounts that are used
in liquidity products such as Loop or Pool. Since both of these products
use key families below 255, we can get by with that number.
The alternative to creating way too many accounts (which increases the
default wallet size by ~250kB) would be to hard code the exact accounts
used by Loop (99) and Pool (210). But that sounds like a bad idea given
that there could always be more accounts being added to those (or other)
products. By making sure the first 255 accounts exist, we have a lot
more flexibility in those products for choosing key families.
With the remote signing instance now not needing to know anything about
addresses or current derivation indices, we don't need to forward any
such calls to that instance and can simplify the RPCKeyRing
considerably.
This commit adds a new component, harness miner, to the itest. This
newly added component is responsible for checking the mempool and blocks
for the itest.
This commit adds a new struct RPCClients to better handle rpc clients.
A private field, rpc, is added to HarnessNode to prevent direct access
to its clients. Inside RPCClients, all clients are exported in case a
test case need access to a specific client.
This adds an integration test that makes sure channel can be funded from
empty wallet using PSBT if the funding transaction contains an output
belonging to the wallet, satisfying the reserve.
This commit removes the context as a param needed when calling methods
of HarnessNode. This change moves the context management inside
HarnessNode, aside from saving us a few lines, it makes the context
creation/timeout less error-prone.
This commit adds a running context to HarnessNode which replaces all the
background context used and also serves as a way to signal quit when the
test is shutting down.
This commit adds a unique request ID that is the same for each gRPC
request and response intercept message or each request/response message
of a gRPC stream.
This commit adds a new unique intercept message ID that is different for
each message sent to an RPC middleware. This will allow us to re-purpose
the existing request ID field into its original intention, which was to
have a unique ID for a full gRPC request/response pair or gRPC stream as
a whole.
Unfortunately, we can't use the graph cache directly here as it doesn't
include all the information we need, since it only includes the minimal
amount of information needed for path finding.
Instead, we use a simple mutex guarded cache that evicts itself after a
certain interval. The default is set small enough that most users
shouldn't really notice. We also provide a way to disable the cache, and
also disable the cache in the itests.
Fixes https://github.com/lightningnetwork/lnd/issues/1232
It looks like in some cases (apparently mostly on Windows) a previous
sub test sometimes leaves some unconfirmed balance in the wallet. That
balance is confirmed in the next test when new coins are sent to the
wallet.
It looks like in some cases a channel is still being closed while we
already try to create a sweep output. In that case the pending closed
channel is still counted as anchor channel and a reserve output is
created. To make sure that doesn't happen, we make sure there are no
pending or open channels of any kind around before we create the sweep
transaction.
The testing framework uses runtime.Goexit() when a test fails which
still allows the defer calls to execute. That's why we should use defer
to close the done channel to allow all goroutines to exit properly.
We update several of our integration tests that exercise different
scenarios with regards to the broadcast of a channel commitment
transaction with HTLCs in-flight to use the new commitment type for
channel leases. We do this to ensure we have complete coverage of said
channel commitment type. This required changing several assumptions
throughout the tests based on when we should expect sweeps to happen.
To simplify the message signing API even further, we refactor the
lnwallet.MessageSigner interface to use a key locator instead of the
public key to identify which key should be signed with.
To simplify the API surface of a remote signer even more, we refactor
the SignMessage and SignMessageCompact calls to only accept a key
locator as we always know what key we're using for signing anyway.
To make it possible to use a remote lnrpc server as a signer for our
wallet, we need to change our main interface to sign the message instead
of the message's digest. Otherwise we'd need to alter the
lnrpc.SignMessage RPC to accept a digest instead of only the message
which has security implications.
To make it possible to use a remote signrpc server as a signer for our
wallet, we need to change our main interface to sign the message instead
of the message's digest. Otherwise we'd need to alter the
signrpc.SignMessage RPC to accept a digest instead of only the message
which has security implications.
We now redirect the mineBlocks function to the mineBlocksSlow function
which waits after each mined block. To reduce the overall time impact of
using that function everywhere, we only wait 20 milliseconds instead of 50ms
after each mined block to give all nodes
some time to process the block. This will still slow down everything by a bit
but reduce flakes that are caused by different sub systems not being
up-to-date.
The latest version of btcd allows its stall handler to be disabled. We
use that new config option to make sure the mining btcd node and the lnd
chain backend btcd node aren't disconnected if some test takes too long
and no new p2p messages are exchanged.
In this commit, a subscription is made to topology updates. For any
NodeAnnouncements received for our peristent peers, we store their newly
advertised addresses. If at the time of receiving these new addresses
there are any existing connection requests, these are updated to reflect
the newly advertised addresses.
In this commit we demonstrate a bug to show that if an inbound peer
changes their listening address to one not advertised in their original
NodeAnnouncement then we will not be able to reconnect to them. This bug
will be fixed in a follow-up commit.
This commit adds an itest to demonstrate that if a peer advertises
multiplie external IP addresses, then they will not all be used to
reconnect to the peer during reconnection. This will be fixed in a
follow-up commit.
The assertNumConnection function currently takes in an 'expected' number
of connections argument and asserts that both alice and bob only each
only have that number of connections. So this fails to be useful if say
alice is also connected to charlie cause then if we call
assertNumConnections between alice and bob it will fail saying there are
2 connections between them since all it does is count alice's total
number of connections. This commit replaces this function with 2 new
functions: assertConnected which asserts that at least one connection
exists between two peers and assertNotConnected which asserts that no
connections exists between the two peers.
With go 1.17 a change to the build flags was implemented:
https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
The formatter now automatically adds the forward-compatible build tag
format and the linter checks for them, so we need to include them in our
code.
This commit fixes a nil pointer issue when a node fails to start up.
Because require.NoErrorf() doesn't abort a test immediately if run
inside a goroutine, this lead to the test continuing with nil node
references which lead to a panic later on.
This commit updates call-sites to use the proper dust limits for
various script types. This also updates the default dust limit used
in the funding flow to be 354 satoshis instead of 573 satoshis.
This commit fixes a flake in the channel status update itest that
occurred if Carol got a channel edge update for a channel before it
heard of the channel in the first place. To avoid that, we wait for
Carol to sync her graph before sending out channel edge or policy
updates.
As always when we touch itest code, we bring the formatting and use of
the require library up to date.
This commit fixes a nil pointer issue when a node fails to start up.
Because require.NoErrorf() doesn't abort a test immediately if run
inside a goroutine, this lead to the test continuing with nil node
references which lead to a panic later on.
Fixes#4383 by adding a new SafeCopyMacaroon function that correctly
clones all caveats and prevents modifications on the copy from affecting
the original.
Fixes#5680.
To make sure we're always reading from the WebSocket connection, we need
to always have an ongoing (but blocking) conn.ReadMessage() call going
on. To achieve this, we do the read in a separate goroutine and write to
a buffered channel. That way we can always read the next message while
the current one is being forwarded. This allows incoming ping messages
to be received and processed which then leads to the deadlines to be
extended correctly.
In this commit we add an extra config for enabling logging to an
external file when using embedded etcd. This can be useful when running
integration tests to see more details about etcd related issues.
Using the default, global ServeMux prevents the same process from
calling `lntest.NewNetworkHarness` multiple times, because we get a
panic when registering HTTP routes.
Instead, we use the ServeMux beloning to the fee service struct.
We add an additional test case to the on-chain fund recovery test that
tries restoring the same wallet from the extended master root key
instead of the seed.
This commit fixes the issue where a wrong context being inherited to
create a timeout context. When a parent context timed out, all its
children contexts also timed out, even the children contexts had a
larger timeout value. This means it only makes sense to inherite from a
parent when its children have smaller timeout value. Given the setup of
the itest, all the timeout contexts need to be created from a context
background(hence no timeout on the parent) unless there's an explicit
timeout bound we want to set.
In this commit, we put the context initialization inside
openChannelAndAssert, thus saving us a few lines and making sure the
context is always properly handled.
In lnd, log messages about channels are generally logged with a
reference to their channel point rather than the short channel id.
Channel point is reorg-resistant and also easier to look up in for
example a block explorer.
In the link however, all log messages are accompanied by short channel
id. This makes it difficult to grep a log for all channel activity. The
PEER message for example which are often crucial to analyse, are logged
with channel points.
This commit modifies the link logging to also use channel points.
Because we now get one more update per channel when closing it, we need
to update our test that looks at the close notifications sent by the
SubscribeChannelEvent RPC.
We want to make sure we can recover funds from a channel that was force
closed by the local node just before the node needed to be restored. We
add a special test case with specific assertions for that scenario.
With one more notification event being dispatched in the local force
close case we need to update one of the integration tests to account for
the additional message.
This commit uses the require library for the link nodes garbage collect
test and fixes a flake that was discovered while hunting for other
flakes. Some times the channel isn't updated fast enough so we can
detech it on first try. So we give it a few more tries to stabilize the
test.
This commit fixes an old flake in the neutrino anchor output tests. It
turns out that sometimes with Neutrino we don't have enough UTXOs in our
wallet to sweep both the local and remote anchor. This is very likely a
timing issue, we need to give the wallet more time to catch up with the
chain and process all transactions to find unspent outputs.
We address this two-fold: We add an additional UTXO to Alice. And then
we also make sure we detect both UTXOs properly after restarting.
From the error in the itest output log it is not clear whether scraping
the profile page caused a test to fail or whether it was just a
follow-up error. We make it a bit more clear with an added message.
To make it easier to see what tranche a failed test was running in, we
add the tranche index to the test name. This corresponds to the number
in the .logs-tranche<index> folder name so the logs can be found much
quicker.
Improve 'ErrReservedValueInvalidated' error string to explain that the
error is triggered by a transaction that would deplete funds reserved for
potential future anchor channel closings (via CPFP)
Add hint that further details can be found in the debug log
Update strings in 'lntest/itest/log_error_whitelist.txt' correspondingly
In this commit, we attempt to fix a race condition that may occur in the
current AMP and MPP tests.
It appears the following scenario is possible:
* The `mppTestContext` [is used to create 6 channels back to
back](https://github.com/lightningnetwork/lnd/blob/master/lntest/itest/lnd_amp_test.go#L43)
* The method used to create the channel ends up calling
[`openChannelAndAssert`](edd4152682/lntest/itest/lnd_mpp_test.go (L300))
which'll open the channel, mine 6 blocks, [then ensure that the
channel gets
advertised](edd4152682/lntest/itest/assertions.go (L78))
* Later on, [we wait for all nodes to hear about all channels on the
network
level](https://github.com/lightningnetwork/lnd/blob/master/lntest/itest/lnd_amp_test.go#L62)
I think the issue here is that we'll potentially already have mined 30
or so blocks before getting to the final nodes, and those nodes may have
already heard about the channel already. This may then cause their
[`lightningNetworkWatcher`](edd4152682/lntest/node.go (L1213))
goroutine to not properly dispatch this, since it's assumed that the
channel hasn't been announced (or notified) when the method is called.
One solution here is to just check if the channel is already in the
node's graph or not, when we go to register the notification. If we do
this in the same state machine as the watcher, then we ensure if the
channel is already known, the client is immediately notified. One thing
that can help us debug this more in the future is adding additional
logging in some of these helper goroutines so we can more easily track
the control flow.
This commit implements this solution by checking to ensure that the
channel isn't already known in our channel graph before attempting to
wait for its notification, as we may already have missed the
notification before this registration request came in.
This commit creates the file lnd_misc_test.go to hold all miscellaneous
tests in the file lnd_test.go. From now on, the lnd_test.go will only be
responsible for handling the "top" level functionalities such as
splitting test cases and run them. Newly created test cases should find
their places in the related test files, or create new one when needed.
This commit refactors the function assertNumConnections to use
wait.NoError. Prior to this commit, `make lint` will fail on this
function. While fixing it, it's noticed that wait.NoError suits the
case so it's refactored to use it.
In #5364 we added a new error path in the `StopDaemon` method to return
an error if shutdown was attempted while a rescan/recover instance was
in progress. Since the wallet actually won't fully stop (atm)
mid-recovery, the call effectively didn't do anything in that scenario,
so we started to return an error to properly reflect that. However this
causes certain itests to fail, as during recovery, the stop attempt will
fail leading to the test itself failing.
In this commit, we wrap the calls to stop a running daemon within a
`wait.NoError` call so we'll continually try to shut down the daemon
rather than quit on the first try.
Fixes#5423.
We do this to trigger a bug that will be resolved in a follow-up commit.
This bug prevents inbound legacy channels from being rejected by the
recipient if they have at least one anchor channel already opened
without an on-chain balance.
This commit refactored the function NewNode to take a *testing.T so that
the unexpected error is checked inside it. The caller is now free from
checking the errors.
This permits an AMP invoice to be "pseudo-reusable", where the invoice
paramters can be used multiple times so long as a new payment address is
supplied. This prevents additional round trips between payer and payee
to obtain a new invoice, even though the payments/invoices won't be
logically associated via the RPC interface like they would when the full
reusable invoices are deployed.
In some rare instances it can happen that the nodes don't find each
other again after one of them has been re-created and the other one has
been restarted in the SCB tests. By making sure the re-created has the
same P2P port again as before, we make sure they can connect to each
other again successfully for executing DLP.
Since there is a lot of connecting and disconnecting between nodes in
the channel backup tests, we try to speed up that process by lowering
the min backoff from 1 second to 50 milliseconds. We also make sure we
never wait more than 1 second if it does take multple attempts. This
should sum up and hopefully speed up our tests a bit.
It seems #5246 introduced a subtle bug that lead to the error "out of
order block: expecting height=1, got height=XXX" some times during
startup. Apparently it can happen that during pruning of the graph tip
some blocks can come in before we start our chain view and the new block
subscription. By querying the chain backend for the best height before
syncing with the graph we ensure that we never miss a block.
