With this change we allow adding hop hints when adding an invoice, even
if its amount is zero.
A couple of new unit test case have been added, and the
`testInvoiceRoutingHints` itest was expanded to account for this
scenario.
In this commit, we introduce two mining methods, `MineBlocks` and
`MineBlocksAndAssertNumTxes`, to `HarnessTest`. These methods are
different from the miner's methods as they would 1) mine the blocks
slowly and 2) assert all active nodes have synced. The miner's methods
are kept for future tests such as mining blocks really fast and see the
behavior of `lnd`.
This commit breaks the large `testChannelBackupRestore` into smaller
tests to take advantage of our parallel testing framework. A smaller
test also makes the debugging easier, hence better code health.
This commit breaks the large `testMultiHopHtlcClaims` into independent
test cases to take the advantage of parallel testing. The old setup will
put the stress of creating and syncing new blocks into a single core,
which causes a much longer test time and sometimes times out.
This commit sends the returned value of pred() inside a goroutine such
that when a predicate function never returns during the first run, the
wait package can timeout properly.
This commit adds a new flag `temptest` so we can run new and old tests
separately. This flag will be removed once the migration from old tests
to new tests is finished.
In this commit, we modify the way we compute the starting ideal fee for
the co-op close transaction. Before thsi commit, channel.CalcFee was
used, which'll compute the fee based on the commitment transaction
itself, rathern than the co-op close transaction. As the co-op close
transaction is potentailly bigger (two P2TR outputs) than the commitment
transaction, this can cause us to under estimate the fee, which can
result in the fee rate being too low to propagate.
To remedy this, we now compute a fee estimate from scratch, based on the
delivery fees of the two parties.
We also add a bug fix in the chancloser unit tests that wasn't caught
due to loop variable shadowing.
The wallet import itest has been updated as well, since we'll now pay
600 extra saothis to close the channel, since we're accounting for the
added weight of the P2TR outputs.
Fixes#6953
We add basic integration test coverage for importing Tapscript
addresses. Note that FundPsbt is not supported with those types of
imported keys, so we need to manually fill in all information in the
PSBT to be able to sweep the funds again.
With this change, transactions created via craftSweepTx will be
standard. Previously, p2wsh/p2pkh scripts passed in via SendCoins would
be weighted as p2wpkh scripts. With a feerate of 1 sat/vbyte,
transactions returned would be non-standard. Luckily, the critical
sweeper subsystem only used p2wpkh scripts so this only affected
callers from the rpcserver.
Also added is an integration test that fails if SendCoins manages
to generate a non-standard transaction. All script types are now
accounted for in getWeightEstimate, which now errors if an unknown
script type is passed in.
In this commit, we add a test case to exercise the fact that if we
request a confirmation notification with the raw block, then it includes
one. We add this to the existing test that uses the conf ntfn rpc call
for simplicity.
In this commit, we add a new option for the existing confirmation
notification system that optionally allows the caller to specify that a
block should be included as well.
The only quirk w/ the implementation here is the neutrino backend:
usually we get filtered blocks, we so need to first fetch the block
again so we can deliver the full block to the notifier. On the notifier
end, it'll only be checking for the transactions we care about, to
sending a full block doesn't affect the correctness.
We also extend the `testBatchConfirmationNotification` test to assert
that a block is only included if the caller specifies it.
In this commit, we change the flow of the rpc middleware registration
a bit. In order to allow a client to add rpc middleware interceptors in
a deterministic order, we now make the server send a "registration
complete" message to the client after compeleting the registration
process so that the client knows when it can go ahead and register the
next client.
Ensure the wallet has synced the blockchain before attempting to spend
funds.
Prior to this fix, I get the following error:
rpc error: code = Unknown desc = wallet couldn't fund PSBT: error
creating funding TX: insufficient funds available to construct
transaction
In this commit, we fix a detected flake: we go to make a payment, and
then want to assert that 2 attempts we required. The existing logic
assumes that the success and the payment will still be on disk. With a
recent PR, we now delete failed payments by default, but after the fact,
in a non-atomic fashion.
We fix this issue simply by having all the nodes keep around failed
payments for the sake of all the old itests that assumed this
information would always be on disk.
Introduced in: https://github.com/lightningnetwork/lnd/pull/6438.
Fixes https://github.com/lightningnetwork/lnd/issues/6711.
In this commit, we add a new field `TapTweak` to be used for key path
spends. Before this commit, we'd overload the existing `WitnessScript`
field to pass this information to the signing context. This was
confusing as for tapscript spends, this was the leaf script, which
mirrors the other script based spending types.
With this new filed, users need to set this to the script root for
keypath spends where the output key commits to a real merkle root, and
nothing when bip 86 spending is being used.
To make the signing even more explicit, we also add a new field called
sign_method with an enum type that differentiates between the different
segwit v0 and v1 signing methods.
Fixes https://github.com/lightningnetwork/lnd/issues/6446.
The inclusion proof field in the TapscriptPartialReveal function was
incorrect. An inclusion proof can be zero or more elements of 32-byte
slices. So an empty inclusion proof can be valid too for a tree that
only consists of a single leaf.
This commit deletes the function `BalancesAtHeight` since its only
usague is to find the push amount, which can be achieved by saving the
initial balances.
Another reason to remove it is to pave the way to incooperate our new
revocation log. If we ever need this function again, we can add it back
by visiting all the revocation logs to calculate the balances at a given
height.
Fixes#6396.
This commit fixes a panic that occurred when trying to sign for a
Taproot output without specifying the full UTXO information for each
input. Instead of panicking an error is now returned.
Because Taproot key spend only spends don't allow us to re-construct the
spent pkScript from the witness alone, we cannot support registering
spend notifications for v1 pkScripts only. We instead require the
outpoint to be specified. This commit makes it possible to only match by
outpoint and also adds an itest for it.
Fixes an issue with SignOutputRaw in remote signing mode where we
weren't able to sign on the remote signer if we only provided the public
key or only the family/index (and not both).
Fixes part of an issue detected in lightninglabs/loop#457.
We need to be able to query the watch-only wallet about a public key
when trying to sign with a key that we don't know the family or index
of. The easiest way to do that is to leverage the wallet's address index
to query the derivation path for a public key.
To give the RPC wallet access to that functionality, we need to expose
the method on the WalletController interface.
In this commit, we add a new integration tests to exercise the fix
introduced in the prior commit. In this test, we reconstruct a scenario
for a 3rd party to sweep an anchor spend after force closing, causing a
prior spend we had to be invalidated. Without the prior commit, this test
fails as the original anchor sweep is still found in the wallet.
Before this commit, we we were trying to sweep an anchor output, and
that output was spent by someone else (not the sweeper), then we would
report this back to the original resolver (allowing it to be cleaned
up), and also remove the set of inputs spent by that transaction from
the set we need to sweep.
However, it's possible that if a user is spending unconfirmed outputs,
then the wallet is holding onto an invalid transaction, as the outputs
that were used as inputs have been double spent elsewhere.
In this commit, we fix this issue by recursively removing all descendant
transactions of our past sweeps that have an intersecting input set as
the spending transaction. In cases where a user spent an unconfirmed
output to funding a channel, and that output was a descendant of the now
swept anchor output, the funds will now properly be marked as available.
Fixes#6241
This commit was previously split into the following parts to ease
review:
- 2d746f68: replace imports
- 4008f0fd: use ecdsa.Signature
- 849e33d1: remove btcec.S256()
- b8f6ebbd: use v2 library correctly
- fa80bca9: bump go modules
Update our test to assert that we have hop hints present when we
expect them, and fix the "alice is a private node" issue that was
previously preventing us from adding hop hints. Asserting that we
have hop hints present in this itest ensures that we'll fail our
itests if a change to SelectHopHints results in our no longer
having hints in this secnario.
If we only set the --datadir flag then lnd will create empty directories
(for letsencrypt and watchtower client) in the default user's home .lnd
directory and will also pick up a config file from there if one exists.
To instruct lnd to only act within the temp itest directory, we need to
use the --lnddir flag instead.
Using the --lnddir flag also overwrites the TLS cert/key path and the
log dir, so we don't need to explicitly set those anymore.
But we need to make sure the node's variables point to the correct
default files/directories.
This sets the `CustomCaveatCondition` value on rpc middleware requests
if one exists. Previously, this value was always blank even if the
macaroon had a value set for its custom caveat condition.
This fixeslightninglabs/loop#437 by adding all accounts that are used
in liquidity products such as Loop or Pool. Since both of these products
use key families below 255, we can get by with that number.
The alternative to creating way too many accounts (which increases the
default wallet size by ~250kB) would be to hard code the exact accounts
used by Loop (99) and Pool (210). But that sounds like a bad idea given
that there could always be more accounts being added to those (or other)
products. By making sure the first 255 accounts exist, we have a lot
more flexibility in those products for choosing key families.
With the remote signing instance now not needing to know anything about
addresses or current derivation indices, we don't need to forward any
such calls to that instance and can simplify the RPCKeyRing
considerably.
This commit adds a new component, harness miner, to the itest. This
newly added component is responsible for checking the mempool and blocks
for the itest.
This commit adds a new struct RPCClients to better handle rpc clients.
A private field, rpc, is added to HarnessNode to prevent direct access
to its clients. Inside RPCClients, all clients are exported in case a
test case need access to a specific client.
This adds an integration test that makes sure channel can be funded from
empty wallet using PSBT if the funding transaction contains an output
belonging to the wallet, satisfying the reserve.
This commit removes the context as a param needed when calling methods
of HarnessNode. This change moves the context management inside
HarnessNode, aside from saving us a few lines, it makes the context
creation/timeout less error-prone.
This commit adds a running context to HarnessNode which replaces all the
background context used and also serves as a way to signal quit when the
test is shutting down.
This commit adds a unique request ID that is the same for each gRPC
request and response intercept message or each request/response message
of a gRPC stream.
This commit adds a new unique intercept message ID that is different for
each message sent to an RPC middleware. This will allow us to re-purpose
the existing request ID field into its original intention, which was to
have a unique ID for a full gRPC request/response pair or gRPC stream as
a whole.
Unfortunately, we can't use the graph cache directly here as it doesn't
include all the information we need, since it only includes the minimal
amount of information needed for path finding.
Instead, we use a simple mutex guarded cache that evicts itself after a
certain interval. The default is set small enough that most users
shouldn't really notice. We also provide a way to disable the cache, and
also disable the cache in the itests.
Fixes https://github.com/lightningnetwork/lnd/issues/1232
It looks like in some cases (apparently mostly on Windows) a previous
sub test sometimes leaves some unconfirmed balance in the wallet. That
balance is confirmed in the next test when new coins are sent to the
wallet.
It looks like in some cases a channel is still being closed while we
already try to create a sweep output. In that case the pending closed
channel is still counted as anchor channel and a reserve output is
created. To make sure that doesn't happen, we make sure there are no
pending or open channels of any kind around before we create the sweep
transaction.
The testing framework uses runtime.Goexit() when a test fails which
still allows the defer calls to execute. That's why we should use defer
to close the done channel to allow all goroutines to exit properly.
We update several of our integration tests that exercise different
scenarios with regards to the broadcast of a channel commitment
transaction with HTLCs in-flight to use the new commitment type for
channel leases. We do this to ensure we have complete coverage of said
channel commitment type. This required changing several assumptions
throughout the tests based on when we should expect sweeps to happen.
To simplify the message signing API even further, we refactor the
lnwallet.MessageSigner interface to use a key locator instead of the
public key to identify which key should be signed with.
To simplify the API surface of a remote signer even more, we refactor
the SignMessage and SignMessageCompact calls to only accept a key
locator as we always know what key we're using for signing anyway.
To make it possible to use a remote lnrpc server as a signer for our
wallet, we need to change our main interface to sign the message instead
of the message's digest. Otherwise we'd need to alter the
lnrpc.SignMessage RPC to accept a digest instead of only the message
which has security implications.
To make it possible to use a remote signrpc server as a signer for our
wallet, we need to change our main interface to sign the message instead
of the message's digest. Otherwise we'd need to alter the
signrpc.SignMessage RPC to accept a digest instead of only the message
which has security implications.
We now redirect the mineBlocks function to the mineBlocksSlow function
which waits after each mined block. To reduce the overall time impact of
using that function everywhere, we only wait 20 milliseconds instead of 50ms
after each mined block to give all nodes
some time to process the block. This will still slow down everything by a bit
but reduce flakes that are caused by different sub systems not being
up-to-date.
The latest version of btcd allows its stall handler to be disabled. We
use that new config option to make sure the mining btcd node and the lnd
chain backend btcd node aren't disconnected if some test takes too long
and no new p2p messages are exchanged.
In this commit, a subscription is made to topology updates. For any
NodeAnnouncements received for our peristent peers, we store their newly
advertised addresses. If at the time of receiving these new addresses
there are any existing connection requests, these are updated to reflect
the newly advertised addresses.
In this commit we demonstrate a bug to show that if an inbound peer
changes their listening address to one not advertised in their original
NodeAnnouncement then we will not be able to reconnect to them. This bug
will be fixed in a follow-up commit.
This commit adds an itest to demonstrate that if a peer advertises
multiplie external IP addresses, then they will not all be used to
reconnect to the peer during reconnection. This will be fixed in a
follow-up commit.
The assertNumConnection function currently takes in an 'expected' number
of connections argument and asserts that both alice and bob only each
only have that number of connections. So this fails to be useful if say
alice is also connected to charlie cause then if we call
assertNumConnections between alice and bob it will fail saying there are
2 connections between them since all it does is count alice's total
number of connections. This commit replaces this function with 2 new
functions: assertConnected which asserts that at least one connection
exists between two peers and assertNotConnected which asserts that no
connections exists between the two peers.
With go 1.17 a change to the build flags was implemented:
https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
The formatter now automatically adds the forward-compatible build tag
format and the linter checks for them, so we need to include them in our
code.
This commit fixes a nil pointer issue when a node fails to start up.
Because require.NoErrorf() doesn't abort a test immediately if run
inside a goroutine, this lead to the test continuing with nil node
references which lead to a panic later on.
This commit updates call-sites to use the proper dust limits for
various script types. This also updates the default dust limit used
in the funding flow to be 354 satoshis instead of 573 satoshis.
This commit fixes a flake in the channel status update itest that
occurred if Carol got a channel edge update for a channel before it
heard of the channel in the first place. To avoid that, we wait for
Carol to sync her graph before sending out channel edge or policy
updates.
As always when we touch itest code, we bring the formatting and use of
the require library up to date.
This commit fixes a nil pointer issue when a node fails to start up.
Because require.NoErrorf() doesn't abort a test immediately if run
inside a goroutine, this lead to the test continuing with nil node
references which lead to a panic later on.
Fixes#4383 by adding a new SafeCopyMacaroon function that correctly
clones all caveats and prevents modifications on the copy from affecting
the original.
Fixes#5680.
To make sure we're always reading from the WebSocket connection, we need
to always have an ongoing (but blocking) conn.ReadMessage() call going
on. To achieve this, we do the read in a separate goroutine and write to
a buffered channel. That way we can always read the next message while
the current one is being forwarded. This allows incoming ping messages
to be received and processed which then leads to the deadlines to be
extended correctly.
In this commit we add an extra config for enabling logging to an
external file when using embedded etcd. This can be useful when running
integration tests to see more details about etcd related issues.
Using the default, global ServeMux prevents the same process from
calling `lntest.NewNetworkHarness` multiple times, because we get a
panic when registering HTTP routes.
Instead, we use the ServeMux beloning to the fee service struct.
We add an additional test case to the on-chain fund recovery test that
tries restoring the same wallet from the extended master root key
instead of the seed.
This commit fixes the issue where a wrong context being inherited to
create a timeout context. When a parent context timed out, all its
children contexts also timed out, even the children contexts had a
larger timeout value. This means it only makes sense to inherite from a
parent when its children have smaller timeout value. Given the setup of
the itest, all the timeout contexts need to be created from a context
background(hence no timeout on the parent) unless there's an explicit
timeout bound we want to set.
In this commit, we put the context initialization inside
openChannelAndAssert, thus saving us a few lines and making sure the
context is always properly handled.
In lnd, log messages about channels are generally logged with a
reference to their channel point rather than the short channel id.
Channel point is reorg-resistant and also easier to look up in for
example a block explorer.
In the link however, all log messages are accompanied by short channel
id. This makes it difficult to grep a log for all channel activity. The
PEER message for example which are often crucial to analyse, are logged
with channel points.
This commit modifies the link logging to also use channel points.
Because we now get one more update per channel when closing it, we need
to update our test that looks at the close notifications sent by the
SubscribeChannelEvent RPC.
We want to make sure we can recover funds from a channel that was force
closed by the local node just before the node needed to be restored. We
add a special test case with specific assertions for that scenario.
With one more notification event being dispatched in the local force
close case we need to update one of the integration tests to account for
the additional message.
This commit uses the require library for the link nodes garbage collect
test and fixes a flake that was discovered while hunting for other
flakes. Some times the channel isn't updated fast enough so we can
detech it on first try. So we give it a few more tries to stabilize the
test.
This commit fixes an old flake in the neutrino anchor output tests. It
turns out that sometimes with Neutrino we don't have enough UTXOs in our
wallet to sweep both the local and remote anchor. This is very likely a
timing issue, we need to give the wallet more time to catch up with the
chain and process all transactions to find unspent outputs.
We address this two-fold: We add an additional UTXO to Alice. And then
we also make sure we detect both UTXOs properly after restarting.
From the error in the itest output log it is not clear whether scraping
the profile page caused a test to fail or whether it was just a
follow-up error. We make it a bit more clear with an added message.
To make it easier to see what tranche a failed test was running in, we
add the tranche index to the test name. This corresponds to the number
in the .logs-tranche<index> folder name so the logs can be found much
quicker.
Improve 'ErrReservedValueInvalidated' error string to explain that the
error is triggered by a transaction that would deplete funds reserved for
potential future anchor channel closings (via CPFP)
Add hint that further details can be found in the debug log
Update strings in 'lntest/itest/log_error_whitelist.txt' correspondingly
In this commit, we attempt to fix a race condition that may occur in the
current AMP and MPP tests.
It appears the following scenario is possible:
* The `mppTestContext` [is used to create 6 channels back to
back](https://github.com/lightningnetwork/lnd/blob/master/lntest/itest/lnd_amp_test.go#L43)
* The method used to create the channel ends up calling
[`openChannelAndAssert`](edd4152682/lntest/itest/lnd_mpp_test.go (L300))
which'll open the channel, mine 6 blocks, [then ensure that the
channel gets
advertised](edd4152682/lntest/itest/assertions.go (L78))
* Later on, [we wait for all nodes to hear about all channels on the
network
level](https://github.com/lightningnetwork/lnd/blob/master/lntest/itest/lnd_amp_test.go#L62)
I think the issue here is that we'll potentially already have mined 30
or so blocks before getting to the final nodes, and those nodes may have
already heard about the channel already. This may then cause their
[`lightningNetworkWatcher`](edd4152682/lntest/node.go (L1213))
goroutine to not properly dispatch this, since it's assumed that the
channel hasn't been announced (or notified) when the method is called.
One solution here is to just check if the channel is already in the
node's graph or not, when we go to register the notification. If we do
this in the same state machine as the watcher, then we ensure if the
channel is already known, the client is immediately notified. One thing
that can help us debug this more in the future is adding additional
logging in some of these helper goroutines so we can more easily track
the control flow.
This commit implements this solution by checking to ensure that the
channel isn't already known in our channel graph before attempting to
wait for its notification, as we may already have missed the
notification before this registration request came in.
