This introduces a BigSize migration that is used to expand the width
of the ChannelStatus and ChannelType fields. Three channel "types"
are added - ZeroConfBit, ScidAliasChanBit, and ScidAliasFeatureBit.
ScidAliasChanBit denotes that the scid-alias channel type was
negotiated for the channel. ScidAliasFeatureBit denotes that the
scid-alias feature bit was negotiated during the *lifetime* of the
channel. Several helper functions on the OpenChannel struct are
exposed to aid callers from different packages.
The RefreshShortChanID has been renamed to Refresh.
A new function BroadcastHeight is used to guard access to the
mutable FundingBroadcastHeight member. This prevents data races.
In this commit, we consolidate the _lease specific_ logic for the
success and timeout HTLC resolvers. We do this with the addition of a
new struct which is then composed via struct embedding with the two
existing structs. This fixes a flake in the integration tests by
ensuring the height is set up front, rather than eventually once the
height matches the lock time.
This prevents a panic where an insert is attempted on a nil map via
updateActiveHTLCs. This panic would occur if the channel arbitrator
was in a buggy state, possibly introduced by power loss or via
SIGKILL.
The main idea is that NotifyContractUpdate adds the ContractUpdate to
a map called unmerged. It is populated in Start by shallow-copying the
activeHTLCs map values (htlcSet). The htlcSets underlying maps are not
copied, and so unmerged will just contain pointers to them. This should
be fine since unmerged will not modify them. At the call-sites of
activeHTLCs, it is updated to include the unmerged sets. This happens
with a mutex and should not cause any data race, even though it is
copying the underlying map pointers. No persistence should be
necessary since on restart, activeHTLCs and unmerged will just be
populated again.
This commit changes the `NewBreachRetribution` to use the new revocation
log format, while maintaining the compatibilty to use an older
revocation log format. Unit tests have been added to make sure a breach
retribution can be created in both log formats.
This also means the watch tower needs to pass the relevant commit tx at
its backup height when creating the breach retribution during backing
up. This is achieved by recording the current remote commitment state
before advancing the remote commitment chain.
This commit was previously split into the following parts to ease
review:
- 2d746f68: replace imports
- 4008f0fd: use ecdsa.Signature
- 849e33d1: remove btcec.S256()
- b8f6ebbd: use v2 library correctly
- fa80bca9: bump go modules
This also changes the chain_watcher and breacharbiter handoff. The
new logic ensures that the channel is only marked as pending closed
when the channel arbitrator has persisted the resolutions and commit
set.
For older nodes, this bucket was never created, so we'll get an error if
we try and query it. In this commit, we catch this error like we do when
a given channel doesn't have the information (but the bucket actually
exists).
Fixes#6155
In this commit, we fix a bug that would cause newly updated nodes to be
unable to start up, if they have an older channel that was closed before
we started to store all the historical state for each channel.
The issue is that we started to write the complete state to disk, but
newer channels don't have it, so when we try to supplement the
resolvers, we run into this error.
Ultimately, we only need this new supplemented information for script
enforcement channels. Ideally we would instead check the channel type
there instead, but it doesn't appear to be available in this context as
is, without further changes.
Fixes https://github.com/lightningnetwork/lnd/issues/6001.
In order to sweep the commitment and HTLC outputs belonging to a
script-enforced leased channel, each resolver must know whether the
additional CLTV clause on the channel initiator applies to them. To do
so, we retrieve the historical channel state stored within the database
and supplement it to the resolvers to provide them with what's needed in
order to sweep the necessary outputs and resolve their respective
contracts.
This commit modifies the channel state machine to be able to derive the
proper commitment and second-level HTLC output scripts required by the
new script-enforced leased channel commitment type.
In this commit, we take an initial step towards converting the existing
breach arbiter and utxo nursery logic into contract resolvers by moving
the files as is, into the `contractcourt` pacakge.
This commit is primarily move only, though we had to massage some
interfaces and config names along the way to make things compile and the
tests run properly.
Adds an optional tx parameter to ForAllOutgoingChannels and FetchChannel
so that data can be queried within the context of an existing database
transaction.
We might want to react to a channel being fully resolved after being
involved in a force close. For this we add a new callback and invoke it
where appropriate.
This commit adds two tests to check that a) the correct deadline is used
given different HTLC sets and b) when sweeping anchors the correct
deadlines are used.
This commit adds a deadline field to mockSweeper that can be used to
track the customized conf target (deadline) used for sweeping anchors.
The relevant test, TestChannelArbitratorAnchors is updated to reflect
that the deadlines are indeed taking effect.
In this commit, we made the change so that when sweeping anchors for the
commitment transactions, we will be aware of the deadline which is
derived from its HTLC set. It's very likely we will use a much larger
conf target from now on, and save us some sats.
This commit adds a new struct AnchorResolutions which wraps the anchor
resolutions for local/remote/pending remote commitment transactions. It
is then returned from NewAnchorResolutions. Thus the caller knows how to
retrieve a certain anchor resolution.
closed
This commit makes the handoff procedure between the breachabiter and
chainwatcher use a function closure to mark the channel pending closed
in the DB. Doing it this way we know that the channel has been markd
pending closed in the DB when ProcessACK returns.
The reason we do this is that we really need a "two-way ACK" to have the
breacharbiter know it can go on with the breach handling. Earlier it
would just send the ACK on the channel and continue. This lead to a race
where breach handling could finish before the chain watcher had marked
the channel pending closed in the database, which again lead to the
breacharbiter failing to mark the channel fully closed.
We saw this causing flakes during itests.
This commit moves the contract breach event dispatch after the channel
close summary has been added to the database. This is important
otherwise it may occur that we attempt to mark the channel fully closed
while the channel close summary is not yet serialized.
This commit moves the logic for sweeping the confirmed second-level
timeout transaction into its own method.
We do a small change to the logic: When setting the spending tx in the
report, we use the detected commitspend instead of the presigned tiemout
tx. This is to prepare for the coming change where the spending
transaction might actually be a re-signed timeout tx, and will therefore
have a different txid.
success tx
This commit makes the HTLC resolutions having non-nil SignDetails
(meaning we can re-sign the second-level transactions) go through the
sweeper. They will be offered to the sweeper which will cluster them and
arrange them on its sweep transaction. When that is done we will further
sweep the output on this sweep transaction as any other second-level tx.
In this commit we do this for the HTLC success resolver and the
accompanying HTLC success transaction.
To make the linter happy, make a pointer to the inner resolver.
Otherwise the linter would complain with
copylocks: literal copies lock value
since we'll add a mutex to the resolver in following commits.
* mod: bump btcwallet version to accept db timeout
* btcwallet: add DBTimeOut in config
* kvdb: add database timeout option for bbolt
This commit adds a DBTimeout option in bbolt config. The relevant
functions walletdb.Open/Create are updated to use this config. In
addition, the bolt compacter also applies the new timeout option.
* channeldb: add DBTimeout in db options
This commit adds the DBTimeout option for channeldb. A new unit
test file is created to test the default options. In addition,
the params used in kvdb.Create inside channeldb_test is updated
with a DefaultDBTimeout value.
* contractcourt+routing: use DBTimeout in kvdb
This commit touches multiple test files in contractcourt and routing.
The call of function kvdb.Create and kvdb.Open are now updated with
the new param DBTimeout, using the default value kvdb.DefaultDBTimeout.
* lncfg: add DBTimeout option in db config
The DBTimeout option is added to db config. A new unit test is
added to check the default DB config is created as expected.
* migration: add DBTimeout param in kvdb.Create/kvdb.Open
* keychain: update tests to use DBTimeout param
* htlcswitch+chainreg: add DBTimeout option
* macaroons: support DBTimeout config in creation
This commit adds the DBTimeout during the creation of macaroons.db.
The usage of kvdb.Create and kvdb.Open in its tests are updated with
a timeout value using kvdb.DefaultDBTimeout.
* walletunlocker: add dbTimeout option in UnlockerService
This commit adds a new param, dbTimeout, during the creation of
UnlockerService. This param is then passed to wallet.NewLoader
inside various service calls, specifying a timeout value to be
used when opening the bbolt. In addition, the macaroonService
is also called with this dbTimeout param.
* watchtower/wtdb: add dbTimeout param during creation
This commit adds the dbTimeout param for the creation of both
watchtower.db and wtclient.db.
* multi: add db timeout param for walletdb.Create
This commit adds the db timeout param for the function call
walletdb.Create. It touches only the test files found in chainntnfs,
lnwallet, and routing.
* lnd: pass DBTimeout config to relevant services
This commit enables lnd to pass the DBTimeout config to the following
services/config/functions,
- chainControlConfig
- walletunlocker
- wallet.NewLoader
- macaroons
- watchtower
In addition, the usage of wallet.Create is updated too.
* sample-config: add dbtimeout option
This adds the scenario to the local force close test cases where a node
force closes one of its channels, then lose state (or do recovery)
before the commmitment is confirmed. Without the previous commit this
would go undetected.
outdated local state
This commit fixes a bug that would cause us to not sweep our local
output in case we force closed, then lost state or attempted recovery.
The reason being that we would use or local commit height when deriving
our scripts, which would be incorrect. Instead we use the extracted
state number to derive the correct scripts, allowing us to sweep the
output.
Allthough being an unlikely scenario, we would leave money on chain in
this case without any warning (since we would just end up with an empty
delay script) and forget about the spend.
Similar to what we did for the local state handling, we extract handling
all known remote states we can act on (breach, current, pending state)
into its own method.
Since we want to handle the case where we lost state (both in case of
local and remote close) last, we don't rely on the remote state number
to check which commit we are looking at, but match on TXIDs directly.
The tests didn't really roll back the channel state, so we would only
rely on the state number to determine whether we had lost state. Now we
properly roll back the channel to a previous state, in preparation for
upcoming changes.
To allow us to grab all of the information we need for our channel arbs
in a more efficient way on startup, we add an optional tx to our lookup
functions required on start.
Similarly as with kvdb.View this commits adds a reset closure to the
kvdb.Update call in order to be able to reset external state if the
underlying db backend needs to retry the transaction.
This commit adds a reset() closure to the kvdb.View function which will
be called before each retry (including the first) of the view
transaction. The reset() closure can be used to reset external state
(eg slices or maps) where the view closure puts intermediate results.
For unconfirmed commit tx anchors, supply the sweeper with cpfp info and
a confirmation target fee estimate.
The sweeper will try to pay for the parent commit tx as long as the
current fee estimate exceeds the pre-signed commit tx fee rate.
Extend the fee estimator to take into account parent transactions with
their weights and fees.
Do not try to cpfp parent transactions that have a higher fee rate than
the sweep tx fee rate.
The sweeper call UpdateParams does not update the exclusive group
property of a pending sweep. This led to anchor outputs being swept
after confirmation with an exclusive group restriction, which is not
necessary.
This commit changes the anchor resolver to not use UpdateParams anymore,
but instead always re-offer the anchor input to the sweeper. The sweeper
is modified so that a re-offering also updates the sweep parameters.
This commit moves all localized instances of mock implementations of
the Signer interface to the lntest/mock package. This allows us to
remove a lot of code and have it housed under a single interface in
many cases.
Follow up labelling of external transactions with labels for the
transaction types we create within lnd. Since these labels will live
a life of string matching, a version number and rigid format is added
so that string matching is less painful. We start out with channel ID,
where available, and a transaction "type". External labels, added in a
previous PR, are not updated to this new versioned label because they
are not lnd-initiated transactions. Label matching can check this case,
then check for a version number.
When a remote peer claims one of our outgoing htlcs on chain, we do
not care whether they claimed with multiple stages. We simply store
the claim outgome then forget the resolver.
Incoming htlcs that are timed out or failed (invalid htlc or invoice
condition not met), save a single on chain resolution because we don't
need to take any actions on them ourselves (we don't need to worry
about 2 stage claims since this is the success path for our peer).
Our current set of reports contain much of the information we will
need to persist contract resolutions. We add a function to create
resolver reports from our exiting set of resolutions.
To allow us to write the outcome of our resolver to disk, we add
optional resolver reports to the CheckPoint function. Variadic params
are used because some checkpoints may have no reports (when the resolver
is not yet complete) and some may have two (in the case of a two stage
resolution).
Previously it wasn't possible to store a preimage in the invoice
database and signal that a payment should not be settled right away. The
only way to hold a payment was to insert the magic UnknownPreimage value
in the invoice database. This commit introduces a distinct flag to
signal that an invoice is a hold invoice and thereby allows the preimage
to be present in the database already.
Preparation for (key send) hodl invoices for which we already know the
preimage.
Add label parameter to PublishTransaction in WalletController
interface. A labels package is added to store generic labels that are
used for the different types of transactions that are published by lnd.
To keep commit size down, the two endpoints that require a label
parameter be passed down have a todo added, which will be removed in
subsequent commits.
This adds a test to the commit sweeper resolver to ensure it behaves
properly if the local node breaches a channel.
In this situation the remote party is expected to sweep the breached
output to itself and therefore the local party won't be able to recover
any funds.
This fixes an issue where the contract court could leave a completely
swept commit tx unresolved if it was swept by the remote party.
This could happen if (our) commit tx just published was actually a
previously revoked state, in which case the remote party would claim the
funds via a justice transaction.
This manifested itself in the testRevokedCloseRetribution integration
test where at the end of the test Bob was left with a pending channel
that never resolved itself.
In this commit, we add some additional logging of the commitments at
play when we detect a channel closure on-chain. This should help to
debug things more in the future as we don't log the full commitments
anywhere. We also now also print the type of commitment as well, as a
follow up from the recent anchor outputs work. In the near future, as we
implement a dynamic commitments update protocol, always logging the
commitment type as well will likely be useful for debugging purposes.
Sweeping anchors and being able to bump the fee was already added in a
previous commit. This commit extends anchor sweeping with an anchor
resolver object that becomes active after the commitment tx confirms.
At that point, the anchors do not serve the purpose of getting the
commitment tranaction confirmed anymore. It is however still possible to
reclaim some of their value if using a low fee rate.
Preparation for anchor resolver. The recovered anchor amount should
still be included in the pending channel report even after it has been
resolved.
This also fixes an existing bug that in some cases caused the recovered
amount from an htlc resolver not to be included in the total.
Start anchor sweep attempts immediately after the commitment transaction
has been published. This makes the anchor known to the sweeper and
allows the user to bump the fee on it to get their commitment
transaction confirmed in case the fee committed too is insufficient for
timely confirmation.
Prior to this change, the trigger height for closed channels was set to
the current best block height. As this height is in some cases used as a
height hint, the spend may have been missed.
We use the fact that we can tell whether the commit is local or remote
by inspecting the witness script. We cannot use the maturity delay
anymore, as we can have delayed to_remote outputs also now.
Co-authored-by: Joost Jager <joost.jager@gmail.com>
This commit adds two new channel statuses which indicate the party that
initatited closing the channel. These statuses are set in conjunction
with the existing commit broadcast status so that we do not need to
migrate existing logic to handle multiple types of closes. This status
is set for locally initiated force closes in this commit because they
follow a similar pattern to cooparative closes, marking the commitment
broadcast then proceeding with tx broadcast. Remote force closes are
added in the following commit, as they are handled differently.
This commit enables the user to specify he is not interested in
automatically close channels with pending payments that their
corresponding htlcs have timed-out.
By requiring a configurable grace period uptime of our node
before closing such channels, we give a chance to the other node to
properly cancel the htlc and avoid unnecessary on-chain transaction.
In mobile it is very important for the user experience as otherwise
channels will be force closed more frequently.
This commit repalces the htlcResolution struct with an interface.
This interface is implemeted by failure, settle and accept resolution
structs. Only settles and fails are exported because the existing
code that handles htlc resolutions uses a nil resolution to indicate
that a htlc was accepted. The accept resolution is used internally
to report on the resolution result of the accepted htlc, but a nil
resolution is surfaced. Further refactoring of all the functions
that call NotifyExitHopHtlc to handle a htlc accept case (rather than
having a nil check) is required.
This commit changes the shouldGoOnChain signature to get the htlc
as parameter. I will allow the function to take decisions based on
whether the htlc is Incoming or Outgoing.
Based on the current channel type, we derive the script used for the
to_remote output. Currently only the unencumbered p2wkh type is used,
but that will change with upcoming channel types.
We abstract away how keys are generated for the different channel types
types (currently tweak(less)).
Intention is that more of the logic that is unique for each commitment
type lives in commitment.go, making the channel state machine oblivious
to the keys and outputs being created on the commitment tx for a given
channel state.
This commit moves handling of invoice not found
errors into NotifyExitHopHtlc and exposes a
resolution result to the calling functions. The
intention of this change is to make calling
functions as naive of the invoice registry's
mechanics as possible.
When NotifyExitHopHtlc is called and an invoice
is not found, calling functions can take action
based on the HtlcResolution's InvoiceNotFound
outcome rather than having to add a special error
check on every call to handle the error.
This commit adds the resolution result obtained
while updating an invoice in the registry to
htlcResolution. The field can be used by calling
functions to determine the outcome of the
update and act appropriately.
This commit adds a constructor for HtlcResolution creation
to enforce provision of all relevant values when an
event is created. A custom construstor which also takes
a preimage is added for settle events.
This commit renames HodlEvent to HtlcResolution
to better reflect the fact that the struct is
only used for htlc settles and cancels, and that
it is not specifically used for hodl invoices.
In this commit, we export the `ResolveContract` method as it's useful as
a way to manually remove active contracts from the chain and channel
arbitrator. Along the way, we also update the method to also attempt to
stop the channel arb if it exists. This allows an external party to
remove all state with a single call. Before this commit, it was assumed
that this method was only called by the channel arb itself, when it was
already on the way to exiting after all contracts were fully resolved.
We also add a set of unit tests to exercise the intended behavior as
this method is now public.
The channel arbitrator no longer passes the direct commitment output to
the nursery for incubation. Instead the resolver itself will await the
csv lock if any.
The reason to change this now is to prevent having to deal with the
(legacy) nursery code for a planned anchor outputs related change to the
commit sweep resolver (also csv lock to_remote).
It is no problem if there are any lingering incubating outputs at the
time of upgrade. This just means that the output will be offered twice
to the sweeper and this doesn't hurt.
With the introduction of additional payload fields for mpp, it becomes
a necessity to have their values available in the on-chain resolution
flow. The incoming contest resolver notifies the invoice registry of the
arrival of a payment and needs to supply all parameters for the registry
to validate the htlc.
In this commit, we create a new chainfee package, that houses all fee
related functionality used within the codebase. The creation of this new
package furthers our long-term goal of extracting functionality from the
bloated `lnwallet` package into new distinct packages. Additionally,
this new packages resolves a class of import cycle that could arise if a
new package that was imported by something in `lnwallet` wanted to use
the existing fee related functions in the prior `lnwallet` package.
In this commit, we fix a bug that would prevent users that had
unresolved contracts at the time of update from starting their nodes.
Before we added the conf commit set, the information needed to
supplement the resolvers was found in the chain action map. As a result,
if the conf commit set is nil, then we also need to check this legacy
data to ensure that we can supplement the resolvers to the best of our
ability based on the available data.
Fixes#3549.
In this commit, we consolidate the number of areas where we derive our
commitment keys. Before this commit, the `isOurCommitment` function in
the chain watcher used a custom routine to derive the expected
scripts/keys for our commitment at that height. With the recent changes,
we now have additional logic in `DeriveCommitmentKeys` that wasn't
copied over to this area. As a result, the prior logic would erroneously
detect if it was our commitment that had hit the chain or not.
In this commit, we remove the old custom code, and use
`DeriveCommitmentKeys` wihtin the chain watcher as well. This ensures
that we only need to maintain the key derivation code in a single place,
preventing future bugs of this nature.
In this commit, we update the logic in the `chainWatcher` to no longer
wait until the DLP point has been populated in the database before we
dispatch the force close summary to any registered clients. Instead, we
can sweep immediately, as we have all the information we need to sweep
the funds (just our key).
In this commit, we update the `commitSweepResolver` to be aware of
tweakless commitments. We'll now use the new behavior of the uni close
summary (leaving out the single tweak) to detect if we're dealing with a
new, or modern commitment. Depending on the commitment type, we'll then
set the witness type accordingly so we can generate the proper signature
within the sweeper.
In this commit, we update the funding workflow to be aware of the new
channel type that doesn't tweak the remote party's output within the
non-delay script on their commitment transaction. To do this, we now
allow the caller of `InnitChannelReservation` to signal if they want the
old or new (tweakless) commitment style.
The funding tests are also updated to test both funding variants, as
we'll still need to understand the legacy format for older nodes.
In this commit, we fix an existing bug in the package, causing
resolutions to be restarted without their required supplementary
information. This can happen if a distinct HTLC set gets confirmed
compared to the HTLCs that we may have had our commitment at time of
close. Due to this bug, on restart certain HTLCS would be rejected as
they would present their state to the invoice registry, but be rejected
due to checks such as amount value.
To fix this, we'll now pass in the set of confirmed HTLCs into the
resolvers when we re-launch them, giving us access to all the
information we need to supplement the HTLCS.
We also add a new test that ensures that the proper fields of a resolver
are set after a restart.
In this commit, we create a new channel arb test context struct as the
current `createTestChannelArbitrator` has several return parameters, and
upcoming changes will likely at first glance need to add one or more
additional parameters. Rather than extend the existing set of return
parameters, we opt to instead create this struct that wraps the existing
state.
Along the way we add several new utility methods to this context, and
use them in the existing tests where applicable:
* `AssertStateTransitions`
* `AssertState`
* `Restart`
* `CleanUp`
Before publishing the close tx to the network and commit to the
StateCommitmentBroadcasted state, we mark the commitment as broadcasted
and store it to the db. This ensures it will get re-published on startup
if we go down.
TestChainArbitratorRepulishCommitment testst that the chain arbitrator
will republish closing transactions for channels marked
CommitementBroadcast in the database at startup.
Earlier the channel arbitrator would fail to recognize channels pending
close that were in the breached state. This lead to the state machine
not progressing correctly, and in some cases crashing since we would
attempt to force close an already closed channel.
A test TestChannelArbitratorForceCloseBreachedChannel is added to
exercise one of these scenarios.
Earlier we would not react to breaches, as these are handled by other
subsystems. Now we advances our state machine in case of breach, such
that we'll gracefully exit, and won't have leftover state in case of a
restart.
A simple test TestChannelArbitratorBreachClose to exercise this behavior
is added.
This commit modifies hodl htlc notification from invoice registry from a
single notification per hash to distinct notifications per htlc. This
prepares for htlc-specific information (accept height) to be added to the
notification.
This commit fixes the 'unable to find incoming resolution' error that
occured when trying to resolve incoming htlcs below the dust limit that
are not actually present on the commitment tx.
