Because the new startup `ChannelMonitor` persistence semantics rely
on new information stored in `ChannelMonitor` only for claims made
in the upgraded code, users upgrading from previous version of LDK
must apply the old `ChannelMonitor` persistence semantics at least
once (as the old code will be used to handle partial claims).
When we discover we've only partially claimed an MPP HTLC during
`ChannelManager` reading, we need to add the payment preimage to
all other `ChannelMonitor`s that were a part of the payment.
We previously did this with a direct call on the `ChannelMonitor`,
requiring users write the full `ChannelMonitor` to disk to ensure
that updated information made it.
This adds quite a bit of delay during initial startup - fully
resilvering each `ChannelMonitor` just to handle this one case is
incredibly excessive.
Over the past few commits we dropped the need to pass HTLCs
directly to the `ChannelMonitor`s using the background events to
provide `ChannelMonitorUpdate`s insetad.
Thus, here we finally drop the requirement to resilver
`ChannelMonitor`s on startup.
When we claim an MPP payment, then crash before persisting all the
relevant `ChannelMonitor`s, we rely on the payment data being
available in the `ChannelManager` on restart to re-claim any parts
that haven't yet been claimed. This is fine as long as the
`ChannelManager` was persisted before the `PaymentClaimable` event
was processed, which is generally the case in our
`lightning-background-processor`, but may not be in other cases or
in a somewhat rare race.
In order to fix this, we need to track where all the MPP parts of
a payment are in the `ChannelMonitor`, allowing us to re-claim any
missing pieces without reference to any `ChannelManager` data.
Further, in order to properly generate a `PaymentClaimed` event
against the re-started claim, we have to store various payment
metadata with the HTLC list as well.
Here we finally implement claiming using the new MPP part list and
metadata stored in `ChannelMonitor`s. In doing so, we use much more
of the existing HTLC-claiming pipeline in `ChannelManager`,
utilizing the on-startup background events flow as well as properly
re-applying the RAA-blockers to ensure preimages cannot be lost.
In the next commit we'll start using (much of) the normal HTLC
claim pipeline to replay payment claims on startup. In order to do
so, however, we have to properly handle cases where we get a
`DuplicateClaim` back from the channel for an inbound-payment HTLC.
Here we do so, handling the `MonitorUpdateCompletionAction` and
allowing an already-completed RAA blocker.
Here we wrap the logic which moves claimable payments from
`claimable_payments` to `pending_claiming_payments` to a new
utility function on `ClaimablePayments`. This will allow us to call
this new logic during `ChannelManager` deserialization in a few
commits.
In a coming commit we'll use the existing `ChannelManager` claim
flow to claim HTLCs which we found partially claimed on startup,
necessitating having a full `ChannelManager` when we go to do so.
Here we move the re-claim logic down in the `ChannelManager`-read
logic so that we have that.
When we claim an MPP payment, then crash before persisting all the
relevant `ChannelMonitor`s, we rely on the payment data being
available in the `ChannelManager` on restart to re-claim any parts
that haven't yet been claimed. This is fine as long as the
`ChannelManager` was persisted before the `PaymentClaimable` event
was processed, which is generally the case in our
`lightning-background-processor`, but may not be in other cases or
in a somewhat rare race.
In order to fix this, we need to track where all the MPP parts of
a payment are in the `ChannelMonitor`, allowing us to re-claim any
missing pieces without reference to any `ChannelManager` data.
Further, in order to properly generate a `PaymentClaimed` event
against the re-started claim, we have to store various payment
metadata with the HTLC list as well.
Here we store the required MPP parts and metadata in
`ChannelMonitor`s and make them available to `ChannelManager` on
load.
When we claim an MPP payment, then crash before persisting all the
relevant `ChannelMonitor`s, we rely on the payment data being
available in the `ChannelManager` on restart to re-claim any parts
that haven't yet been claimed. This is fine as long as the
`ChannelManager` was persisted before the `PaymentClaimable` event
was processed, which is generally the case in our
`lightning-background-processor`, but may not be in other cases or
in a somewhat rare race.
In order to fix this, we need to track where all the MPP parts of
a payment are in the `ChannelMonitor`, allowing us to re-claim any
missing pieces without reference to any `ChannelManager` data.
Further, in order to properly generate a `PaymentClaimed` event
against the re-started claim, we have to store various payment
metadata with the HTLC list as well.
Here we take the first step, building a list of MPP parts and
metadata in `ChannelManager` and passing it through to
`ChannelMonitor` in the `ChannelMonitorUpdate`s.
When we started tracking which channels had MPP parts claimed
durably on-disk in their `ChannelMonitor`, we did so with a tuple.
This was fine in that it was only ever accessed in two places, but
as we will start tracking it through to the `ChannelMonitor`s
themselves in the coming commit(s), it is useful to have it in a
struct instead.
In aa09c33a17 we added a new secret
in `ChannelManager` with which to derive inbound `PaymentId`s. We
added read support for the new field, but forgot to add writing
support for it. Here we fix this oversight.
`or_default` is generally less readable than writing out the thing
we're writing, as `Default` is opaque but explicit constructors
generally are not. Thus, we ignore the clippy lint (ideally we
could invert it and ban the use of `Default` in the crate entirely
but alas).
These structs are meant for MonitoringUpdatingPersister implementation, but some
external implementations may still reuse them, so going to make them public.
There is a decent amount of shared code in these two methods so we make
an attempt to share that code here by introducing the
`InitialRemoteCommitmentReceiver` trait. This trait will also come in
handy when we need similar commitment_signed handling behaviour for
dual-funded channels.
Rename parameters used when calculating success probability to make it
clear that the total mount in-flight should be used rather than the
payment amount.
Commit df52da7b31 modified
ProbabilisticScorer to apply some penalty amount multipliers to the
total amount flowing over the channel. However, the commit updated the
docs for base_penalty_amount_multiplier_msat even though that behavior
didn't change. This commit reverts those docs.
Commit df52da7b31 modified
ProbabilisticScorer to apply some penalty amount multipliers (e.g.,
liquidity_penalty_amount_multiplier_msat) to the total amount flowing
over the channel (i.e., including inflight HTLCs), not just the payment
in question. This led to over-penalizing in-use channels. Instead, only
apply the total amount when calculating success probability.
In a previous commit we updated the fee-bump-rate of claims against
HTLC timeouts on counterparty commitment transactions so that
instead of immediately attempting to bump every block we consider
the fact that we actually have at least `MIN_CLTV_EXPIRY_DELTA`
blocks to do so, and bumping at the appropriate rate given that.
Here we test that by adding an extra check to an existing test
that we do not bump in the very next block after the HTLC timeout
claim was initially broadcasted.
For outbound HTLCs, the counterparty can spend the output
immediately. This fixes the `counterparty_spendable_height` in the
`PackageTemplate` claiming outbound HTLCs on local commitment
transactions, which was previously spuriously set to the HTLC
timeout (at which point *we* can claim the HTLC).
Now that the module only contains some implementations of
serialization for the `lightning_types::features` structs, there's
no reason for it to be public.
This adds a simple test that the gossip message buffer in
`PeerManager` is limited, including the new behavior of bypassing
the limit when the broadcast comes from the
`ChannelMessageHandler`.
In testing, its useful to be able to tell the `SocketDescriptor` to
pretend the system network buffer is full, which we add here by
creating a new `hang_writes` flag. In order to simplify
constructing, we also add a new constructor which existing tests
are moved to.
When our `ChannelMessageHandler` creates gossip broadcast
`MessageSendEvent`s, we generally want these to be reliably
delivered to all our peers, even if there's not much buffer space
available.
Here we do this by passing an extra flag to `forward_broadcast_msg`
which indicates where the message came from, then ignoring the
buffer-full criteria when the flag is set.
