Historically, we used `vec_type` for all TLV Vec reads/writes, but
it is asymmetric and thus somewhat confusing - on the write side it
always writes a TLV entry, even if there are zero elements. On the
read side, it happily accepts a missing TLV, providing a
zero-length vector.
In 85b573ddad a new `optional_vec`
TLV format was added which was symmetric, but only supports
optional vecs.
Now that we've migrated entirely to the new `required_vec` TLV
type, we can entirely remove the awkward `vec_type`.
* `HolderSignedTx::htlc_outputs` has always been written since it
was converted to TLVs in 86641ea680.
* `ChanelMonitorUpdateStep::*::htlc_outputs` have been written
since the enum was converted to TLVs in 86641ea680.
If a channel has been closed, there may still be some
`ChannelMonitorUpdate`(s) which are pending completion. These
in-flight updates may also be blocking another channel from letting
an update fly, e.g. for forwarded payments where the payment
preimage will be removed from the downstream channel after the
upstream channel has closed.
Luckily all the infrastructure to handle this case is already in
place - we just need to process the
`monitor_update_blocked_actions` for closed channels.
We currently assume the owner of `ChannelMonitor`s won't persist
the `ChannelMonitor` while `Event`s are being processed. This is
fine, except (a) its generally hard to do so and (b) the
`ChainMonitor` doesn't even do this.
Thus, in rare cases, a user could begin processing events which
are, generated by connecting a transaction or a new best-block,
take some time to do so, and while doing so process a further chain
event, causing persistece. This could lose the event being
processed alltogether, which could lose the user funds.
This should be very rare, but may have been made slightly more
reachable with (a) the async event processing making it more
common to do networking in event handling, (b) the new future
generation in the `ChainMonitor`, which now wakes the
`background-processor` directly when chain actions happen on the
`ChainMonitor`.
This was a regression resulting from f2453b7 since we now process events
in a loop until there aren't any left. Processing events is done in
batches and they are not removed until we're done processing each batch.
Since handling a `PendingHTLCsForwardable` event will call back into the
`ChannelManager`, we'll still see the original forwarding event not
removed. Phantom payments will need an additional forwarding event
before being claimed to make them look real by taking more time.
Right now the only real way to verify the node and channel
announcements is to call `update_node_from_announcement`/
`update_channel_from_announcement`. If you want to do some
processing before you add to your network graph then you need to
manually verify the signature. This adds some nice helper functions
to make it easier.
I tried to do the same for channel update but it did not seem as
easy so figured that is fine to punt on since I don't see many
people doing manual things with channel updates.
Because `ChannelMonitorUpdate`s can complete asynchronously and
out-of-order now, a `commitment_signed` `ChannelMonitorUpdate` from
a downstream channel could complete prior to the preimage
`ChannelMonitorUpdate` on the upstream channel. In that case, we may
not get a `update_fulfill_htlc` replay on startup. Thus, we have to
ensure any payment preimages contained in that downstream update are
re-claimed on startup.
Here we do this during the existing walk of the `ChannelMonitor`
preimages for closed channels.
* `PhantomRouteHints::channels` has been written since the struct
was added in 410eb05365.
* `HTLCSource::path_hops` has been written since the struct was
converted to TLVs in 66784e32fe.
* `BlindedTail::hops` has always been written since it was
introduced in 64c26c8a79.
* `PaymentParameters::clear_hints` has always been written since
it was introduced as `Payee::route_hitns` in 46b68c517d.
* `CommitmentTransaction::htlcs` has always been written since the
struct was converted to TLVs in 66784e32fe.
* `HolderCommitmentTransaction::counterparty_htlc_sigs` have always
been written since the struct was converted to TLVs in
c8bc1b6d3d.
* `PaymentPathFailed::path` was added without an optional compat
wrapper in e5310dd5f0.
* `PaymentPathSuccessful::path` has always been written since the
event was added in 2c4f16d5e3.
* `Probe{Failed,Successful}::path` have always been written since
the events were added in eb8bce0d16.
This converts some required TLVs to `required_vec` which are, in
fact, required (and have been written forever).
* `HTLCFailReason` hasn't changed since many structs were converted
to TLVs in 66784e32fe.
* `NodeInfo::channels` has been written since `NetworkGraph`
structs were converted to TLVs in 321b19c4d9.
* Several test-only TLV writes were converted.
Historically, we used `vec_type` for all TLV Vec reads/writes, but
it is asymmetric and thus somewhat confusing - on the write side it
always writes a TLV entry, even if there are zero elements. On the
read side, it happily accepts a missing TLV, providing a
zero-length vector.
In 85b573ddad a new `optional_vec`
TLV format was added which was symmetric, but only supports
optional vecs. This adds the corresponding required form, always
writing a TLV and ensuring it is present.
This commit makes use of the added enum to calculate the dust
exposure threshold based on the current fee rate. This also updates
tests to ensure it works as intended.
With fee rates rising dramatically in mid-April 2023, thresholds for
what is considered dust have risen, often exceeding our previous dust
exposure threshold of 5k sats. This causes all payments and HTLC
forwards between 5k sats and new dust thresholds to fail.
This commit changes our max dust exposure config knob from a fixed
upper limit to a `MaxDustHTLCExposure` enum with an additional variant
to allow setting our max dust exposure to a multiplier on the current
high priority feerate.
To remain backwards compatible we'll always write the fixed limit if
it's set, or its default value in its currently reserved TLV.
We also now write an odd TLV for the new enum, so that previous
versions can safely ignore it upon downgrading, while allowing us to
make use of the new type when it's written.
In 1ce2beb774,
`Channel::blocked_monitor_updates` was moved to an even TLV to
ensure downgrades with vec entries are forbidden. However, the
serialized type remained `vec_type`, which is always written.
Instead, `optional_vec` must be used.
In 9dfe42cf86,
`ChannelMonitorUpdate`s were stored in `Channel` while they were
being processed. Because it was possible (though highly unlikely,
due to various locking likely blocking persistence) an update was
in-flight (even synchronously) when a `ChannelManager` was
persisted, the new updates were persisted via an odd TLV.
However, in 4041f0899f these pending
monitor updates were moved to `ChannelManager`, with appropriate
handling there. Now the only `ChannelMonitorUpdate`s which are
stored in `Channel` are those which are explicitly blocked, which
requires the async pipeline.
Because we don't support async monitor update users downgrading to
0.0.115 or lower, we move to persisting them via an even TLV. As
the odd TLV storage has not yet been released, we can do so
trivially.
Fixes#2317.
Such implementation allows `MonotonicTime` to go backward up to 10
years on all platforms. On some platforms (e.g. iOS) `Instant` is
represented as `u64` of nanoseconds since the boot of the system.
Obviously such implementation does not allow to go backward before the
time of the boot.
Co-authored-by: Andrei <andrei.i@posteo.de>
Co-authored-by: Jeffrey Czyz <jkczyz@gmail.com>
This ensures freshly initialized nodes can proceed to create unexpired
invoices without a call to `best_block_updated`, since an invoice's
expiration delta is applied to `highest_seen_timestamp`.
