Previously, we were requiring any `UPDATE` onion errors to include
a `channel_update`, as the spec mandates[1]. If we see an onion
error which is missing one we treat it as a misbehaving node that
isn't behaving according to the spec and simply remove the node.
Sadly, it appears at least some versions of CLN are such nodes, and
opt to not include `channel_update` at all if they're returning a
`temporary_channel_failure`. This causes us to completely remove
CLN nodes from our graph after they fail to forward our HTLC.
While CLN is violating the spec here, there's not a lot of reason
to not allow it, so we go ahead and do so here, treating it simply
as any other failure by letting the scorer handle it.
[1] The spec says `Please note that the channel_update field is
mandatory in messages whose failure_code includes the UPDATE flag`
however doesn't repeat it in the requirements section so its not
crazy that someone missed it when implementing.
While these transactions were still valid, we incorrectly assumed that
they would propagate with a locktime of `current_height + 1`, when in
reality, only those with a locktime strictly lower than the next height
in the chain are allowed to enter the mempool.
In a future commit, we plan to correctly enforce that the spending
transaction has a valid locktime relative to the chain for the node
broascasting it in `TestBroadcaster::broadcast_transaction` to. We catch
up these test node instances to their expected height, such that we do
not fail said enforcement.
The `height` argument passed to `OnchainTxHandler::block_disconnected`
represents the height being disconnected, and not the current height.
Due to the incorrect assumption, we'd generate a claim with a locktime
in the future.
Ultimately, we shouldn't be generating claims within
`block_disconnected`. Rather, we should retry the claim at a later block
height, since the bitcoin blockchain does not ever roll back without
connecting a new block. Addressing this is left for future work.
This attempts to rebroadcast/fee-bump each pending claim a monitor is
tracking for a force-closed channel. This is crucial in preventing
certain classes of pinning attacks and ensures reliability if
broadcasting fails. For implementations of `FeeEstimator` that also
support mempool fee estimation, we may broadcast a fee-bumped claim
instead, ensuring we can also react to mempool fee spikes between
blocks.
In the next commit, we plan to extend the `OnchainTxHandler` to retry
pending claims on a timer. This timer may fire with much more frequency
than incoming blocks, so we want to avoid manually bumping feerates
(currently by 25%) each time our fee estimator provides a lower feerate
than before.
Unfortunately, the RAII types used by `RwLock` are not `Send`, which is
why they can't be held over `await` boundaries. In order to allow
asynchronous events processing in multi-threaded environments, we here
allow to process events without holding the `total_consistency_lock`.
To match the local signatures found in test vectors, we must make sure
we don't use any additional randomess when generating signatures, as
we'll arrive at a different signature otherwise.
Previously, our local signatures would always be deterministic, whether
we'd grind for low R value signatures or not. For peers supporting
SegWit, Bitcoin Core will generally use a transaction's witness-txid, as
opposed to its txid, to advertise transactions. Therefore, to ensure a
transaction has the best chance to propagate across node mempools in the
network, each of its broadcast attempts should have a unique/distinct
witness-txid, which we can achieve by introducing random nonce data when
generating local signatures, such that they are no longer deterministic.
This allows the `InMemorySigner` to produce its own randomness, which we
plan to use when generating signatures in future work.
We can no longer derive `Clone` due to the `AtomicCounter`, so we opt to
implement it manually.
For offers where the signing pubkey is derived, the keys need to be
extracted from the Offer::metadata in order to sign an invoice.
Parameterize InvoiceBuilder such that a build_and_sign method is
available for this situation.
Stateless verification of Invoice for Offer
Verify that an Invoice was produced from a Refund constructed by the
payer using the payer metadata reflected in the Invoice. The payer
metadata consists of a 128-bit encrypted nonce and possibly a 256-bit
HMAC over the nonce and Refund TLV records (excluding the payer id)
using an ExpandedKey.
Thus, the HMAC can be reproduced from the refund bytes using the nonce
and the original ExpandedKey, and then checked against the metadata. If
metadata does not contain an HMAC, then the reproduced HMAC was used to
form the signing keys, and thus can be checked against the payer id.
Add support for deriving a transient payer id for each Refund from an
ExpandedKey and a nonce. This facilitates payer privacy by not tying any
Refund to any other nor to the payer's node id.
Additionally, support stateless Invoice verification by setting payer
metadata using an HMAC over the nonce and the remaining TLV records,
which will be later verified when receiving an Invoice response.
Verify that an Invoice was produced from an InvoiceRequest constructed
by the payer using the payer metadata reflected in the Invoice. The
payer metadata consists of a 128-bit encrypted nonce and possibly a
256-bit HMAC over the nonce and InvoiceRequest TLV records (excluding
the payer id) using an ExpandedKey.
Thus, the HMAC can be reproduced from the invoice request bytes using
the nonce and the original ExpandedKey, and then checked against the
metadata. If metadata does not contain an HMAC, then the reproduced HMAC
was used to form the signing keys, and thus can be checked against the
payer id.
Add support for deriving a transient payer id for each InvoiceRequest
from an ExpandedKey and a nonce. This facilitates payer privacy by not
tying any InvoiceRequest to any other nor to the payer's node id.
Additionally, support stateless Invoice verification by setting payer
metadata using an HMAC over the nonce and the remaining TLV records,
which will be later verified when receiving an Invoice response.
InvoiceRequestBuilder has a field containing InvoiceRequestContents.
When deriving the payer_id from the remaining fields, a struct is needed
without payer_id as it not optional. Refactor InvoiceRequestContents to
have an inner struct without the payer_id such that
InvoiceRequestBuilder can use it instead.
Verify that an InvoiceRequest was produced from an Offer constructed by
the recipient using the Offer metadata reflected in the InvoiceRequest.
The Offer metadata consists of a 128-bit encrypted nonce and possibly a
256-bit HMAC over the nonce and Offer TLV records (excluding the signing
pubkey) using an ExpandedKey.
Thus, the HMAC can be reproduced from the offer bytes using the nonce
and the original ExpandedKey, and then checked against the metadata. If
metadata does not contain an HMAC, then the reproduced HMAC was used to
form the signing keys, and thus can be checked against the signing
pubkey.
Add an iterator that yields TlvRecords over a range of a TlvStream.
Useful for verifying that, e.g., an InvoiceRequest was sent in response
to an Offer constructed by the intended recipient.
Add support for deriving a transient signing pubkey for each Offer from
an ExpandedKey and a nonce. This facilitates recipient privacy by not
tying any Offer to any other nor to the recipient's node id.
Additionally, support stateless Offer verification by setting its
metadata using an HMAC over the nonce and the remaining TLV records,
which will be later verified when receiving an InvoiceRequest.
Now that we leverage a package's `height_timer` even for untractable
packages, there's no need to have it be an `Option` anymore. We aim to
not break compatibility by keeping the deserialization of such as an
`option`, and use the package's `height_original` when not present. This
allows us to retry packages from older `ChannelMonitor` versions that
have had a failed initial package broadcast.
Untractable packages are those which cannot have their fees updated once
signed, hence why they weren't retried. There's no harm in retrying
these packages by simply re-broadcasting them though, as the fee market
could have spontaneously spiked when we first broadcast it, leading to
our transaction not propagating throughout node mempools unless
broadcast manually.
This finally completes the piping of the `payment_metadata` from
from the BOLT11 invoice on the sending side all the way through the
onion sending + receiving ends to the user on the receive events.
When we receive an HTLC, we want to pass the `payment_metadata`
through to the `PaymentClaimable` event. This does most of the
internal refactoring required to do so - storing a
`RecipientOnionFields` in the inbound HTLC tracking structs,
including the `payment_metadata`.
In the future this struct will allow us to do MPP keysend receipts
(as it now stores an Optional `payment_secret` for all inbound
payments) as well as custom TLV receipts (as the struct is
extensible to store additional fields and the internal API supports
filtering for fields which are consistent across HTLCs).
If we receive an HTLC and are processing it a potential MPP part,
we always continue in the per-HTLC loop if we call the `fail_htlc`
macro, thus its nice to actually do the `continue` therein rather
than at the callsites.
If we add an entry to `claimable_payments` we have to ensure we
actually accept the HTLC we're considering, otherwise we'll end up
with an empty `claimable_payments` entry.
