* Return unwrapped decryption failure onion
When we fail to decrypt an onion failure packet, we should return the
result of the unwrapping process. When using trampoline, this will let
us properly re-encrypt the failure and relay it upstream to the previous
trampoline node, until it reaches the sender.
* Refactor HTLC failure creation
We refactor the shared secret extraction to a dedicated function.
* Refactor HTLC failure reason
We previously used an `Either[ByteVector, FailureMessage]` to encode:
- a downstream error that we couldn't decrypt and must re-wrap (left)
- a local error that we must encrypt (right)
This won't be sufficient for trampoline, because we will need to handle
the following cases:
- a downstream error that we couldn't decrypt and must re-wrap
- a local error for the node who created the *outer* onion (which we
encrypt with the sphinx shared secret of the outer onion)
- a local error for the node who created the *trampoline* onion (which
we encrypt with the sphinx shared secret of the trampoline onion and
then with the shared secret of the outer onion)
We thus introduce a trait, which currently only contains the first two
cases. We will extend this trait when adding support for trampoline
failures. This is a pure refactoring without any behavior changes so
far, which will simplify the future trampoline changes.
* Include trampoline onion in final trampoline payloads
When we receive a (non-blinded) payment that uses trampoline, we keep
the trampoline onion to be able to distinguish this payment from a
non-trampoline payment.
When a channel force-closes, we publish our commit tx and HTLC txs.
HTLC transactions conflict with our peer's transactions that also spend
the HTLC outputs. If our peer is able to get their transaction confirmed
before ours, we should stop retrying to publish our HTLC transaction as
that will never succeed.
Since we didn't check the output status, we kept retrying until the
channel was closed (which requires waiting for the `to_self_delay`).
The retries always fail at funding time: `bitcoind` returns an error
saying that the UTXO cannot be found (which is expected because it has
already been spent by our peer). This creates a lot of unnecessary
retries and a lot of noise in the logs.
This scenario usually happens when our peer didn't send the preimage
before force-closing the channel, but reveals it on-chain before the
HTLC timeout: when that happens we kept retrying to publish our HTLC
timeout transaction, which cannot succeed.
We now check the output status in our publishing preconditions, and
immediately abort if the output has already been spent by a confirmed
transaction.
* Fix offer description documentation
And remove the `currency` fields as we have no short-term plans to
support currency conversion in `eclair`.
* Relax `payment_constraints` requirement in final blinded payload
We don't always need to include a `payment_constraints` field for
ourselves: it's fine to accept payment that don't contain one as
long as we created the `encrypted_recipient_data`, which we can
verify using the `path_id`. We were too restrictive for no good
reason.
* Allow omitting `total_amount` in blinded payments
If the `total_amount` field isn't provided, we can safely default to
using the `amount`, which saves space in the onion. Note that we keep
always encoding it in the outgoing payments we send, we're simply more
permissive when receiving payments.
* Refactor `decryptEncryptedRecipientData`
We extract a helper method for decrypting encrypted recipient data
which will be used when decrypting trampoline blinded paths.
* Use relay methods in `PaymentOnion.IntermediatePayload.NodeRelay`
In order to support blinded trampoline payments, we won't have access to
a direct `amount_to_forward` field, but will use a `payment_relay` TLV
instead, which only allows calculating the outgoing amount from the
incoming amount (same thing for the expiry).
We refactor this to simplify the diff when introducing blinded trampoline
payments.
We refactor trampoline-to-legacy payments to use a dedicated class, like
what we do for trampoline-to-blinded-paths payments. This allows us to
supports two encodings for those payments:
- one where the trampoline onion contains a dummy payload for the
recipient that must be ignored (current Phoenix wallets), which
wastes space in the onion for legacy reasons
- one where we don't include a dummy payload for the recipient, which
is more efficient and similar to trampoline-to-blinded-paths
When we're the introduction node of a trampoline blinded path, we
previously only allowed our custom `wallet_node_id` format when a
`short_channel_id` was not included. But we actually can allow plain
`node_id`s as well, as we only need to know the public key.
We've rejected some payments in the past months because they included an
`outgoing_node_id` that didn't use the wallet format: by removing that
limitation we ensure that those payments will be correctly relayed in
the future.
We previously supported sending arbitrary trampoline payments in eclair,
which added a bunch of complexity to many of our payment components.
This was necessary because the first version of Phoenix was based on the
eclair codebase. But Phoenix isn't based on eclair anymore and is now
using https://github.com/acinq/lightning-kmp, which is the library that
mobile wallets should use.
Eclair is only meant to be used for server nodes that relay payments and
have access to the full network graph, so it doesn't make sense ever to
send trampoline payments (full source-routing will always be better). We
thus refactor our trampoline client code to only be used for tests (to
ensure that trampoline relay and trampoline receive are implemented
correctly). We isolate the test payment lifecycle in a dedicated actor
(`TrampolinePaymentLifecycle`) which simplifies other payment components
(such as `PaymentInitiator`).
This refactoring will make it easier to support the official version of
trampoline while maintaining backwards-compatibility for older Phoenix
wallets that use the non-official trampoline version.
Bitcoin Core 27.2 is out with some bug fixes. None of the bugs fixed
seem to impact us, but there's always a possibility that a security
issue was also fixed, so it's better to upgrade!
The balance estimates are sometimes outside the range [0, 1] which should never happen. We add some logs to help debug it.
We also stop calling `TimestampSecond.now()` multiple times during the computation and reuse the same value which may fix the issue.
The codec that we were using to serialize `LightningMessage`s in the eclair cluster didn't support `UnknownMessage`. This resulted in those messages being dropped by the front and never reaching the backend node.
If a payer is buggy and tries to pay the same invoice multiple times, it can lead to an edge case where the recipient accepted the first one and purchased liquidity for it, but didn't purchase additional liquidity and thus cannot receive the duplicate payments.
Also, when replaying parts that were waiting for an on-the-fly funding, we set `commit = false` to all parts, instead of just the last one. This optimization caused the payment to be stuck if the last part was unexpectedly rejected (which would for example happen in the buggy payer case described above, before we rejected those extra payments).
---------
Co-authored-by: t-bast <bastien@acinq.fr>
If the latest splice transaction doesn't confirm, we allow exchanging
`tx_init_rbf` and `tx_ack_rbf` to create another splice transaction to
replace it. We use the same funding contribution as the previous splice.
When 0-conf isn't used, we reject `splice_init` while the previous
splice transaction hasn't confirmed. Our peer should either use RBF
instead of creating a new splice, or they should wait for our node
to receive the block that confirmed the previous transaction. This
protects against chains of unconfirmed transactions.
When using 0-conf, we reject `tx_init_rbf` and allow creating chains of
unconfirmed splice transactions: using RBF with 0-conf can lead to one
side stealing funds, which is why we prevent it.
If our peer was buying liquidity but tries to cancel the purchase with
an RBF attempt, we reject it: this prevents edge cases where the seller
may end up adding liquidity to the channel without being paid in return.
When providing on-the-fly funding with the `from_future_htlc` payment
type, the liquidity provider is paying mining fees for the funding
transaction while trusting that the remote node will accept the HTLCs
afterwards and thus pay a liquidity fees. If the remote node fails the
HTLCs, the liquidity provider doesn't get paid. At that point it can
disable the channel and try to actively double-spend it. When we detect
such behavior, we immediately disable `from_future_htlc` to limit the
exposure to liquidity griefing: it can then be re-enabled by using the
`enableFromFutureHtlc` RPC, or will be automatically re-enabled if the
remote node fulfills the HTLCs after a retry.
We record whether a liquidity purchase happened as part of every funding
transaction with the purchased amount and fees. We don't record how the
payment was made, because:
- it creates more backwards-compatibility risk if the liquidity ads spec
changes
- it's most likely unnecessary once the purchase happened, and can only
be stored in the liquidity DB for auditing
We initially supported splicing with a poor man's quiescence, where we
allowed splice messages if the commitments were already quiescent.
We've shipped support for quiescence since then, which means that new
even nodes relying on experimental splicing should support quiescence.
We can thus remove support for the non-quiescent version.
Independently of target and tolerance ratios, transactions must be
publishable in our local mempool. We must be careful that fee ranges
stay above this min fee level.
In theory we shouldn't need to care about the upper range, but I'm not
sure that, depending on the fee estimator, the min fee is always lower
than arbitrary block targets.
We add an optional feature that lets on-the-fly funding clients accept
payments that are too small to pay the fees for an on-the-fly funding.
When that happens, the payment amount is added as "fee credit" without
performing an on-chain operation. Once enough fee credit has been
obtained, we can initiate an on-chain operation to create a channel or
a splice by paying part of the fees from the fee credit.
This feature makes more efficient use of on-chain transactions by
trusting that the seller will honor our fee credit in the future. The
fee credit takes precedence over other ways of paying the fees (from
the channel balance or future HTLCs), which guarantees that the fee
credit eventually converges to 0.
Co-authored-by: Pierre-Marie Padiou <pm47@users.noreply.github.com>
* Add `on_the_fly_funding` feature bit and messages
Add the (disabled by default) `on_the_fly_funding` feature bit and
codecs for the corresponding messages:
- `will_add_htlc`
- `will_fail_htlc`
- `will_fail_malformed_htlc`
- `cancel_on_the_fly_funding`
We also add a TLV to `update_add_htlc` to notify the recipient that we
relayed less data than what the onion encodes, in exchange for the fees
of the specified funding transaction.
* Add `non_initiator_pays_commit_fees` channel flag
We add a non-standard channel flag to `open_channel2` to allow wallets
to ask their peer to pay the commit tx fees, even when they're not the
channel opener. This is necessary for on-the-fly funding, until we can
move to 0-fee commit txs which will make it obsolete.
* Allow underpaying feerate when using future HTLCs
When an interactive-tx session is created for a liquidity purchase that
uses future HTLCs to pay fees, the initiator may not have enough funds
to honor the target feerate. We allow the transaction anyway, because
we want to get paid for the liquidity we're providing. If the feerate
is too low and the transaction doesn't confirm, we can double-spend it
if we need that liquidity elsewhere.
* Add `funding_fee` field to `CMD_ADD_HTLC`
This commit adds the funding fee field to HTLCs, but never sets it.
We update a lot of test files, but there is no functional change.
* Implement on-the-fly funding
Implement the on-the-fly funding protocol: when a payment cannot be
relayed because of a liquidity issue, we notify the `Peer` actor that
we'd like to trigger on-the-fly funding if available. If available, we
we send a funding proposal to our peer and keep track of its status.
Once a matching funding transaction is signed, we persist this funding
attempt and wait for the additional liquidity to be available (once the
channel is ready or the splice locked). We will then frequently try to
relay the payment to get paid our liquidity fees. If the payment keeps
getting rejected, or we cannot connect to our peer, we abandon the
payment when it reaches its CLTV expiry, which ensures that the upstream
channels are not at risk.
When using on-the-fly funding, we use a single channel with our peer.
If they try to open another channel while one is available, we reject
their request and expect a splice instead.
We send to our peers an optional message that tells them the feerates
we'd like to use for funding channels. This lets them know which values
are acceptable to us, in case we reject their funding requests.
This is using an odd type and will be automatically ignored by existing
nodes who don't support that feature.
Co-authored-by: Pierre-Marie Padiou <pm47@users.noreply.github.com>
* Add support for extensible liquidity ads
The initiator of `open_channel2`, `tx_init_rbf` and `splice_init` can
request funding from the remote node. The non-initiator node will:
- let the open-channel-interceptor plugin decide whether to provide
liquidity for new channels or not, and how much
- always honor liquidity requests on existing channels (RBF and splice)
when funding rates have been configured
Liquidity ads are included in the `node_announcement` message, which
lets buyers compare sellers and connect to sellers that provide rates
they are comfortable with. They are also included in the `init` message
which allows providing different rates to specific peers.
This implements https://github.com/lightning/bolts/pull/1153. We
currently use the temporary tlv tag 1339 while we're waiting for
feedback on the spec proposal.
* Add `channelCreationFee` to liquidity ads
Creating a new channel has an additional cost compared to adding
liquidity to an existing channel: the channel will be closed in the
future, which will require paying on-chain fees. Node operators can
include a `channel-creation-fee-satoshis` in their liquidity ads to
cover some of that future cost.
* Add liquidity purchases to the `AuditDb`
Whenever liquidity is purchased, we store it in the `AuditDb`. This lets
node operators gather useful statistics on their peers, and which ones
are actively using the liquidity that is purchased.
We store minimal information about the liquidity ads itself to be more
easily compatible with potential changes in the spec.
When nodes only have private channels, they must include routing hints
in their Bolt 11 invoices to be able to receive payments. We add a
parameter to the `createinvoice` RPC for this. Note that this may leak
the channel outpoint if `scid_alias` isn't used.
Fixes#2802
We have a couple of such errors in our logs for blinded payment relay,
and we currently don't know what the sender did wrong because we don't
log the faulty TLV tag. This should make it easier to debug.
Older lnd nodes are unable to mutual close channels while HTLCs are
in-flight. This triggers unexpected force-closes which can be costly
for node operators. We ignore the errors coming from lnd that match
this case to avoid that.
* Update CI test with latest bitcoin core (switch from autotools to cmake)
bitcoin core now uses cmake instead of autotools.
CI test is triggered by a cron job but can now also be triggered manually.
* Don't spawn anchor tx publisher if commit is confirmed
It is inefficient to spawn a tx publisher for anchor txs if we already
know that the commit tx is confirmed: we will make calls to our bitcoin
node that can easily be avoided. This can matter when force-closing a
large number of channels with frequent disconnections (e.g. wallets).
* Improve `TxTimeLocksMonitor` performance
When publishing a transaction that has CSV delays, we previously used
the watcher and set a `minDepth` on the parent transaction matching
the CSV delay of the child transaction. While this was very simple,
it was unnecessarily expensive for large CSV delays: the watcher would
check for tx confirmations at every block, even when the CSV delay is
very large. When we force-close a large number of channels, it results
in a very large number of RPC calls to our `bitcoind` node.
We don't use the watcher in the `TxTimeLocksMonitor` anymore: instead
we check the parent confirmations once, and then we check again after
the CSV delay.
* Add relative delay hints to `ZmqWatcher`
When we tell the `ZmqWatcher` to watch for confirmations on transactions
that have a relative delay, it is highly inefficient to call our bitcoin
node at every new block to check for confirmations (especially when the
parent transaction isn't even confirmed). We now tell the watcher about
the relative delay, which lets it check for confirmations only at block
heights where we expect the transaction to reach its minimum depth. This
is especially useful to improve performance for delayed transactions
that usually use a CSV of at least 720 blocks.
We refactor `NodeRelay.scala` to re-order some steps. The steps are:
1. Fully receive the incoming payment
2. Resolve the next node (unwrap blinded paths if needed)
3. Wake-up the next node if necessary (mobile wallet)
4. Relay outgoing payment
Note that we introduce a wake-up step, that can be extended to include
mobile notifications. We introduce that same wake-up step in channel
relay and message relay. We also allow relaying data to contain a wallet
`node_id` instead of an scid. When that's the case, we start by waking
up that wallet node before we try relaying onion messages or payments.
This wake-up step doesn't contain any logic right now apart from waiting
for the peer to connect, if it isn't connected already. But it can easily be
extended to send a mobile notification to prompt the wallet to connect.
* Reject new static_remote_key channels
We will still load and use existing static_remote_key channels, we can still open static_remote_key channels, but we will not accept new static_remote_key channels.
This behaviour can be overridden by setting `eclair.channel.accept-incoming-static-remote-key-channels` to `true`.
* Reject new obsolete incoming channels
We reject new incoming channels that don't even support `option_static_remotekey` (which is assumed to be on in the BOLTs).
Unit tests have been modified to use static_remote_key or anchor channels (default used to be the obsolete "standard" channel).
---------
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
There weren't any logs when relaying onion messages, which makes it
impossible to troubleshoot. While we're working on cross-compatibility,
we keep those logs at the `INFO` level, and will change some of them to
`DEBUG` once stabilized.
We move the `Upstream` trait closer to the `Origin`, and make it more
obvious than a hot `Origin` is:
- an `Upstream` referencing the upstream HTLCs
- an actor requesting the outgoing payment
We also improve the cold trampoline relay class to record the incoming
HTLC amount, which we previously didn't bother encoding but is useful to
compute the fees collected during relay. To ensure backwards-compat, it
is set to `0 msat` for pending HTLCs. It will only affect HTLCs that
were pending during the upgrade, which is acceptable.
When we require inputs to be confirmed, we can reliably check whether
they are unspent. We can't reliably check this for unconfirmed inputs,
because they could be valid but simply not in our mempool, in which
case bitcoind would incorrectly consider them unspendable.
We want to reject unspendable inputs early to immediately fail the
funding attempt, instead of waiting to detect the double-spend later.
