We now support splicing on public channels: once the splice transaction
is confirmed and locked on both sides, nodes will exchange announcement
signatures that allows them to create a `channel_announcement` that they
then broadcast to the network.
This requires reworking the data model to include the announcement and
the real `short_channel_id` in each commitment, which lets us cleanly
distinguish real `short_channel_id`s from aliases (which are set at the
channel level regardless of the current commitments).
The flow now becomes:
- when the funding transaction of a commitment confirms, we set the
corresponding real `short_channel_id` in that commitment
- if the channel is public and we've received `channel_ready` or
`splice_locked`, we send our `announcement_signatures`
- if we receive `announcement_signatures` for a commitment for which
the funding transaction is unconfirmed, we stash it and replay it
when the transaction confirms
- if we receive `announcement_signatures` for a confirmed commitment,
and we don't have a more recently announced commitment, we generate
a `channel_announcement`, store it with the commitment and update
our router data
When creating a `channel_update` for a public channel, we always use
the `short_channel_id` that matches the latest announcement we created.
This is very important to guarantee that nodes receiving our updates
will not discard them because they cannot match it to a channel.
For private channels, we stop allowing usage of the `short_channel_id`
for routing: `scid_alias` MUST be used, which ensures that the channel
utxo isn't revealed.
Note that when migrating to taproot channels, `splice_locked` will be
used to transmit nonces for the announcement signatures, which will be
compatible with the existing flow (and similarly, `channel_ready` will
be used for the initial funding transaction). They are retransmitted
on reconnection to ensure that the announcements can be generated.
Now that we wait for at least 6 confirmations before considering a
channel confirmed, we can simplify our channel announcement logic.
Whenever a channel reaches the confirmed state, it can be announced
to the network (if nodes wish to announce it). We thus don't need
the "deeply buried" state and the "temporary" scid anymore.
The logic is much simpler to follow: when the channel confirms, we
internally update the real scid to match the confirmed funding tx
and send our `announcement_signatures`. When we receive our peer's
`announcement_signatures`, we stash them if the funding tx doesn't
have enough confirmations yet, otherwise we announce the channel and
create a new `channel_update` that uses the real scid.
Whenever we create a `channel_update`, we simply look at whether the
channel is announced or not to choose which scid to use.
This will make it much simpler to announce splice transactions, which
don't need a "deeply buried" state either and will instead simply rely
on whether the splice transaction is confirmed or not to generate
`announcement_signatures`.
We previously supported sending arbitrary trampoline payments in eclair,
which added a bunch of complexity to many of our payment components.
This was necessary because the first version of Phoenix was based on the
eclair codebase. But Phoenix isn't based on eclair anymore and is now
using https://github.com/acinq/lightning-kmp, which is the library that
mobile wallets should use.
Eclair is only meant to be used for server nodes that relay payments and
have access to the full network graph, so it doesn't make sense ever to
send trampoline payments (full source-routing will always be better). We
thus refactor our trampoline client code to only be used for tests (to
ensure that trampoline relay and trampoline receive are implemented
correctly). We isolate the test payment lifecycle in a dedicated actor
(`TrampolinePaymentLifecycle`) which simplifies other payment components
(such as `PaymentInitiator`).
This refactoring will make it easier to support the official version of
trampoline while maintaining backwards-compatibility for older Phoenix
wallets that use the non-official trampoline version.
* Add `on_the_fly_funding` feature bit and messages
Add the (disabled by default) `on_the_fly_funding` feature bit and
codecs for the corresponding messages:
- `will_add_htlc`
- `will_fail_htlc`
- `will_fail_malformed_htlc`
- `cancel_on_the_fly_funding`
We also add a TLV to `update_add_htlc` to notify the recipient that we
relayed less data than what the onion encodes, in exchange for the fees
of the specified funding transaction.
* Add `non_initiator_pays_commit_fees` channel flag
We add a non-standard channel flag to `open_channel2` to allow wallets
to ask their peer to pay the commit tx fees, even when they're not the
channel opener. This is necessary for on-the-fly funding, until we can
move to 0-fee commit txs which will make it obsolete.
* Allow underpaying feerate when using future HTLCs
When an interactive-tx session is created for a liquidity purchase that
uses future HTLCs to pay fees, the initiator may not have enough funds
to honor the target feerate. We allow the transaction anyway, because
we want to get paid for the liquidity we're providing. If the feerate
is too low and the transaction doesn't confirm, we can double-spend it
if we need that liquidity elsewhere.
* Add `funding_fee` field to `CMD_ADD_HTLC`
This commit adds the funding fee field to HTLCs, but never sets it.
We update a lot of test files, but there is no functional change.
* Implement on-the-fly funding
Implement the on-the-fly funding protocol: when a payment cannot be
relayed because of a liquidity issue, we notify the `Peer` actor that
we'd like to trigger on-the-fly funding if available. If available, we
we send a funding proposal to our peer and keep track of its status.
Once a matching funding transaction is signed, we persist this funding
attempt and wait for the additional liquidity to be available (once the
channel is ready or the splice locked). We will then frequently try to
relay the payment to get paid our liquidity fees. If the payment keeps
getting rejected, or we cannot connect to our peer, we abandon the
payment when it reaches its CLTV expiry, which ensures that the upstream
channels are not at risk.
When using on-the-fly funding, we use a single channel with our peer.
If they try to open another channel while one is available, we reject
their request and expect a splice instead.
The channel initiator traditionally pays the commit tx fees, but we may
want to override that when providing services to wallet users. We thus
split the current `isInitiator` flag into two flags:
- `isChannelOpener`
- `paysCommitTxFees`
We always set `paysCommitTxFees` to the same value as `isChannelOpener`.
Custom feature bits may override that behavior if necessary.
Note that backwards compatibility is preserved since our previous `bool8`
codec encodes `true` as `0xff` and `false` as `0x00`.
To save space in the offer, we can skip the node id for offers that use blinded paths.
The node id used to sign the invoice will be the last blinded node id of the path used to request the invoice.
We also make the description optional for offers without amount.
Allow sending messages to self
Fixes corner cases caused by compact encoding of node ids. Every message to be relayed now follows the same path and `MessageRelay` can relay to self.
When opening an important channel we may be willing to pay a high feerate that we wouldn't want to use when consolidating UTXOs. However because we delegate the transaction creation to bitcoin core, we can sometimes be surprised by funding transactions that are much bigger than expected. This PR protects us from such cases by refusing to open the channel if the transaction fee is too high.
Based on original work from @thomash-acinq.
For fighting jamming attempts, or even just to detect one, we need to know how fast relayed HTLCs are fulfilled. We now measure this and store it in the audit database. Previously the "IN" and "OUT" directions for the same HTLC were storing the same timestamp (corresponding to when the HTLC is fulfilled), we now use the timestamp at which we received the UpdateAddHtlc for the "IN" direction.
We add a `cpfpbumpfees` API that lets node operators bump the fees
of a package of unconfirmed transactions.
Node operators can for example ensure their funding txs confirm before
they hit the `2016` funding timeout. It's also very useful when you have
a long chain of unconfirmed funding transactions and/or mutual close
transactions and want to bump them all at once.
NB: the node operator needs to figure out which outpoints belong to him
(which should be fairly easy using existing APIs).
We now reject onion message payloads that contain unexpected fields and classify final payloads as being either an invoice request, an invoice response, an error or an invalid payload.
Each of these cases are mutually exclusive, it is not allowed to send both an invoice request and an invoice at the same time for instance.
Invalid payloads are not dropped immediately so that if they are the response we were waiting for, we can stop waiting and return an error without retrying.
Main behavior changes (see commit messages for details):
- channel opening errors are returned with a 200/OK status from the api
- we return a success in the case of dual-funding or rbf, if the interactive tx has completed, even if the publish fails
- for rbf, we send the success response later in the flow: only when the rbf flow is successful, as opposed to when we initiate it
This is a prerequisite to splices, but also a first step towards reworking the channel request/response mechanism.
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
The order of the elements in a TLV stream is an implementation detail that will disappear with serialization. Equality between TlvStream shouldn't depend on this order.
For that we use `Set`s instead of `Iterable`s.
* Add ChannelOpened event
Emit an event when a channel is opened and ready to process payments.
This event is distinct from the `ChannelCreated` event which is sent
earlier, once we think a funding transaction has been successfully created
(but cannot guarantee when we're not the initiator).
* Add event to websocket
This is a breaking change, but it should be ok since the previous event
wasn't reliable: it was emitted at a time where we couldn't guarantee
that the channel would really confirm.
We add an actor that waits for a given peer to be connected and ready to
process payments. This is useful in the context of async payments for
the receiver's LSP.
When sending an outgoing multi-part payment, we forward the preimage back
to the sender as soon as we receive the first `update_fulfill_htlc`.
This is particularly useful when relaying trampoline payments, to be able
to propagate the fulfill upstream as early as possible.
However this meant that callers of the HTTP API would receive this
preimage event instead of the final payment result, which was quite bad.
We now disable this first event when used with the `--blocking` argument,
which ensures that the API always returns the payment result.
Fixes#2389
We were previously directly creating onion payloads inside the various
payment state machines and manipulating tlv fields. This was a layering
violation that was somewhat ok because in most cases we only needed to
create the onion payload for the recipient at the beginning of the payment
flow and didn't need to modify it, except for a small change in the MPP
case.
This forced us to handle trampoline onions directly in the payment
initiator and will not work for blinded payments, where we can only build
the onion payload for the recipient after we've chosen the routes and how
to split the amount.
We clean this up by introducing payment recipients that abstract away the
creation of onion payloads. This makes it much easier to integrate blinded
payments. It also allows us to clean up the way we do trampoline payments
and potentially support splitting across multiple trampoline routes (not
included in this PR as this change isn't immediately needed).
It also lets us simplify the MultiPartPaymentLifecycle FSM, by moving the
logic of computing how much remains to be sent and what fee can be used
to the route calculation component.
We previously only supported `scid_alias` and `zero_conf` when used in
combination with `anchors_zero_fee_htlc_tx`, but we have no way of
advertising that to our peers, which leads to confusing failures when
opening channels.
Some nodes that don't have good access to a utxo pool may not migrate to
anchor outputs, but may still want to use `scid_alias` (and potentially
`zero_conf` as well).
Fixes#2394
Add a message flag to channel update to specify that an update is private
and should not be rebroadcast to other nodes.
We log an error if a private channel update gets into the rebroadcast list.
See https://github.com/lightning/bolts/pull/999
When creating a blinded route, we expose the last blinding point (that the
last node will receive). This lets the recipient derive the corresponding
blinded private key, which they may use to sign an invoice.
We add support for generating Bolt 12 invoices and storing them in our
payments DB to then receive blinded payments.
We implement the receiving part once a blinded payment has been decrypted.
This uses the same payment flow as non-blinded payments, with slightly
different validation steps.
Note that we put a random secret in the blinded paths' path_id field
to verify that an incoming payment uses a valid blinded route generated
by us. We store that as an arbitrary byte vector to allow future changes
to this strategy.
* Separate tlv decoding from content validation
When decoding a tlv stream, we previously also validated the
stream's content at decoding time. This was a layer violation,
as checking that specific tlvs are present in a stream is not
an encoding concern.
This was somewhat fine when we only had very basic validation
(presence or absence of specific tlvs), but blinded paths
substantially changed that because one of the tlvs must be
decrypted to yield another tlv stream that also needs to have
its content validated.
This forced us to have an overly complex trait hierarchy in
PaymentOnion.scala and expose a blinding key in classes that
shouldn't care about whether blinding is used or not.
We now decouple that into two distinct steps:
* codecs simply return tlv streams and verify that tlvs are
correctly encoded
* business logic case classes (such as ChannelRelayPayload)
should be instantiated with a new `validate` method that
takes tlv streams and verifies mandatory/forbidden tlvs
This lets us greatly simplify the trait hierarchy and deal
with case class that only contain fully decrypted and valid
data.
* Improve tests
There was redundancy in the wrong places: route blinding codec tests were
testing route blinding decryption and were missing content validation.
We also change the behavior of the route blinding decode method to return
the blinding override when present, instead of letting higher level
components duplicate that logic.
* Use hierarchical namespaces
As suggested by @pm47
* Small PR comments
* Remove confusing comment
When not provided, we previously used unrestricted `from` and `to`
parameters when reading from the DB. This can create issues when
accidentally reading too much data.
We now limit this to the last 24 hours, unless explicitly set by the
caller. Note that when reading a lot of data, it's recommended to use
postgres and read from a replica to ensure there's no impact on the running
eclair instance.
Fixes#2383
It is now possible to specify a DNS host name as one of your
`server.public-ips` addresses.
DNS host names will not be resolved until eclair attempts to
connect to the peer.
See https://github.com/lightning/bolts/pull/911
The specification is removing support for old channel updates that didn't
include an `htlc_maximum_msat` (https://github.com/lightning/bolts/pull/996).
Every implementation has been generating updates containing this field for
years, so we can safely reject updates that don't contain it.
We now use either our local alias or the real scid in the channel update
that we store internally. When we send that channel update directly to
our peer, we override it to use the remote alias when it makes sense.
We make it explicit that conversion from a string to an scid uses the tx
coordinates and can fail.
We also clean up compiler warnings in updated test files.
This PR does two main things:
- introduce a new `FixtureSpec` base class for tests that involve a fixture. See the scaladoc for more info.
- add new simple integration tests in package `integration.basic`. They are based on `MinimalNodeFixture`, which is a full setup for a node with real actors, except the bitcoin part (watcher/wallet) which is mocked. They are much lighter than our previous integration tests, which allow us to keep each test individual, as opposed to having all tests of the same suite depend on each other. We can define more complex fixtures with any number of nodes.
Other minor improvements:
- update scalatest version
- simplify `ChannelStateTestsHelperMethods`
- replace all === by ==
Triple equals allows customizing the equality method, but we never use
that.
Detailed errors (which is what we were looking for) are provided by
mixing in scalatest's `Assertions` and using the regular `assert(x==y)`
syntax. The `Assertions` trait is already mixed in by default in all
scalatest suites.
This is easier to use than having to decide which params we should look
into (local or remote). It will also be easier to integrate with dual funding.
Rename initialFeeratePerKw: this name was very confusing.
This feerate only applies to the commit tx, so we make that explicit.
We have two sources for channel routing parameters:
- channel updates
- routing hints in invoices
Instead of generating fake `channel_update`s from routing hints, we
define a `ChannelRelayParams` that can be built from announcements
or routing hints.
This is cleaner but also is a first step to decorrelate the identifier
of a channel in our graph, from whatever identifies the source.
* Add a "stop" API method
This API call was added for certain uses cases where killing the process was impractical but internally it just calls `sys.exit()`.
Eclair is designed to shutdown cleanly when its process is killed and this is still the recommended way of stopping it.
We use a version of bitcoin-lib that uses bitcon-kmp under the hood.
Its API is the same as before, but its package is now fr.acinq.bitcoin.scalacompat
The `channelbalances` API call retrieves information about the balances of all local channels, not just those with usable outgoing balances that are enabled for sending.
This change also adds the `isEnabled` attribute to the json results for both the new `channelbalances` and old `usablebalances` API calls.
* use `NodeAddress` everywhere instead of `InetAddress`
This makes us control more strictly when and where name resolution happens, which is important in a security-hardened setup. The `InetAddress` jdk class indeed does a lot of things behind the scenes, but now we restrict it to tcp-related classes like `Client` and `Server`.
Also, in _cluster mode_ all outgoing connections (including tor) are now made on the front.
* upgrade guava and rewrite nodeuri tests
There is no reason to use the version of guava targetting android anymore. Also `HostAndPort` was in beta in our current lib version.
We now use guava's `InetAddresses.toUriString()` to format host string, instead of manually adding brackets.
Reworked `NodeURI` tests:
- less repetition with one single test and multiple `testCases`
- focus on non-reg (no need to verify what we know we don't support)
An invoice that doesn't explicitly set min_final_cltv_expiry has an default min_final_cltv_expiry of 18.
This default allows invoices to be slightly shorter but does not mean that the min_final_cltv_expiry can be unknown and needs to be provided from somewhere else.
Refactoring of `PaymentRequest` to pave the way for bolt12 invoices
`PaymentRequest` is renamed to `Invoice` and is now a trait extended by `Bolt11Invoice` and `Bolt12Invoice`. `Bolt12Invoice` will come in a future PR.
Co-authored-by: t-bast <bastuc@hotmail.fr>
* consistency in feature names
Since we're not always following the `option_` prefix from the spec,
let's at least be consistent with ourselves.
* add alternative to `def nodeFees()`
We now have this better type to remove ambiguity.
We should use it wherever it makes sense.
There shouldn't be any business logic change in this commit.
This effectively reverts #2112 and implements it differently.
Having a single very conservative max value is simpler and the risk of regression is very low. Also fixed a wrong comment, previous max wasn't `11/04/2262` but a huge value.
