When opening an important channel we may be willing to pay a high feerate that we wouldn't want to use when consolidating UTXOs. However because we delegate the transaction creation to bitcoin core, we can sometimes be surprised by funding transactions that are much bigger than expected. This PR protects us from such cases by refusing to open the channel if the transaction fee is too high.
Based on original work from @thomash-acinq.
This version of logback fixes the following CVE:
"a potential denial of service attack on a centralized logback receiver
when a third party controlling a remote appender connects to said
receiver and could shut down or slow down logging of events."
Eclair isn't affected since we don't use logback receivers, but if there
are applications or plugins that depend on eclair and use logback
receivers, it's better to use the logback version containing the fix.
* Make Eclair manage bitcoin core's wallet private keys
We create an empty watch-only wallet and import public descriptors generated by Eclair.
Bitcoin Core can fund transaction and manage utxos, but can no longer sign transactions.
* Check that spent amounts and utxos are consistent before we sign a PSBT
PSBT utxo fields include the amount that are being spent by the PSBT inputs, but there is a "fee attack"
where using amounts that are lower than what is actually spent may make us sign a tx that spends much more
in fees than we think.
* Check that non-segwit uxto has been provided and inputs are signed with SIGHASH_ALL
* Verify that Bitcoin Core's fee match what we specified
When we call Bitcoin Core's `fundrawtransaction` RPC method, we check that the fee that we pay match the fee rate that we requested.
The fee is computed using the utxo information that Bitcoin Core adds to our PSBT before we sign it.
We can safely used this information because if Bitcoin Core lies about the value of the inputs that we're spending then the signature we produce will also not be valid (it commits to the value being spent).
When we're adding wallet inputs to "bump" the fees of a parent transaction we need to take the whole package into account when we verify the actual fee rate, which is why some internal methods were modified to return the package weight that was used as reference when `fundrawtransaction` was called.
* Check that fundrawtransaction does not add more than 1 change output
* Validate addresses and keys generated by bitcoin core
When eclair manages private keys, make sure that we can re-compute addresses and keys
generated by bitcoin core.
* Add a separate configuration file for Eclair's onchain signer
Eclair's onchain signer now has its own `eclair-signer.conf` configuration file in HOCON format.
It includes BIP39 mnemonic codes and passphrase, a wallet name and a timestamp.
When an `eclair-signer.conf` file is found, Eclair's API will return descriptors that can be imported into an
empty watch-only Bitcoin Wallet.
When wallet name in `eclair-signer.conf` matches the name of the Bitcoin Core wallet defined in `eclair.conf`
(`eclair.bitcoind.wallet`), Eclair will bypass Bitcoin Core and sign onchain transactions directly.
* Skip validation of local non-change outputs:
Local non-change outputs send to an external address, for splicing out funds for example.
We assume that these inputs are trusted i.e have been created by a trusted API call and our local
onchain key manager will not validate them during the signing process.
* Document why we use a separate, specific file for the onchain key manager
Using a new signer section is eclair.conf would be simpler but "leaks" because it becomes available everywhere
in the code through the actor system's settings instead of being contained to where it is actually needed
and could potentially be exposed through a bug that "exports" the configuration (through logs, ....)
though this is highly unlikely.
* Additional changes to delegate bitcoin core keys to eclair (#2726)
Refactor the `BitcoinCoreClient` and `LocalOnChainKeyManager` to:
- rely less on exceptions
- use more idiomatic scala (reduce dependency on kotlin types)
- provide more detailed logs
We also simplify the `useEclairSigner` field in `BitcoinCoreClient`.
The complexity of handling the case where there was an on-chain key
manager but for a different wallet than the one configured isn't
something that should be used, so it wasn't worth supporting.
Some checks were inconsistent and are now unified:
- checking the exact `scriptPubKey` in our outputs in TODO and TODO
- we allow using `fundTransaction` with a tx that already includes a
change output (which may happen when RBF-ing a transaction)
- `getP2wpkhPubkeyHashForChange` didn't verify the returned key
We completely separate the two cases in `signPsbt`, because otherwise
in the non eclair-backed case, we were calling bitcoind's `processpsbt`
twice for no good reason, which is bad for performance.
We also decouple the `OnChainKeyManager` from the `BitcoinCoreClient`.
This lets users keep running their eclair node with a bitcoin client that
owns the private key while configuring the on-chain key manager for a
future bitcoin client that will leverage this on-chain key manager.
Users can use the eclair APIs to get the master xpub and descriptors to
properly configure their next bitcoin core node, and switch to it once it
has synchronized the descriptors.
* Simplify replaceable tx funding
We were previously signing twice (with makes a call to `bitcoind`),
just to get the final weights and adjust the change outputs. This was
unnecessary, as we can adjust the weights before adding inputs.
We were also duplicating the checks where we verify that `bitcoind` is
malicious. We only need to check that once, during the final signing step.
---------
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
Node operators may disable automatic fee-bumping on their local commit if
they don't have anything at stake (no pending HTLCs), which saves fees in
most cases. A drawback in that case is that if the commitment doesn't
confirm quickly enough, the remote's funds are also locked.
This can be an issue for LSPs, where the remote peer doesn't have a good
ability to fee-bump commit txs. We give more control to the node operator
by letting them fee-bump local commit txs explicitly through the RPC to
unblock wallet users funds.
* Update code dependencies
I also wanted to update logback, but I'm hitting issues because of our
custom logger in `FixtureSpec` (`LoggingEventAware` not found).
* Update build dependencies
We're now using mvn 3.9.2 to build eclair, which reports warnings in some
of the build plugins we use. Updating plugins fixes most of the warnings,
and the remaining warnings have to be fixed by the plugins themselves
to support mvn 4.x.
For fighting jamming attempts, or even just to detect one, we need to know how fast relayed HTLCs are fulfilled. We now measure this and store it in the audit database. Previously the "IN" and "OUT" directions for the same HTLC were storing the same timestamp (corresponding to when the HTLC is fulfilled), we now use the timestamp at which we received the UpdateAddHtlc for the "IN" direction.
When sending a message, the postman can now ask the router to find a route using channels only.
The same route is also used as a reply path when applicable.
The graph data structure has been updated to include both active and disabled edges.
The graph now contains features for vertices.
This RPC allows to access the historic channel data without
relying on third party services like LN explorers.
Note that when the `remoteNodeId` filter is not provided, this
query may be expensive on nodes with a lot of closed channels.
We add a `cpfpbumpfees` API that lets node operators bump the fees
of a package of unconfirmed transactions.
Node operators can for example ensure their funding txs confirm before
they hit the `2016` funding timeout. It's also very useful when you have
a long chain of unconfirmed funding transactions and/or mutual close
transactions and want to bump them all at once.
NB: the node operator needs to figure out which outpoints belong to him
(which should be fairly easy using existing APIs).
Included doc folder in the assembly zip. ** wildcard indicates all the files in the current folder and all the files in the subfolders of the current folder.
Fixes#1645
We now reject onion message payloads that contain unexpected fields and classify final payloads as being either an invoice request, an invoice response, an error or an invalid payload.
Each of these cases are mutually exclusive, it is not allowed to send both an invoice request and an invoice at the same time for instance.
Invalid payloads are not dropped immediately so that if they are the response we were waiting for, we can stop waiting and return an error without retrying.
Add support for both splice-in and splice-out in Eclair. Mixing concurrent local/remote splice-in/splice-out is wired, although not supported in the API.
The implementation differs from the current wip BOLT proposal on at least the following points:
- we use a poor man's _quiescence_ protocol which just rejects the splice if the channel is not idle
- splice txs always _spend_ the previous funding/splice tx, even if it isn't confirmed yet and could theoretically be RBFed. This is done to be compatible with zero-conf splices
- the persistence/reconnection follows the logic described in https://gist.github.com/t-bast/1ac31f4e27734a10c5b9847d06db8d86.
We add a new `fundingTxIndex` to `Commitment`, which has two nice advantages:
- making debug much easier compared to dealing with txid:
`splice=1 is now active, removed=0 remaining=2,1`
- allowing to discriminate between initial funding, splices, rbf, and
combinations thereof.
We closely mimick RBFing the initial funding tx (e.g. `RbfStatus` vs `SpliceStatus`).
---------
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
Main behavior changes (see commit messages for details):
- channel opening errors are returned with a 200/OK status from the api
- we return a success in the case of dual-funding or rbf, if the interactive tx has completed, even if the publish fails
- for rbf, we send the success response later in the flow: only when the rbf flow is successful, as opposed to when we initiate it
This is a prerequisite to splices, but also a first step towards reworking the channel request/response mechanism.
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
Each RBF attempt adds more data that we need to store and process,
so we want to limit our peers to a reasonable use of RBF.
We send a warning to let them know that they are close to reaching the
limits.
The order of the elements in a TLV stream is an implementation detail that will disappear with serialization. Equality between TlvStream shouldn't depend on this order.
For that we use `Set`s instead of `Iterable`s.
To avoid bash interpreting '&' as background job metacharacter,
properly quote the string containing it.
Otherwise you get output like the following:
eclair-node[3517981]: No java installations was detected.
eclair-node[3518011]: Please go to https://adoptopenjdk.net/?variant=openjdk11
eclair-node[3518012]: /usr/bin/eclair-node: line 285: and: command not found
* Add ChannelOpened event
Emit an event when a channel is opened and ready to process payments.
This event is distinct from the `ChannelCreated` event which is sent
earlier, once we think a funding transaction has been successfully created
(but cannot guarantee when we're not the initiator).
* Add event to websocket
This is a breaking change, but it should be ok since the previous event
wasn't reliable: it was emitted at a time where we couldn't guarantee
that the channel would really confirm.
We add an actor that waits for a given peer to be connected and ready to
process payments. This is useful in the context of async payments for
the receiver's LSP.
When sending an outgoing multi-part payment, we forward the preimage back
to the sender as soon as we receive the first `update_fulfill_htlc`.
This is particularly useful when relaying trampoline payments, to be able
to propagate the fulfill upstream as early as possible.
However this meant that callers of the HTTP API would receive this
preimage event instead of the final payment result, which was quite bad.
We now disable this first event when used with the `--blocking` argument,
which ensures that the API always returns the payment result.
Fixes#2389
We were previously directly creating onion payloads inside the various
payment state machines and manipulating tlv fields. This was a layering
violation that was somewhat ok because in most cases we only needed to
create the onion payload for the recipient at the beginning of the payment
flow and didn't need to modify it, except for a small change in the MPP
case.
This forced us to handle trampoline onions directly in the payment
initiator and will not work for blinded payments, where we can only build
the onion payload for the recipient after we've chosen the routes and how
to split the amount.
We clean this up by introducing payment recipients that abstract away the
creation of onion payloads. This makes it much easier to integrate blinded
payments. It also allows us to clean up the way we do trampoline payments
and potentially support splitting across multiple trampoline routes (not
included in this PR as this change isn't immediately needed).
It also lets us simplify the MultiPartPaymentLifecycle FSM, by moving the
logic of computing how much remains to be sent and what fee can be used
to the route calculation component.
The default values were generated once when the eclair node starts instead
of being recomputed for every request. Fixes#2475.
We also add pagination to the `listPendingInvoice` API as a follow-up for
#2474.
We previously only supported `scid_alias` and `zero_conf` when used in
combination with `anchors_zero_fee_htlc_tx`, but we have no way of
advertising that to our peers, which leads to confusing failures when
opening channels.
Some nodes that don't have good access to a utxo pool may not migrate to
anchor outputs, but may still want to use `scid_alias` (and potentially
`zero_conf` as well).
Fixes#2394
Add a message flag to channel update to specify that an update is private
and should not be rebroadcast to other nodes.
We log an error if a private channel update gets into the rebroadcast list.
See https://github.com/lightning/bolts/pull/999
When creating a blinded route, we expose the last blinding point (that the
last node will receive). This lets the recipient derive the corresponding
blinded private key, which they may use to sign an invoice.
We add support for generating Bolt 12 invoices and storing them in our
payments DB to then receive blinded payments.
We implement the receiving part once a blinded payment has been decrypted.
This uses the same payment flow as non-blinded payments, with slightly
different validation steps.
Note that we put a random secret in the blinded paths' path_id field
to verify that an incoming payment uses a valid blinded route generated
by us. We store that as an arbitrary byte vector to allow future changes
to this strategy.
* Separate tlv decoding from content validation
When decoding a tlv stream, we previously also validated the
stream's content at decoding time. This was a layer violation,
as checking that specific tlvs are present in a stream is not
an encoding concern.
This was somewhat fine when we only had very basic validation
(presence or absence of specific tlvs), but blinded paths
substantially changed that because one of the tlvs must be
decrypted to yield another tlv stream that also needs to have
its content validated.
This forced us to have an overly complex trait hierarchy in
PaymentOnion.scala and expose a blinding key in classes that
shouldn't care about whether blinding is used or not.
We now decouple that into two distinct steps:
* codecs simply return tlv streams and verify that tlvs are
correctly encoded
* business logic case classes (such as ChannelRelayPayload)
should be instantiated with a new `validate` method that
takes tlv streams and verifies mandatory/forbidden tlvs
This lets us greatly simplify the trait hierarchy and deal
with case class that only contain fully decrypted and valid
data.
* Improve tests
There was redundancy in the wrong places: route blinding codec tests were
testing route blinding decryption and were missing content validation.
We also change the behavior of the route blinding decode method to return
the blinding override when present, instead of letting higher level
components duplicate that logic.
* Use hierarchical namespaces
As suggested by @pm47
* Small PR comments
* Remove confusing comment
Add support for bumping the fees of a dual funding transaction.
We spawn a transient dedicated actor: if the RBF attempt fails, or if we
are disconnected before completing the protocol, we should forget it.
Add more tests for scenarios where an unconfirmed channel is force-closed,
where the funding transaction that confirms may not be the last one.
When not provided, we previously used unrestricted `from` and `to`
parameters when reading from the DB. This can create issues when
accidentally reading too much data.
We now limit this to the last 24 hours, unless explicitly set by the
caller. Note that when reading a lot of data, it's recommended to use
postgres and read from a replica to ensure there's no impact on the running
eclair instance.
Fixes#2383
It is now possible to specify a DNS host name as one of your
`server.public-ips` addresses.
DNS host names will not be resolved until eclair attempts to
connect to the peer.
See https://github.com/lightning/bolts/pull/911
The specification is removing support for old channel updates that didn't
include an `htlc_maximum_msat` (https://github.com/lightning/bolts/pull/996).
Every implementation has been generating updates containing this field for
years, so we can safely reject updates that don't contain it.
We now use either our local alias or the real scid in the channel update
that we store internally. When we send that channel update directly to
our peer, we override it to use the remote alias when it makes sense.
Bolt11 uses routing hints to add extra edges for the path-finding, blinded routes are also extra edges added to the path-finding but they have a different format. The new type `ExtraEdge` is more flexible and can be expanded to cover new kinds of extra edges such as blinded routes.
Allow eclair to expose a metrics scraping endpoint for Prometheus
for users who don't want to rely on Kamon's hosted infrastructure.
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
We make it explicit that conversion from a string to an scid uses the tx
coordinates and can fail.
We also clean up compiler warnings in updated test files.
