Add AB testing framework:
- Experiments are added by adding a section in router.path-finding config. Each experiment can have different parameters.
- Traffic is randomly split among the different experiments. The size of each experiment is configurable. 0% experiments don't affect traffic but can be triggered manually with the API.
- Metrics are recorded in the audit database
We make it a serialization detail, which it should be. The `derive`
method from `scodec` makes it very easy to do. We should probably always
use a dedicated class to handle flags, instead of using the `byte` codec
and binary operators.
This allows to remove the `require` in the `ChannelUpdate` definition,
which recently bit us in testing.
The only annoying thing is that we still need to expose a `messageFlags`
method in order to populate the `ChannelDisabled` error message.
We also typeify channel flags, as an alternative to passing around a `Byte`.
Add `closing_signed` `fee_range` TLV as described in
https://github.com/lightningnetwork/lightning-rfc/pull/847
We also refactor the negotiating state, add many tests and fix#1742.
Add new fields to the `close` API to let users configure their preferred
fees for mutual close.
Bolt11 invoices can either contain a string description or a hash
of that description (if it's too big or should stay private).
The API didn't let users provide a description hash, which is now
possible.
Co-authored-by: Pierre-Marie Padiou <pm47@users.noreply.github.com>
For path finding we always need `routeParams`, however all of the messages asking for finding a route have `routeParams` as an optional parameter. This makes it hard to enforce that a given payments uses a given `routeParams` as there may be a `None` somewhere that reverts to the global defaults.
It seems that the reason all the `routeParams` are optional is that the messages asking for finding a route are sometimes used when we already have a route. This is a hacky solution and I've tried to replace these messages by traits that work both when we want to find a route and when we already have it.
I'm trying to enable AB testing which requires using different `routeParams` for different payments and this is a blocker.
* Set relay fees per node and save them to database
- Fees are set per node instead of per channel (setting different fees for different channels to the same node is most probably an error)
- Fees are saved to a database so that we can keep a trace of historic fees and new channels with a known node use the fee that we set and not the default fee.
It returns an overall balance, separating onchain, offchain, and
removing duplicates (e.g. mutual closes that haven't reached min depth
still have an associated channel, but they already appear in the
on-chain balance). We also take into account known preimages, even if
the htlc hasn't been formally resolved.
Metrics have also been added.
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
The `payment_secret` feature was made mandatory in #1810 and is the default
in other implementations as well. We can thus force it to be available when
decoding onion payloads, which simplifies downstream components (no need
to handle the case where a `payment_secret` may be missing anymore).
We also rename messages in `PaymentInitiator` to remove the confusion with
Bolt 11 payment requests.
Electrum support was provided for mobile wallets, server nodes should always
run a bitcoind node as this provides more control (especially for utxo
management for anchor outputs channels).
Since wallets will use https://github.com/acinq/eclair-kmp instead of eclair,
we can now remove Electrum and API fee providers from eclair.
We also removed 3rd-party fee API providers that were only used on wallets.
We now use a regular release zip, and use it in the awseb bundle.
The default logback configuration now logs to a file, like
`eclair-node`, and we use a dedicated logback configuration for AWS
beanstalk.
By default the front reads the node secret key from the node seed in
the standard `.eclair` directory.
It can be useful to override the default relay fees when opening channels
to specific nodes.
Note that these initial relay fees are not persisted in the DB. That means
that if your node reboots before the funding transaction confirms, the
channel will be opened with the default relay fees, not the overridden values.
Fixes#1507
Plugins can now implement custom commitment schemes, different from specified ones.
This requires extra care when restarting a node, as some HTLCs may be in a broken state
and need to be failed upstream.
Plugins must identify these HTLCs and implement the right trait to provide these HTLCs to
the PostRestartHtlcCleaner, which will watch them and fail/fulfill them accordingly.
Don't swallow bitcoind exceptions: we wrap them but preserve the
original one.
Allow configuring bitcoin core wallet: it makes sense to allow users
to use a different wallet from the default one.
There's one important caveat: once set, users shouldn't change it while
they have open channels. We mention it clearly in the documentation.
Fixes#1538
Allow plugins to register unknown features and message types they're able to handle.
This allows plugins to add new features independently of what eclair-core understands.
Plugins are able to receive and send arbitrary lightning messages, and advertise support
for non-standard features freely.
* Type fee rates info
Fixes#1188
* Fix vsize comment
This is an alternative to #1425.
This may not correctly represent what Bitcoin Core does, it's likely that
we can in fact use a value smaller than 253, but this shows how we choose
to err on the side of safety with that calculation.
* Add 1008 feerate block target
Fixes#1486
* Activate wumbo by default
This is safe as `max-funding-satoshis` is set to 16777215 sats, which is
the non-wumbo limit.
If users want to increase the maximum channel size, they can update this
configuration value.
* Update default minFinalCltvExpiryDelta
See https://github.com/lightningnetwork/lightning-rfc/pull/785
* Set minFinalCltvExpiryDelta in invoices
Our default fulfill-safety-window is now greater than the spec's default
min-final-expiry-delta in invoices, so we need to explicitly tell payers
what value they must use.
Otherwise we may end up closing channels if a block is produced while we're
waiting for our peer to accept an UpdateFulfillHtlc.
* Extract faulty channels selection from PaymentLifecycle
Move the logic of figuring out which channels/nodes should be ignored
when retrying after a payment failure out of the PaymentLifecycle.
We can figure this out looking only at the `PaymentFailure` generated,
and the multi-part logic could leverage these helpers.
* Refactor RouteResponse
It was useless to return `ignoreNodes` and `ignoreChannels`, it's rather
the responsibility of the caller (PaymentLifecycle) to store and update
these sets.
Preparing for the MPP move inside the router, we introduce a Route class
and let RouteResponse return a collection of Routes.
This creates some ugliness in PaymentLifecycle because of the `routePrefix`,
but this is just temporary: the `routePrefix` "hack" will be removed soon.
This is almost a drop-in replacement. I had to relaxed compiler
parameters to allow deprecated features though.
Main changes:
- relaxed compiler parameters to minimize impact (e.g. allow
deprecated features)
- `scala.collection.JavaConverters` -> `scala.jdk.CollectionConverters`
- `MultiMap` -> `MultiDict`
Compilation is 25% faster on my machine, compiler is a bit more strict
(it found an "invalid comparison" bug).
Do all the changes that will be required and are already possible to
minimize the diff:
- update dependencies
- `'something` -> `Symbol("something")`
- `BigDecimal.xValue()` -> `BigDecimal.xValue`
- `Map.filterKeys` -> `Map.filterKeys.toMap` (same for `Map.mapValues`)
- `def myMethod(...)` -> `def myMethod(...): Unit`
* front now handles ping/sync
Peer has been split in two and now handles only channel related stuff.
A new `PeerConnection` class is in charge of managing the BOLT 1 part
(init handshake, pings) and has the same lifetime as the underlying
connection.
Also, made `TransportHandler` be a child of `PeerConnection` by making
the `remoteNodeId` an attribute of the state of `PeerConnection` instead
of a constructor argument (since we cannot be sure of the remote nodeid
before the auth handshake is done). Now we don't need to worry about
cleaning up the underlying `TransportHandler` if the `PeerConnection`
dies.
* remove `Authenticator`
Instead of first authenticating a connection, then passing it to the
`PeerConnection` actor, we pass the connection directly to the
`PeerConnection` and let it handle the crypto handshake, before the LN
init. This removes a central point of management and makes things easier
to reason about. As a side effect, the `TransportHandler` actor is now a
child of `PeerConnection` which gives us a guarantee that it dies when
its parent dies.
* separated connection logic from `Peer`
The `ReconnectionTask` actor handles outgoing connections to a peer. The
goal is to free
the `Peer` actor from the reconnection logic and have it just react to
already established
connections, independently of whether those connections are incoming or
outgoing.
The base assumption is that the `Peer` will send its state transitions
to the `ReconnectionTask` actor.
This is more complicated than it seems and there are various corner
cases to consider:
- multiple available addresses
- concurrent outgoing connections and conflict between
automated/user-requested attempts
- concurrent incoming/outgoing connections and risk of reconnection
loops
- etc.
Co-Authored-By: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
* Support wumbo channels:
- use bits 18, 19
- compute the min depth for the funding transaction according to the channel size
- update routing heuristics for a wumbo world:
- the lower bound is the 25th percentile of current channel capacity on the network
- the higher bound is the most common capacity for wumbo channels
- add 'max-funding-satoshis' configuration key to allow to set the maximum channel size that will be accepted
With MPP and Trampoline (and particularly the combination of the two),
we need to keep track of multiple amounts, recipients and fees.
There's a trampoline fee and a fee to reach the first trampoline node.
The trampoline nodes must appear in the route, but not as payment recipients.
Adding new fields to payment events and DB structs lets us distinguish those.
We also relax the spec requirement about feature graph dependencies.
The requirement to include `var_onion_optin` in invoice feature bits
was added after the first Phoenix release.
Phoenix users will thus have non spec-compliant invoices in their
payment history.
We accept invoices that don't set this field; this is a harmless
spec violation (as long as we set it in new invoices).
