Add config fields for max dust htlc exposure.
These configuration fields let node operators decide on the amount of dust
htlcs that can be in-flight in each channel.
In case the channel is force-closed, up to this amount may be lost in
miner fees.
When sending and receiving htlcs, we check whether they would overflow
our configured dust exposure, and fail them instantly if they do.
A large `update_fee` may overflow our dust exposure by removing from the
commit tx htlcs that were previously untrimmed.
Node operators can choose to automatically force-close when that happens,
to avoid risking losing large dust amounts to miner fees.
The script wasn't length-delimited.
Fortunately this feature was disabled by default.
Since no-one reported the issue, we can probably just do this simple fix.
Unfortunately, `context.log` is *not* thread safe and shouldn't be used
in future continuation. We should instead use `pipeToSelf` when we want
to act on the results of a `Future`.
Allow any feerate when using anchor outputs and we're fundee.
This will prevent unwanted channel closure.
This can be unsafe in a high fee environment if the commit tx is below
the propagation threshold. However, even when we discover it it's too late
anyway, so our only option is to wait for package relay to save the day.
Ensure feerate is always above propagation threshold when we're funder.
We lift the limit configured by the node operator when it is below the
network propagation threshold.
We previously computed the on-chain fees paid by us after the fact, when
receiving a notification that a transaction was confirmed. This worked
because lightning transactions had a single input, which we stored in
our DB to allow us to compute the fee.
With anchor outputs, this mechanism doesn't work anymore. Some txs have
their fees paid by a child tx, and may have more than one input.
We completely change our model to store every transaction we publish,
along with the fee we're paying for this transaction. We then separately
store every transaction that confirms, which lets us join these two data
sets to compute how much on-chain fees we paid.
This has the added benefit that we can now audit every transaction that
we tried to publish, which lets node operators audit the anchor outputs
internal RBF mechanism and all the on-chain footprint of a given channel.
It's quite cumbersome to investigate complex MPP payment failures.
We need to grep on the parent ID, then group together logs for each child
payment, and then we're ready for some analysis.
Most of the time, a quick look at the breakdown of all intermediate failures
is all we need to diagnose the problem. This PR adds such a log line.
Having basic documentation in-place by providing examples in
`eclair.conf` is great and very convenient, but in the case of
path-finding, defining experiments take so much space that it makes
the whole configuration file actually more complicated to understand.
And since we don't want to enable experiments by default, the user still
has to figure out what to change to actually enable AB-testing.
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
* make custom serializers objects instead of classes
* reorder json format definition
* use minimal serializers
Having custom serializers depend on external format would introduce an
infinite recursion at runtime if not careful. Thankfully, none of our
serializers use it, so we may as well remove the possibility entirely.
* simplify serializers further
We don't need to type the serializers: this is required for deserializing,
not serializing, and we are not using it.
The fact that be had a type mismatch here shows it:
```scala
object TransactionSerializer extends MinimalSerializer[TransactionWithInputInfo]
```
* new generic json serializer
Instead of providing a `MyClass => JValue` conversion method, we
provide a `MyClass => MyClassJson` method, with the assumption that
`MyClassJson` is serializable using the base serializers.
The rationale is that it's easier to define the structure with types
rather than by building json objects.
This also means that the serialization of attributes of class C is out
of the scope when defining the serializer for class C. See for example
how `DirectedHtlcSerializer` doesn't need anymore to bring in
lower level serializers.
It also has the advantage of removing recursion from custom serializers
which sometimes generated weird stack overflows.
We are slowly dropping support for non-segwit outputs, as proposed in
https://github.com/lightningnetwork/lightning-rfc/pull/894
We can thus safely allow dust limits all the way down to 354 satoshis.
In very rare cases where dust_limit_satoshis is negotiated to a low value,
our peer may generate closing txs that will not correctly relay on the
bitcoin network due to dust relay policies.
When that happens, we detect it and force-close instead of completing the
mutual close flow.
The fee that's recorded in the path-finding metrics should count the local channel. Without it we record failing payments with a fee budget larger than the fee recorded for the successful payment to the same node.
* optionally record path-funding metrics
This is useful in feature branches.
Also rename `recordMetrics` to `recordPathFindingMetrics`.
* rename channel features attribute
It was a remainder of init features, which can be activated or not. But
in the context of the `ChannelFeatures` object, that naming was
confusing because all features are activated in that context.
* minor refactoring on channel_type
Moved some logic outside `Peer`.
* refactor `RouteParams`
`PathFindingConf` and `RouteParams` have almost the same set of params,
but some of them don't have the same structure (flat vs hierarchical).
It's useful to have the release notes directly inside the repository
instead of relying on Github: that lets for example users read them offline.
It also lets us fill the release notes every time we add relevant features
on `master` instead of having to do them all at once when we decide to make
a release.
The separation between `ExtendedBitcoinClient` and `BitcoinCoreWallet` has
become very blurry since anchor ouputs: eclair now requires fee bumping
utilities from the underlying bitcoin wallet, and it's not yet clear what
the interface should be. The on-chain utility methods that were added to
the eclair API also made it awkward to cleanly separate concerns.
We completely remove the `BitcoinCoreWallet` and merge it inside the
bitcoin client. We may in the future re-introduce a cleaner on-chain wallet
abstraction, but that can only happen once we have stable fee bumping
mechanisms.
Add new --format parameter to the findRoute* API calls.
This lets the caller decide whether they want to receive
a list of nodeIds or shortChannelIds to identify the route.
Co-authored-by: Pierre-Marie Padiou <pm47@users.noreply.github.com>
Add support for https://github.com/lightningnetwork/lightning-rfc/pull/824
When the channel type is anchor outputs with zero fee htlc txs, we set
the fees for the htlc txs to 0.
An important side-effect is that it changes the trimmed to dust calculation,
and outputs that were previously dust can now be included in the commit tx.
* disable pg lock auto-release in tests
It relies on akka's coordinated shutdown and causes the test jvm to
halt.
* fixup! Remove `messageFlags` from `ChannelUpdate` (#1941)
Add AB testing framework:
- Experiments are added by adding a section in router.path-finding config. Each experiment can have different parameters.
- Traffic is randomly split among the different experiments. The size of each experiment is configurable. 0% experiments don't affect traffic but can be triggered manually with the API.
- Metrics are recorded in the audit database
We make it a serialization detail, which it should be. The `derive`
method from `scodec` makes it very easy to do. We should probably always
use a dedicated class to handle flags, instead of using the `byte` codec
and binary operators.
This allows to remove the `require` in the `ChannelUpdate` definition,
which recently bit us in testing.
The only annoying thing is that we still need to expose a `messageFlags`
method in order to populate the `ChannelDisabled` error message.
We also typeify channel flags, as an alternative to passing around a `Byte`.
When using Tor, your onion address is by default added to your
list of public addresses and advertised through the gossip layer
to allow peers to find your node and connect to it.
We now allow opting out of that mechanism by setting
`tor.publish-onion-address = false` in your `eclair.conf`.
This will ensure your onion address is not advertised, and peers
will not be able to connect to you unless you've connected to
them first.
This lets plugins define variants of the `ChannelRestored` when
using non-standard channel types, while ensuring these channels
are correctly integrated in the list of local channels for payment
relaying.
Add `closing_signed` `fee_range` TLV as described in
https://github.com/lightningnetwork/lightning-rfc/pull/847
We also refactor the negotiating state, add many tests and fix#1742.
Add new fields to the `close` API to let users configure their preferred
fees for mutual close.
Use an event `ChannelUpdateParametersChanged` for the sole purpose of tracking changes to channel_update.
Also, conf change at restore is now treated like a regular fee update. We do handle `CMD_UPDATE_RELAY_FEES` in both `OFFLINE` and `SYNCING`, because there may be a race between `CMD_UPDATE_RELAY_FEES` and
`ChannelRestablish`. And there was no good reason to behave differently in those states anyway.
* fix updateRelayFee api call
The `Register` should be used to channel actors, not the `Router`.
The former tracks all channels, whereas the latter only contains
channels in certain states. We only query the `Router` when we need
reference to external (public) nodes and channels.
The GUI has been deprecated a long time ago, and doesn't inform users about
potential risks, such as RBF-ing funding txs.
We should instead incentivize users to use the CLI and read the documentation,
which ensures they will know about potential pitfalls.
Bolt11 invoices can either contain a string description or a hash
of that description (if it's too big or should stay private).
The API didn't let users provide a description hash, which is now
possible.
Co-authored-by: Pierre-Marie Padiou <pm47@users.noreply.github.com>
We now query blockchain watchdogs over Tor when it's activated.
Some watchdogs are automatically disabled because they have no
support for Tor.
We also let users change the list of watchdogs that should be run
from their `eclair.conf`.
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
For path finding we always need `routeParams`, however all of the messages asking for finding a route have `routeParams` as an optional parameter. This makes it hard to enforce that a given payments uses a given `routeParams` as there may be a `None` somewhere that reverts to the global defaults.
It seems that the reason all the `routeParams` are optional is that the messages asking for finding a route are sometimes used when we already have a route. This is a hacky solution and I've tried to replace these messages by traits that work both when we want to find a route and when we already have it.
I'm trying to enable AB testing which requires using different `routeParams` for different payments and this is a blocker.
* Disable ZMQ high watermark
This should prevent messages from being dropped.
We also configure the socket before subscribing to topics and connecting.
* Switch ZMQ raw block to block hash only
We were receiving raw blocks on the ZMQ socket, whereas we don't use it.
We only use this event as a trigger that a new block has been found, so we
can save bandwidth by switching to block hash subscriptions instead.
* Regularly check blocks
In case we haven't received block events on the ZMQ socket, we regularly
check whether we're behind.
There's a very annoying race condition in that test, and we can end up
emitting events twice on slow machines. This isn't an issue so we stop
verifying that we avoid duplicate events.
* Never retry anchor publish when commit confirmed
We used to check the feerate before checking whether the commit tx was
already confirmed: when the commit feerate was good enough, we would
respawn a publish actor every block whereas the commit tx was already
confirmed.
* Abandon evicted replaceable txs
This ensures the bitcoind wallet won't keep the transaction around and
avoid using its inputs for other transactions.
Fixes#1898
Scale MPP partial amounts based on the total amount we want to
send and the number of parts we allow.
This avoids polluting the results with cheap routes that don't have the
capacity to route when the amount is big.
- Add a base weight ratio. Allows to give more weight to the fee itself. Setting the base to 1 is like not using weight ratios.
- Add checks that the configuration values make sense (all weights should be nonnegative and sum to 1). Checks are performed when loading the configuration.
- Add a virtual cost per hop which allows prioritizing short paths. The cost per hop has a base component and a proportional one, similar to channel fees.
