The switchboard is our singleton actor entry point to all of our peers and
peer connections, it can be useful to have it deliver basic information
about our peers.
We also take this opportunity to fix a bug: we were not emitting the
`LastChannelClosed` event when it happened while the peer was
disconnected.
We introduced a task to regularly compute network statistics (mostly about
channel parameters such as expiry and fees).
The goal was to use this information in the MPP split algorithm to decide
whether to split a payment or not.
But we haven't used it, and I'm not sure anymore that it's useful at all.
If node operators are interested in network statistics, an ad-hoc
on-the-fly computation would make more sense.
Default upper bound was `Long.MaxValue unixsec` which overflowed when converted to `TimestampMilli`. We now enforce `min` and `max` values on timestamp types.
API tests didn't catch it because eclair is mocked and the conversion happens later.
Fixes#2031.
Add a new log file for important notifications that require an action from
the node operator.
Using a separate log file makes it easier than grepping specific messages
from the standard logs, and lets us use a different style of messaging,
where we provide more information about what steps to take to resolve
the issue.
We rely on an event sent to the event stream so that plugins can also pick
it up and connect with notification systems (push, messages, mails, etc).
Add options to ignore specific channels or nodes for
findRoute* APIs, and an option to specify a flat maximum
fee.
With these new parameters, it's now possible to do circular
rebalancing of your channels.
Co-authored-by: Roman Taranchenko <romantaranchenko@Romans-MacBook-Pro.local>
Co-authored-by: t-bast <bastuc@hotmail.fr>
We define `TimestampSecond` and `TimestampMilli` for second and millisecond precision UNIX-style timestamps.
Let me know what you think of the syntaxic sugar, I went for `123456 unixsec` and `123456789 unixms`.
Json serialization is as follows for resp. second and millisecond precision. Note that in both case we display the unix format in second precision, but the iso format is more precise:
```
{
"iso": "2021-10-04T14:32:41Z",
"unix": 1633357961
}
{
"iso": "2021-10-04T14:32:41.456Z",
"unix": 1633357961
}
```
We are slowly dropping support for non-segwit outputs, as proposed in
https://github.com/lightningnetwork/lightning-rfc/pull/894
We can thus safely allow dust limits all the way down to 354 satoshis.
In very rare cases where dust_limit_satoshis is negotiated to a low value,
our peer may generate closing txs that will not correctly relay on the
bitcoin network due to dust relay policies.
When that happens, we detect it and force-close instead of completing the
mutual close flow.
Add new --format parameter to the findRoute* API calls.
This lets the caller decide whether they want to receive
a list of nodeIds or shortChannelIds to identify the route.
Co-authored-by: Pierre-Marie Padiou <pm47@users.noreply.github.com>
Add support for https://github.com/lightningnetwork/lightning-rfc/pull/824
When the channel type is anchor outputs with zero fee htlc txs, we set
the fees for the htlc txs to 0.
An important side-effect is that it changes the trimmed to dust calculation,
and outputs that were previously dust can now be included in the commit tx.
Add AB testing framework:
- Experiments are added by adding a section in router.path-finding config. Each experiment can have different parameters.
- Traffic is randomly split among the different experiments. The size of each experiment is configurable. 0% experiments don't affect traffic but can be triggered manually with the API.
- Metrics are recorded in the audit database
When using Tor, your onion address is by default added to your
list of public addresses and advertised through the gossip layer
to allow peers to find your node and connect to it.
We now allow opting out of that mechanism by setting
`tor.publish-onion-address = false` in your `eclair.conf`.
This will ensure your onion address is not advertised, and peers
will not be able to connect to you unless you've connected to
them first.
Add `closing_signed` `fee_range` TLV as described in
https://github.com/lightningnetwork/lightning-rfc/pull/847
We also refactor the negotiating state, add many tests and fix#1742.
Add new fields to the `close` API to let users configure their preferred
fees for mutual close.
Bolt11 invoices can either contain a string description or a hash
of that description (if it's too big or should stay private).
The API didn't let users provide a description hash, which is now
possible.
Co-authored-by: Pierre-Marie Padiou <pm47@users.noreply.github.com>
* Set relay fees per node and save them to database
- Fees are set per node instead of per channel (setting different fees for different channels to the same node is most probably an error)
- Fees are saved to a database so that we can keep a trace of historic fees and new channels with a known node use the fee that we set and not the default fee.
A json column has been added to the few tables that contains an
opaque serialized blob:
- `local_channels.data`
- `nodes.data`
- `channels.channel_announcement`, `channels.channel_update_x`
We can now access all the individual data fields from SQL.
For the serialization, we use the same serializers than the one
that were previously used by the API. They have been moved to the
`eclair-core` module and simplified a bit.
There are two json data types in Postgres: `JSON` and `JSONB`. We use
the latter one, which is more recent, and allows indexing.
An alternative to this PR would have been to use columns, but:
- there would have been a *lot* of columns for the channel data
- every modification of our types would have required a db migration
NB: to handle non-backwards compatible changes in the json serializersi,
all the json columns can be recomputed on restart by setting
`eclair.db.reset-json-columns=true`.
Change in in ChannelCodecsSpec:
The goal of this test is to make sure that, in addition to successfully
decoding data that encoded with an older codec, we actually read the
correct data. Just because there is no error doesn't mean that we
interpreted the data properly. For example we could invert a
`payment_hash` and a `payment_preimage`.
We can't compare object to object, because the current version of the
class has probably changed too. That's why we compare using the json
representation of the data, that we amend to ignore new or modified
fields.
After doing a manual comparison, I updated the test to use the current
json serializers, and replaced the test data with the latest json
serialization. This allows us to remove all the tweaks that we added
over time to take into account new and updated fields.
It returns an overall balance, separating onchain, offchain, and
removing duplicates (e.g. mutual closes that haven't reached min depth
still have an associated channel, but they already appear in the
on-chain balance). We also take into account known preimages, even if
the htlc hasn't been formally resolved.
Metrics have also been added.
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
The `payment_secret` feature was made mandatory in #1810 and is the default
in other implementations as well. We can thus force it to be available when
decoding onion payloads, which simplifies downstream components (no need
to handle the case where a `payment_secret` may be missing anymore).
We also rename messages in `PaymentInitiator` to remove the confusion with
Bolt 11 payment requests.
Plugins can extend the `RouteProvider` trait to enrich the API with
custom calls, removing the need to setup a separate endpoint on a
different port.
When routes clash between plugins, the second one is simply ignored.
Plugin developers should prepend their route with their plugin name
to avoid such silent clashes.
In case of catastrophic failures of the `SecureRandom` instance, we add
a secondary randomness source that we mix into the random stream.
This is a somewhat weak random source and should not be used on its own,
but it doesn't hurt to xor it with the output of `SecureRandom`.
We use an actor that listens to events in the system and inject them
in our weak pseudo-RNG.
* Add trampoline info to auditDB
Add a new table containing the recipient and amount sent to the recipient in case of trampoline relaying.
When using trampoline, the recipient may not be the next node on the path.
Re-work the `CommitPublished` types to work better with anchor outputs.
We previously stored the txs spending utxos that we could claim: this
doesn't make sense anymore if these txs may be RBF-ed, because the final
tx will be different from the initial one.
We instead track what `OutPoint`s we can claim, and the information
necessary to claim them. This way we can in the future let a different
actor finalize the txs that spend these outpoints (set the fees and sign).
We also add information on mutual close txs to immediately identify our
output and its amount: this makes auditing how much sats we'll get back
very easy from the API when we have many channels to watch.
This commit contains a DB migration of the channel data types, but in a
backwards-compatible way: we can still read from old data. The only
scenario impacted is channels that started force-closing before the migration.
They need special care to handle the fact that they had less data than
migrated channels, which is why we keep some legacy code around.
We incorrectly applied error handlers at each sub-route instead of applying
it after grouping all sub-routes together. The result was that only `getinfo`
could actually be called.
Refactor the API handlers.
Split handlers and directives in several files to make them more composable.
Co-authored-by: Pierre-Marie Padiou <pm47@users.noreply.github.com>
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
We previously used a Set, which means you could theoretically have a feature
that is both activated as `optional` and `mandatory`.
We change that to be a Map `feature -> support`.
It can be useful to override the default relay fees when opening channels
to specific nodes.
Note that these initial relay fees are not persisted in the DB. That means
that if your node reboots before the funding transaction confirms, the
channel will be opened with the default relay fees, not the overridden values.
Fixes#1507
Plugins can now implement custom commitment schemes, different from specified ones.
This requires extra care when restarting a node, as some HTLCs may be in a broken state
and need to be failed upstream.
Plugins must identify these HTLCs and implement the right trait to provide these HTLCs to
the PostRestartHtlcCleaner, which will watch them and fail/fulfill them accordingly.
* Send payment through specific channels
While `sendtoroute` was letting you provide a custom route as a list of
nodes, it made it difficult to have fine-grain control over the channels
used by payments.
The API now allows choosing the exact channels that will be used for the
payment; in particular, this will be helpful when consolidating and
rebalancing multiple channels to a given node.
Fixes#1472
* Fix serialization regression
#1520 introduced a regression in the serialization of channel data,
impacted the `channels` and `channel` APIs.
We restrict the custom command response serializer to revert to the
previous behavior.
Send basic channel events to websockets listeners:
* Channel open initiated
* Channel state change
* Channel closed
We only send basic, high-level data about these events.
If the listener is interested in details, it should call the `channelInfo`
API to get all of the channel's data.
Fixes#1509
* typeify response to command messages
* Status.Failure(AddHtlcFailed)->RES_ADD_HTLC_FAILED
`AddHtlcFailed` is renamed `RES_ADD_FAILED` and it is just a particular
kind of `CommandError`
* remove Status.Failure from payment lifecycle
* use replyTo pattern in relayers
* return ForwardMessage in CommandResponse
Instead of having the channel send `Relayer.ForwardMessage` to the
relayer, we encapsulate the `Relayer.ForwardMessage` within a
`CommandResponse[CMD_ADD_HTLC]`.
It looks like a cosmetic change, but it's not: now when the relayer
sends a `CMD_ADD_HTLC` to the channel, it will receive one or more
`CommandResponse[CMD_ADD_HTLC]`, for example:
- success scenario:
- `RES_SUCCESS[CMD_ADD_HTLC]`
- `RES_ADD_COMPLETED[RelayBackward.RelayFulfill]`
- htlc failed by downstream
- `RES_SUCCESS[CMD_ADD_HTLC]`
- `RES_ADD_SETTLED[RelayBackward.RelayFail]`
- command rejected:
- `RES_ADD_FAILED[_]`
- peer disconnected before signing:
- `RES_SUCCESS[CMD_ADD_HTLC]`
- `RES_ADD_FAILED[ChannelUnavailable]`
In addition to that, `RelayMessage` have been slightly refactored, to
better separate between `RelayForward` and `RelayBackward`.
This paves the way for typing `CMD_ADD_HTLC.replyTo` to
`ActorRef[CommandResponse[CMD_ADD_HTLC]]` and have the channel send all
related messages to the `replyTo` actor. Note that the `RelayForward`
actor will always be sent to the relayer, which makes sense since there
was no related command (the htlc was sent by the peer).
NB: `CMD_ADD_HTLC` is a special case, for all other commands there is
exactly one `CommandResponse[CMD_*]`, either `RES_SUCCESS[CMD_*]` or
`RES_FAILURE[CMD_*]`.
* use replyTo pattern in payment lifecycle
We were already close to this pattern with constructs like `case class
WaitingForRoute(sender: ActorRef, ...`
* typeify Origin/Upstream classes
The relationship between `Origin` and `Upstream` was obscure and they were
defined in two different files.
`Upstream` is the source of the payment in the context of a chain
of htlcs. For example, in the case of a typical relayed payment, it
would be an incoming htlc.
`Origin` is the source of the payment in the application: it can be an
actor, or nothing if the reference was lost after a restart.
Instead of using an `Option[ActorRef]` to differentiate between
known/unknown origin, new `Hot/Cold` traits that extend `Origin` have
been introduced. This means that now the `PostRestartHtlcCleaner` only
deals with `Cold` origins, whereas the `NodeRelayer` only handles `Hot`
origins. The channel codec will encode from both `Hot/Cold` origins, but
will only decode to `Cold`.
* refactor response types
Generalize the `CommandResponse[Command]` pattern for all commands.
There seem to be something ambiguous about the way we deal with closing
commands during the initialization phase of the channel. We used to
conflate `CommandResponse[CMD_CLOSE]` and
`ChannelOpenResponse.ChannelClosed` but those are not sent to the same
actors.
It turns out our testing of the `EclairImpl` class is very weak. We
could use this class in `IntegrationSpec` instead of sending raw
messages to channels.
* handle channel-relay in the post-htlc-restart too
* Activate wumbo by default
This is safe as `max-funding-satoshis` is set to 16777215 sats, which is
the non-wumbo limit.
If users want to increase the maximum channel size, they can update this
configuration value.
* Update default minFinalCltvExpiryDelta
See https://github.com/lightningnetwork/lightning-rfc/pull/785
* Set minFinalCltvExpiryDelta in invoices
Our default fulfill-safety-window is now greater than the spec's default
min-final-expiry-delta in invoices, so we need to explicitly tell payers
what value they must use.
Otherwise we may end up closing channels if a block is produced while we're
waiting for our peer to accept an UpdateFulfillHtlc.
Do all the changes that will be required and are already possible to
minimize the diff:
- update dependencies
- `'something` -> `Symbol("something")`
- `BigDecimal.xValue()` -> `BigDecimal.xValue`
- `Map.filterKeys` -> `Map.filterKeys.toMap` (same for `Map.mapValues`)
- `def myMethod(...)` -> `def myMethod(...): Unit`
