A json column has been added to the few tables that contains an
opaque serialized blob:
- `local_channels.data`
- `nodes.data`
- `channels.channel_announcement`, `channels.channel_update_x`
We can now access all the individual data fields from SQL.
For the serialization, we use the same serializers than the one
that were previously used by the API. They have been moved to the
`eclair-core` module and simplified a bit.
There are two json data types in Postgres: `JSON` and `JSONB`. We use
the latter one, which is more recent, and allows indexing.
An alternative to this PR would have been to use columns, but:
- there would have been a *lot* of columns for the channel data
- every modification of our types would have required a db migration
NB: to handle non-backwards compatible changes in the json serializersi,
all the json columns can be recomputed on restart by setting
`eclair.db.reset-json-columns=true`.
Change in in ChannelCodecsSpec:
The goal of this test is to make sure that, in addition to successfully
decoding data that encoded with an older codec, we actually read the
correct data. Just because there is no error doesn't mean that we
interpreted the data properly. For example we could invert a
`payment_hash` and a `payment_preimage`.
We can't compare object to object, because the current version of the
class has probably changed too. That's why we compare using the json
representation of the data, that we amend to ignore new or modified
fields.
After doing a manual comparison, I updated the test to use the current
json serializers, and replaced the test data with the latest json
serialization. This allows us to remove all the tweaks that we added
over time to take into account new and updated fields.
It returns an overall balance, separating onchain, offchain, and
removing duplicates (e.g. mutual closes that haven't reached min depth
still have an associated channel, but they already appear in the
on-chain balance). We also take into account known preimages, even if
the htlc hasn't been formally resolved.
Metrics have also been added.
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
The `payment_secret` feature was made mandatory in #1810 and is the default
in other implementations as well. We can thus force it to be available when
decoding onion payloads, which simplifies downstream components (no need
to handle the case where a `payment_secret` may be missing anymore).
We also rename messages in `PaymentInitiator` to remove the confusion with
Bolt 11 payment requests.
Plugins can extend the `RouteProvider` trait to enrich the API with
custom calls, removing the need to setup a separate endpoint on a
different port.
When routes clash between plugins, the second one is simply ignored.
Plugin developers should prepend their route with their plugin name
to avoid such silent clashes.
In case of catastrophic failures of the `SecureRandom` instance, we add
a secondary randomness source that we mix into the random stream.
This is a somewhat weak random source and should not be used on its own,
but it doesn't hurt to xor it with the output of `SecureRandom`.
We use an actor that listens to events in the system and inject them
in our weak pseudo-RNG.
* Add trampoline info to auditDB
Add a new table containing the recipient and amount sent to the recipient in case of trampoline relaying.
When using trampoline, the recipient may not be the next node on the path.
Re-work the `CommitPublished` types to work better with anchor outputs.
We previously stored the txs spending utxos that we could claim: this
doesn't make sense anymore if these txs may be RBF-ed, because the final
tx will be different from the initial one.
We instead track what `OutPoint`s we can claim, and the information
necessary to claim them. This way we can in the future let a different
actor finalize the txs that spend these outpoints (set the fees and sign).
We also add information on mutual close txs to immediately identify our
output and its amount: this makes auditing how much sats we'll get back
very easy from the API when we have many channels to watch.
This commit contains a DB migration of the channel data types, but in a
backwards-compatible way: we can still read from old data. The only
scenario impacted is channels that started force-closing before the migration.
They need special care to handle the fact that they had less data than
migrated channels, which is why we keep some legacy code around.
We incorrectly applied error handlers at each sub-route instead of applying
it after grouping all sub-routes together. The result was that only `getinfo`
could actually be called.
Refactor the API handlers.
Split handlers and directives in several files to make them more composable.
Co-authored-by: Pierre-Marie Padiou <pm47@users.noreply.github.com>
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
We previously used a Set, which means you could theoretically have a feature
that is both activated as `optional` and `mandatory`.
We change that to be a Map `feature -> support`.
Added additional method to Eclair like findRoute but allowing for 2 nodeIds.
Also added a new endpoint to the http Api "findroutebetweennodes" which
takes sourceNode and targetNode as params.
Fixes#1068
We now use a regular release zip, and use it in the awseb bundle.
The default logback configuration now logs to a file, like
`eclair-node`, and we use a dedicated logback configuration for AWS
beanstalk.
By default the front reads the node secret key from the node seed in
the standard `.eclair` directory.
It can be useful to override the default relay fees when opening channels
to specific nodes.
Note that these initial relay fees are not persisted in the DB. That means
that if your node reboots before the funding transaction confirms, the
channel will be opened with the default relay fees, not the overridden values.
Fixes#1507
Plugins can now implement custom commitment schemes, different from specified ones.
This requires extra care when restarting a node, as some HTLCs may be in a broken state
and need to be failed upstream.
Plugins must identify these HTLCs and implement the right trait to provide these HTLCs to
the PostRestartHtlcCleaner, which will watch them and fail/fulfill them accordingly.
* Send payment through specific channels
While `sendtoroute` was letting you provide a custom route as a list of
nodes, it made it difficult to have fine-grain control over the channels
used by payments.
The API now allows choosing the exact channels that will be used for the
payment; in particular, this will be helpful when consolidating and
rebalancing multiple channels to a given node.
Fixes#1472
* Fix serialization regression
#1520 introduced a regression in the serialization of channel data,
impacted the `channels` and `channel` APIs.
We restrict the custom command response serializer to revert to the
previous behavior.
When using MPP, if we can't find a route, we need to add an entry to the
DB. Otherwise when users query their payment status, nothing will be
returned which is a bad UX.
Fixes#1512
Send basic channel events to websockets listeners:
* Channel open initiated
* Channel state change
* Channel closed
We only send basic, high-level data about these events.
If the listener is interested in details, it should call the `channelInfo`
API to get all of the channel's data.
Fixes#1509
Allow plugins to register unknown features and message types they're able to handle.
This allows plugins to add new features independently of what eclair-core understands.
Plugins are able to receive and send arbitrary lightning messages, and advertise support
for non-standard features freely.
* introduce a new relay identifier
In a channel relay, it will be unique for all retries.
In a trampoline relay, it is equal to what previously was the parent
payment id.
* moved and cleaned up remaining relayer tests
Now all relay-related tests are in the `relayer` package.
* typeify response to command messages
* Status.Failure(AddHtlcFailed)->RES_ADD_HTLC_FAILED
`AddHtlcFailed` is renamed `RES_ADD_FAILED` and it is just a particular
kind of `CommandError`
* remove Status.Failure from payment lifecycle
* use replyTo pattern in relayers
* return ForwardMessage in CommandResponse
Instead of having the channel send `Relayer.ForwardMessage` to the
relayer, we encapsulate the `Relayer.ForwardMessage` within a
`CommandResponse[CMD_ADD_HTLC]`.
It looks like a cosmetic change, but it's not: now when the relayer
sends a `CMD_ADD_HTLC` to the channel, it will receive one or more
`CommandResponse[CMD_ADD_HTLC]`, for example:
- success scenario:
- `RES_SUCCESS[CMD_ADD_HTLC]`
- `RES_ADD_COMPLETED[RelayBackward.RelayFulfill]`
- htlc failed by downstream
- `RES_SUCCESS[CMD_ADD_HTLC]`
- `RES_ADD_SETTLED[RelayBackward.RelayFail]`
- command rejected:
- `RES_ADD_FAILED[_]`
- peer disconnected before signing:
- `RES_SUCCESS[CMD_ADD_HTLC]`
- `RES_ADD_FAILED[ChannelUnavailable]`
In addition to that, `RelayMessage` have been slightly refactored, to
better separate between `RelayForward` and `RelayBackward`.
This paves the way for typing `CMD_ADD_HTLC.replyTo` to
`ActorRef[CommandResponse[CMD_ADD_HTLC]]` and have the channel send all
related messages to the `replyTo` actor. Note that the `RelayForward`
actor will always be sent to the relayer, which makes sense since there
was no related command (the htlc was sent by the peer).
NB: `CMD_ADD_HTLC` is a special case, for all other commands there is
exactly one `CommandResponse[CMD_*]`, either `RES_SUCCESS[CMD_*]` or
`RES_FAILURE[CMD_*]`.
* use replyTo pattern in payment lifecycle
We were already close to this pattern with constructs like `case class
WaitingForRoute(sender: ActorRef, ...`
* typeify Origin/Upstream classes
The relationship between `Origin` and `Upstream` was obscure and they were
defined in two different files.
`Upstream` is the source of the payment in the context of a chain
of htlcs. For example, in the case of a typical relayed payment, it
would be an incoming htlc.
`Origin` is the source of the payment in the application: it can be an
actor, or nothing if the reference was lost after a restart.
Instead of using an `Option[ActorRef]` to differentiate between
known/unknown origin, new `Hot/Cold` traits that extend `Origin` have
been introduced. This means that now the `PostRestartHtlcCleaner` only
deals with `Cold` origins, whereas the `NodeRelayer` only handles `Hot`
origins. The channel codec will encode from both `Hot/Cold` origins, but
will only decode to `Cold`.
* refactor response types
Generalize the `CommandResponse[Command]` pattern for all commands.
There seem to be something ambiguous about the way we deal with closing
commands during the initialization phase of the channel. We used to
conflate `CommandResponse[CMD_CLOSE]` and
`ChannelOpenResponse.ChannelClosed` but those are not sent to the same
actors.
It turns out our testing of the `EclairImpl` class is very weak. We
could use this class in `IntegrationSpec` instead of sending raw
messages to channels.
* handle channel-relay in the post-htlc-restart too
* Type fee rates info
Fixes#1188
* Fix vsize comment
This is an alternative to #1425.
This may not correctly represent what Bitcoin Core does, it's likely that
we can in fact use a value smaller than 253, but this shows how we choose
to err on the side of safety with that calculation.
* Add 1008 feerate block target
Fixes#1486
It can be useful to sign arbitrary messages with the key associated with our node_id (to prove ownership of a node).
Adds 2 new API commands:
eclair-cli signmessage --msg=${message}
eclair-cli verifymessage --msg=${message} --sig=${signature}
