* consistency in feature names
Since we're not always following the `option_` prefix from the spec,
let's at least be consistent with ourselves.
* add alternative to `def nodeFees()`
This feature is now ready to be widely deployed.
The transaction format has been tested for a long time
(between Phoenix iOS and our node) and automatic
fee-bumping has been implemented in #2113.
This data class is actually used for more than logging, we also use it to
track the id of a publish request.
We also switch to a different MDC key for transaction publishing instead
of re-using the paymentId one.
We now have this better type to remove ambiguity.
We should use it wherever it makes sense.
There shouldn't be any business logic change in this commit.
* Add deadline information to tx publication
When a channel is force-closed, we tell the `TxPublisher` the block before
which the transaction should be confirmed. Then it will be up to the
`TxPublisher` to set the feerate appropriately and fee-bump whenever
it becomes necessary.
* Evaluate feerate at tx broadcast time
We previously evaluated the feerate when we issued the command to publish
a transaction. However, the transaction may actually be published many
blocks later (especially for HTLC-timeout transactions, for which we may
issue commands long before the timeout has been reached).
We change the command to include the deadline, and evaluate the
feerate to meet that deadline when we're ready to broadcast a valid
transaction.
* Retry conflicting replaceable transactions
We should retry replaceable transactions whenever we find a conflict in
the mempool: as we get closer to the deadline, we will try higher fees.
This ensures we either succeed in replacing the mempool transaction, or
it gets confirmed before the deadline, which is good as well.
In particular, this handles the case where we restart our node. The mempool
conflict can be a transaction that we submitted ourselves, but we've
lost the context in the restart. We want to keep increasing the fees of
that transaction as we get closer to the deadline, so we need this retry
mechanism.
* Regularly bump transaction fees
Once we've published a first version of a replaceable transaction, at every
new block we evaluate whether we should RBF it.
When we choose to RBF, we try to reuse the previous wallet inputs and
simply lower the change amount. When that's not possible, we ask bitcoind
to add more wallet inputs.
We want to be able to send several messages immediately after opening the connection which is not supported by Google's rate limiter that assumes that we had already reached the maximum rate when the rate limiter was created.
We create our own rate limiter that does what we need.
We have a few requirements inside the sphinx code that can make the onion
creation throw exceptions (e.g. onion payloads cannot exceed 1300 bytes).
This usually isn't an issue as our code doesn't generate payloads that
could reach those limits, but the introduction of payment_metadata changes
this: invoices may contain too much data for onion payloads, and we need
to gracefully handle failures.
Note that it has already bitten us when people included many routing hints
in invoices and we were trying to pay with trampoline.
* Filter init, node and invoice features
We should explicitly filter features based on where they can be included
(`init`, `node_announcement` or `invoice`) as specified in Bolt 9.
* Add support for sending `payment_metadata`
Whenever we find a payment metadata field in an invoice, we send it in
the onion payload for the final recipient.
* Include `payment_metadata` in all invoices
We include a payment metadata in every invoice we generate. This lets us
see whether our payers support it or not, which is important data to have
before we make it mandatory and use it for storage-less invoices.
See https://github.com/lightning/bolts/pull/912 for reference.
This makes the distinction between final and replaceable txs more obvious.
Do note that this doesn't mean that final txs cannot be replaced.
Transaction replacement logic may happen outside of the tx publisher
components, which will change the txid and replace the existing attempts.
This effectively reverts #2112 and implements it differently.
Having a single very conservative max value is simpler and the risk of regression is very low. Also fixed a wrong comment, previous max wasn't `11/04/2262` but a huge value.
These transactions are much simpler than HTLC transactions: we don't add
new inputs, we just decrease the output amount and we can unilaterally sign
the updated transaction.
Use the amount WITHOUT the fees when computing the success probability of routing through an edge.
This fixes a problem where the computed probability could be negative, leading to a shorter path after adding an edge.
This can be useful to match a preimage to a given claim-htlc-success
transaction without going through the process of re-creating the transaction
from scratch.
We already include the payment hash in htlc-success transactions for that
reason.
We should regularly run against bitcoind master, to detect if a change to
bitcoind is affecting us.
This will let us detect changes that may break lightning before bitcoind
releases them, giving us a chance to rectify it.
We really don't want to run this workflow for every pull request or every
merge to master, we instead run it twice per week, which should give
us enough time to detect dangerous changes.
This change makes it possible for a non-standard channel to reply
with `CommandFailure` which in turn will be filtered out in `channelsInfo`.
The reason is that currently hosted channels have to return something
to make the whole API call succeed and whatever they return would
break external tools which only expect standard channel formats.
A few of our test helper functions started using too many default arguments,
which makes them a bit messy to use in tests, especially when we want to
add new arguments.
We change this to use method overloading instead, which makes it easier to
add new overloads without changes to existing tests.
In channel tests, we set a 60 seconds timeout on the receiver end,
otherwise we sometimes run into timeout on slow CI machines that run a
lot of tests in parallel.
It makes sense to do the same for onion messages, to help us figure out
whether there is a race condition or it's just the machines that are slow.
With anchor outputs, we need to keep utxos available for fee bumping.
Having enough on-chain funds to claim HTLCs in case channels force-close
is necessary to guarantee funds safety.
There is no perfect formula to estimate how much we should reserve: it
depends on the feerate when the channels force-close, the number of
impacted HTLCs and their amount.
We use a very rough estimation, to avoid needlessly spamming node operators
while still notifying them when the situation becomes risky.
We were previously subscribing to all payment events, which includes
`PaymentSettlingOnChain`, for which there is nothing to do at the
DbEventHandler level.
We now register to each concrete event instead of registering to a generic
trait.
We've seen users bump into these limits often because many nodes now run
behind Tor on poor hardware, so it makes sense to make our default values
more robust.
The switchboard is our singleton actor entry point to all of our peers and
peer connections, it can be useful to have it deliver basic information
about our peers.
We also take this opportunity to fix a bug: we were not emitting the
`LastChannelClosed` event when it happened while the peer was
disconnected.
We introduced a task to regularly compute network statistics (mostly about
channel parameters such as expiry and fees).
The goal was to use this information in the MPP split algorithm to decide
whether to split a payment or not.
But we haven't used it, and I'm not sure anymore that it's useful at all.
If node operators are interested in network statistics, an ad-hoc
on-the-fly computation would make more sense.
When using the API to update the relay fees of a channel, the changes are saved in the DB and then the channel is updated. If the channel can't be updated, because it is not ready yet for instance, that's fine because it will use the values from the DB as soon as it is ready. So there is no need to return an error in that case.
Fixes#2085
Whenever a remote commitment is confirmed in which we received htlcs, we
generate a claim-htlc-success transaction even if we don't have the preimage
yet (so we can't broadcast it).
This created a confusing log line:
"tx generation success: desc=claim-htlc-success"
We want to create that log line only when we have the preimage and the node
operator can expect the HTLC to be redeemed on-chain.
For some reason, when sending to a pre-defined channel route, we only
looked at the public channel graph. This was incorrect, we should also
include private channels and routing hints.
Fixes#2081
If the remote commit confirms before our local commit, there is no reason
to try to publish our HTLC transactions, we will instead directly claim
the htlc outputs from the remote commit.
We previously checked timelocks before checking preconditions, which in
this case means we would be waiting for a confirmation on our local commit
forever. We now check preconditions before timelocks, and added a
precondition that verifies that the remote commit isn't confirmed before
publishing our HTLC txs.
The output of `getsentinfo` didn't include the `nodeId` of the failing node.
This PR adds it, as it can be used by external apps when they build routes
themselves instead of relying on eclair's internals (e.g. channel rebalancing).
The scodec magic was quite hard to read, and the use of the prefix wasn't
very intuitive since Sphinx uses both a prefix and a suffix.
Also added more codec tests.
We rename the EncryptedRecipientData types.
The data it contains is namespaced to usages for route blinding, so we
make that explicit.
This way if future scenarios use another kind of encrypted tlv stream
we won't have name clashes (e.g. encrypted state backup).
We also update the route blinding test vectors to the final spec version.
Add basic support for onion messages (lightning/bolts#759)
Add functions and codecs to create, read and process onion messages. Does not use any of them yet.
We previously relied on bitcoind's dumpprivkey RPC in some of our tests.
That RPC isn't available with descriptor wallets, and descriptor wallets
are now the default wallet type.
We previously created restrictions in Sphinx.scala to only allow using it
for two types of onions: a 1300 bytes one for HTLCs and a 400 bytes one
for trampoline.
This doesn't make sense anymore. The latest version of trampoline allows
any onion size, and onion messages also allow any onion size. The Sphinx
protocol doesn't care either about the size of the payload.
Another reason to remove it is that it wasn't working that well with
pattern matching because of type erasure.
So now the caller must explicitly set the length of the payload, which is
more flexible. Verifying that the correct length is used is deferred to
higher level components.
Fixes#1995, which was due to a pattern matching error for the expected response type of `sendToX` helper methods in `EclairImpl`, and had nothing to do with json serialization. Added a few non-reg tests.
In the second commit I also set a basic "ok" json serializer for all default `RES_SUCCESS` messages, but didn't follow https://github.com/ACINQ/eclair/issues/1995#issuecomment-940821678, because we would either completely break backwards compatibility, or create inconsistency with non-default command responses like `RES_GETINFO`, and with other API calls not related to channels.
If a _local_ mutual close transaction is published from outside of the actor state machine, the channel will fail to recognize it, and will move to the `ERR_INFORMATION_LEAK` state. We could instead log a warning and handle it gracefully, since no harm has been done.
This is different from a local force close, because we do not keep the fully-signed local commit tx anymore, so an unexpected published tx would indeed be very fishy in that case. But we do store the best fully-signed, read-to-publish mutual close tx in the channel data so we must be ready to handle the case where the operator manually publishes it for whatever reason.
We use to store UNIX timestamps in the `waitingSince` field before
moving to block count. In order to ensure backward compatibility, we
converted from timestamps to blockheight based on the value. Anything
over 1 500 000 was considered a timestamp. But this value is much too
low: on testnet the blockheight is already greater than 2 000 000.
We can use 1 500 000 000 instead, which is somewhere in 2017.
Another way to deal with this would be to simply remove this
compatibility code.
