Give different types to init features / node announcement features / invoice features.
We now accept more invoices (we would previously reject invoices if they required a feature that we don't support but that we know is irrelevant for invoices).
Created new files for pages that were in the wiki, but not already in the docs directory. Also made following fixes to README.md and existing files in the docs directory:
* update bolt links to avoid redirect
* link to logging guide from logging section (README.md)
* fixed typo in Backup section and capitalization of Bitcoin Core (README.md)
* Alice does not need trampoline feature enabled (TrampolinePayments.md)
* link to 2021 edition of Trampoline PR (TrampolinePayments.md)
* fixed API examples and removed quotes from password (API.md)
* use --nodeIds for sendtoroute examples (TrampolinePayments.md and MultipartPayments.md)
* update CLI example 3 to use jq (Usage.md)
* fix typo in docs/FAQ.md
* updated Guide.md to point to all pages that are guides
Makes the code a bit cleaner and fixes a bug where `Postman` could respond with both a failure to send and later a `NoReply` after the timeout in case we were expecting a reply.
Refactoring of `PaymentRequest` to pave the way for bolt12 invoices
`PaymentRequest` is renamed to `Invoice` and is now a trait extended by `Bolt11Invoice` and `Bolt12Invoice`. `Bolt12Invoice` will come in a future PR.
Co-authored-by: t-bast <bastuc@hotmail.fr>
If a channel is force-closed before the funding transaction is confirmed,
broadcasting our local commit can be a problem if the funding
tx is double spent. When that happens, the channel stays stuck in the
closing state trying to publish a commit tx with an invalid input.
If we haven't even seen the funding tx in the mempool, we have no way of
being sure that it was double spent, so we would need to keep trying
forever, which pollutes the logs with publishing errors.
Whenever the funding transaction isn't confirmed and we have nothing
at stake, we now directly go to the closed state without publishing our
commitment. This will be an issue for peers who lost state and rely on
us for dataloss protection, but it's not worth exposing ourselves to that
annoying edge case. Our peers should be able to at least keep state
long enough for the funding tx to confirm or for them to force-close.
When we restart, if a downstream channel is closing with a revoked commit,
we should fail the corresponding htlcs in upstream channels.
Otherwise they will be ignored until they timeout, which will cause the
upstream channel to force-close.
If a funding tx is double-spent, we can't publish our commit tx. However,
the previous code would retry very regularly in a loop, polluting the logs.
When that happens, we now only retry when a new block is found.
This has the same kind of impact as the typing of `Features`. In particular we can now explicitly set the values in `eclair.conf`:
`eclair.channel.channel-flags.announce-channel=true`. I took the opportunity to move this channel-related config key in its own config section, with the goal that we move the other fields in a follow-up PR.
It also has the nice side effect of providing a pretty json formatting out of the box:
```json
"channelFlags": {
"announceChannel": true
}
```
The `open` method of the API has been updated: `channelFlags` has been replaced by a new `announceChannel` parameter.
The PaymentInitiator now stores all pending payments, where it previously
only stored trampoline payments (for which it handled the retries).
Whenever the DB doesn't contain any pending payment we also ask the
payment initiator whether a payment attempt is being started, which lets
us provide more accurate information with the `sentinfo` RPC.
An optimistic schedule for confirmation during force-close scenarios
could in theory help malicious peers starve us of our utxos, making it
hard to fee-bump HTLC transactions in high-fee environments.
If we detect that our safe utxos count is low, we try to get force-close
transactions confirmed as quickly as possible, which will yield a new utxo
that can be used to fund other transactions.
Instead of always using the last known peer address from the `PeersDb`,
we randomly pick the address from the `PeersDb` and the `NetworkDb`.
We use clearnet or tor addresses depending on the node configuration:
- tor not supported: always use clearnet addresses
- tor supported but not enabled for clearnet addresses: use clearnet
addresses when available
- tor supported and enabled for clearnet addresses: use tor
addresses when available
When using postgres, at startup we optionnally run a few basic safety
checks, e.g. the number of local channels, how long since the last local
channel update, etc. The goal is to make sure that we are connected to
the correct database instance.
This adds the option to report a remote IP address back to a connecting
peer using the `init` message. A node can decide to use that information
to discover a potential update to its public IP address (NAT) and use
that for a `node_announcement` update message containing the new address.
See https://github.com/lightningnetwork/lightning-rfc/pull/917
* consistency in feature names
Since we're not always following the `option_` prefix from the spec,
let's at least be consistent with ourselves.
* add alternative to `def nodeFees()`
This feature is now ready to be widely deployed.
The transaction format has been tested for a long time
(between Phoenix iOS and our node) and automatic
fee-bumping has been implemented in #2113.
This data class is actually used for more than logging, we also use it to
track the id of a publish request.
We also switch to a different MDC key for transaction publishing instead
of re-using the paymentId one.
We now have this better type to remove ambiguity.
We should use it wherever it makes sense.
There shouldn't be any business logic change in this commit.
* Add deadline information to tx publication
When a channel is force-closed, we tell the `TxPublisher` the block before
which the transaction should be confirmed. Then it will be up to the
`TxPublisher` to set the feerate appropriately and fee-bump whenever
it becomes necessary.
* Evaluate feerate at tx broadcast time
We previously evaluated the feerate when we issued the command to publish
a transaction. However, the transaction may actually be published many
blocks later (especially for HTLC-timeout transactions, for which we may
issue commands long before the timeout has been reached).
We change the command to include the deadline, and evaluate the
feerate to meet that deadline when we're ready to broadcast a valid
transaction.
* Retry conflicting replaceable transactions
We should retry replaceable transactions whenever we find a conflict in
the mempool: as we get closer to the deadline, we will try higher fees.
This ensures we either succeed in replacing the mempool transaction, or
it gets confirmed before the deadline, which is good as well.
In particular, this handles the case where we restart our node. The mempool
conflict can be a transaction that we submitted ourselves, but we've
lost the context in the restart. We want to keep increasing the fees of
that transaction as we get closer to the deadline, so we need this retry
mechanism.
* Regularly bump transaction fees
Once we've published a first version of a replaceable transaction, at every
new block we evaluate whether we should RBF it.
When we choose to RBF, we try to reuse the previous wallet inputs and
simply lower the change amount. When that's not possible, we ask bitcoind
to add more wallet inputs.
We want to be able to send several messages immediately after opening the connection which is not supported by Google's rate limiter that assumes that we had already reached the maximum rate when the rate limiter was created.
We create our own rate limiter that does what we need.
We have a few requirements inside the sphinx code that can make the onion
creation throw exceptions (e.g. onion payloads cannot exceed 1300 bytes).
This usually isn't an issue as our code doesn't generate payloads that
could reach those limits, but the introduction of payment_metadata changes
this: invoices may contain too much data for onion payloads, and we need
to gracefully handle failures.
Note that it has already bitten us when people included many routing hints
in invoices and we were trying to pay with trampoline.
* Filter init, node and invoice features
We should explicitly filter features based on where they can be included
(`init`, `node_announcement` or `invoice`) as specified in Bolt 9.
* Add support for sending `payment_metadata`
Whenever we find a payment metadata field in an invoice, we send it in
the onion payload for the final recipient.
* Include `payment_metadata` in all invoices
We include a payment metadata in every invoice we generate. This lets us
see whether our payers support it or not, which is important data to have
before we make it mandatory and use it for storage-less invoices.
See https://github.com/lightning/bolts/pull/912 for reference.
This makes the distinction between final and replaceable txs more obvious.
Do note that this doesn't mean that final txs cannot be replaced.
Transaction replacement logic may happen outside of the tx publisher
components, which will change the txid and replace the existing attempts.
This effectively reverts #2112 and implements it differently.
Having a single very conservative max value is simpler and the risk of regression is very low. Also fixed a wrong comment, previous max wasn't `11/04/2262` but a huge value.
These transactions are much simpler than HTLC transactions: we don't add
new inputs, we just decrease the output amount and we can unilaterally sign
the updated transaction.
Use the amount WITHOUT the fees when computing the success probability of routing through an edge.
This fixes a problem where the computed probability could be negative, leading to a shorter path after adding an edge.
This can be useful to match a preimage to a given claim-htlc-success
transaction without going through the process of re-creating the transaction
from scratch.
We already include the payment hash in htlc-success transactions for that
reason.
We should regularly run against bitcoind master, to detect if a change to
bitcoind is affecting us.
This will let us detect changes that may break lightning before bitcoind
releases them, giving us a chance to rectify it.
We really don't want to run this workflow for every pull request or every
merge to master, we instead run it twice per week, which should give
us enough time to detect dangerous changes.
This change makes it possible for a non-standard channel to reply
with `CommandFailure` which in turn will be filtered out in `channelsInfo`.
The reason is that currently hosted channels have to return something
to make the whole API call succeed and whatever they return would
break external tools which only expect standard channel formats.
A few of our test helper functions started using too many default arguments,
which makes them a bit messy to use in tests, especially when we want to
add new arguments.
We change this to use method overloading instead, which makes it easier to
add new overloads without changes to existing tests.
In channel tests, we set a 60 seconds timeout on the receiver end,
otherwise we sometimes run into timeout on slow CI machines that run a
lot of tests in parallel.
It makes sense to do the same for onion messages, to help us figure out
whether there is a race condition or it's just the machines that are slow.
With anchor outputs, we need to keep utxos available for fee bumping.
Having enough on-chain funds to claim HTLCs in case channels force-close
is necessary to guarantee funds safety.
There is no perfect formula to estimate how much we should reserve: it
depends on the feerate when the channels force-close, the number of
impacted HTLCs and their amount.
We use a very rough estimation, to avoid needlessly spamming node operators
while still notifying them when the situation becomes risky.
