* disable pg lock auto-release in tests
It relies on akka's coordinated shutdown and causes the test jvm to
halt.
* fixup! Remove `messageFlags` from `ChannelUpdate` (#1941)
Add AB testing framework:
- Experiments are added by adding a section in router.path-finding config. Each experiment can have different parameters.
- Traffic is randomly split among the different experiments. The size of each experiment is configurable. 0% experiments don't affect traffic but can be triggered manually with the API.
- Metrics are recorded in the audit database
We make it a serialization detail, which it should be. The `derive`
method from `scodec` makes it very easy to do. We should probably always
use a dedicated class to handle flags, instead of using the `byte` codec
and binary operators.
This allows to remove the `require` in the `ChannelUpdate` definition,
which recently bit us in testing.
The only annoying thing is that we still need to expose a `messageFlags`
method in order to populate the `ChannelDisabled` error message.
We also typeify channel flags, as an alternative to passing around a `Byte`.
When using Tor, your onion address is by default added to your
list of public addresses and advertised through the gossip layer
to allow peers to find your node and connect to it.
We now allow opting out of that mechanism by setting
`tor.publish-onion-address = false` in your `eclair.conf`.
This will ensure your onion address is not advertised, and peers
will not be able to connect to you unless you've connected to
them first.
This lets plugins define variants of the `ChannelRestored` when
using non-standard channel types, while ensuring these channels
are correctly integrated in the list of local channels for payment
relaying.
Add `closing_signed` `fee_range` TLV as described in
https://github.com/lightningnetwork/lightning-rfc/pull/847
We also refactor the negotiating state, add many tests and fix#1742.
Add new fields to the `close` API to let users configure their preferred
fees for mutual close.
Use an event `ChannelUpdateParametersChanged` for the sole purpose of tracking changes to channel_update.
Also, conf change at restore is now treated like a regular fee update. We do handle `CMD_UPDATE_RELAY_FEES` in both `OFFLINE` and `SYNCING`, because there may be a race between `CMD_UPDATE_RELAY_FEES` and
`ChannelRestablish`. And there was no good reason to behave differently in those states anyway.
* fix updateRelayFee api call
The `Register` should be used to channel actors, not the `Router`.
The former tracks all channels, whereas the latter only contains
channels in certain states. We only query the `Router` when we need
reference to external (public) nodes and channels.
The GUI has been deprecated a long time ago, and doesn't inform users about
potential risks, such as RBF-ing funding txs.
We should instead incentivize users to use the CLI and read the documentation,
which ensures they will know about potential pitfalls.
Bolt11 invoices can either contain a string description or a hash
of that description (if it's too big or should stay private).
The API didn't let users provide a description hash, which is now
possible.
Co-authored-by: Pierre-Marie Padiou <pm47@users.noreply.github.com>
We now query blockchain watchdogs over Tor when it's activated.
Some watchdogs are automatically disabled because they have no
support for Tor.
We also let users change the list of watchdogs that should be run
from their `eclair.conf`.
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
For path finding we always need `routeParams`, however all of the messages asking for finding a route have `routeParams` as an optional parameter. This makes it hard to enforce that a given payments uses a given `routeParams` as there may be a `None` somewhere that reverts to the global defaults.
It seems that the reason all the `routeParams` are optional is that the messages asking for finding a route are sometimes used when we already have a route. This is a hacky solution and I've tried to replace these messages by traits that work both when we want to find a route and when we already have it.
I'm trying to enable AB testing which requires using different `routeParams` for different payments and this is a blocker.
* Disable ZMQ high watermark
This should prevent messages from being dropped.
We also configure the socket before subscribing to topics and connecting.
* Switch ZMQ raw block to block hash only
We were receiving raw blocks on the ZMQ socket, whereas we don't use it.
We only use this event as a trigger that a new block has been found, so we
can save bandwidth by switching to block hash subscriptions instead.
* Regularly check blocks
In case we haven't received block events on the ZMQ socket, we regularly
check whether we're behind.
There's a very annoying race condition in that test, and we can end up
emitting events twice on slow machines. This isn't an issue so we stop
verifying that we avoid duplicate events.
* Never retry anchor publish when commit confirmed
We used to check the feerate before checking whether the commit tx was
already confirmed: when the commit feerate was good enough, we would
respawn a publish actor every block whereas the commit tx was already
confirmed.
* Abandon evicted replaceable txs
This ensures the bitcoind wallet won't keep the transaction around and
avoid using its inputs for other transactions.
Fixes#1898
Scale MPP partial amounts based on the total amount we want to
send and the number of parts we allow.
This avoids polluting the results with cheap routes that don't have the
capacity to route when the amount is big.
- Add a base weight ratio. Allows to give more weight to the fee itself. Setting the base to 1 is like not using weight ratios.
- Add checks that the configuration values make sense (all weights should be nonnegative and sum to 1). Checks are performed when loading the configuration.
- Add a virtual cost per hop which allows prioritizing short paths. The cost per hop has a base component and a proportional one, similar to channel fees.
We didn't handle shutdown messages in the negotiating state, whereas they
may be received upon reconnection. This created some unnecessary warnings
in the logs.
We previously made a single payment attempt per trampoline fee.
Since our channel selection for the first attempt is deterministic, if we
encountered a local failure with that channel, the retries with higher
trampoline fees were hitting the exact same error, whereas we should
instead try a different channel.
* Set relay fees per node and save them to database
- Fees are set per node instead of per channel (setting different fees for different channels to the same node is most probably an error)
- Fees are saved to a database so that we can keep a trace of historic fees and new channels with a known node use the fee that we set and not the default fee.
HTLCs are stored in a `Set`, and doing `htlcs.map(_.amount).sum` will
deduplicate identical amounts, resulting in an erroneous balance
calculation. The `Set` must be converted to a `List` first.
Identical htlcs amount are quite frequent due to splitting for AMP.
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
Akka-HTTP can throw this error when we're lazily serializing
response objects. We don't need that laziness, so we set the
strict mode at the top-level of our http stack.
Fixes#1855
* Implement option-upfront-shutdown-script
* Do not activate option_upfront_shutdown_Script by defaut
Users will need to explicitly activate it.
* Send back a warning when we receive an invalid shutdown script
This is a simpler approach to completely parallelizing the handling of
payments, where we simply parallelize the fetch from the database.
This brings a ~30% performance improvement in performance in `PerformanceIntegrationSpec`.
Our test suite is putting a lot of strain on our CI machines, and we start
hitting some timeouts. There were places where we would put a 60 seconds
timeout on an `awaitCond` but inside we'd still use the default 15 seconds
timeout.
Delegate the payment request generation, signature and db write to a short-lived child actor.
There is small (~5%) gain in performance in `PerformanceIntegrationSpec` because what matters is the db write, and it is not parallelized in WAL mode.
Fail outgoing _unsigned_ htlcs on force close, just like we do when disconnected.
Fixes#1829.
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
This test was added to lightning-kmp where this case wasn't correctly
handled: https://github.com/ACINQ/lightning-kmp/pull/278
It is correctly handled by eclair, but an additional test doesn't hurt.
[Write-Ahead Logging](https://sqlite.org/wal.html) is both much more performant in general, and more suited to our particular access patterns.
With a simple throughput performance test, it improves performance by a factor of 5-20x depending on the sync flag.
version | throughput
-------------------------------|-------------
mode=journal sync=normal (*)| 11 htlc/s
mode=journal sync=full| 7 htlc/s
mode=wal sync=normal| 248 htlc/s
mode=wal sync=full (**)| 62 htlc/s
(*) previous setting
(**) new setting
I went with a conservative new setting of wal+full sync, which is both 5x more performant, and more secure than what we had before.
> In WAL mode when synchronous is NORMAL (1), the WAL file is synchronized before each checkpoint and the database file is synchronized after each completed checkpoint and the WAL file header is synchronized when a WAL file begins to be reused after a checkpoint, but no sync operations occur during most transactions. With synchronous=FULL in WAL mode, an additional sync operation of the WAL file happens after each transaction commit. The extra WAL sync following each transaction help ensure that transactions are durable across a power loss. Transactions are consistent with or without the extra syncs provided by synchronous=FULL. If durability is not a concern, then synchronous=NORMAL is normally all one needs in WAL mode.
Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
Move balance check test to their own file instead of
adding bloat to the NormalStateSpec tests.
Remove unnecessary parts that belonged to the
NormalStateSpec test.
