The probability for a success forward of x amount on a channel is now
computed as:
P_success(x) = b * P_rene(x)
where P_rene(x) is the probability distribution proposed by Rene
Pickhardt with a uniform distribution on the known liquidity of the
channel and 'b' is a new parameter we add in this PR, by the name of
"base probability". The "base probability" represents the probability
for a channel in the network choosen at random to be able to forward at
least 1msat, ie. of being alive, non-depleted and with at least one HTLC
slot. We don't know the value of 'b', but for the moment we assume that
it is 0.98 and use that as default.
As a consequence the probability cost becomes non-linear and non-convex
because of the additional constant term:
Cost(x) = - log P_success(x)
= - log b - - log P_rene(x)
= - log b + Cost_rene(x)
We currently don't handle well base fees and neither this additional
"base probability" but we can as a first approximation linearize the
cost function in this way:
Cost_linear(x) = (- log b)*x + Cost_rene_linear(x)
Changelog-Added: renepay: Add a dev parameter to renepay that represents
a constant probability of availability for all channels in the network.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Commit 901342b50d ("pyln-testing: require
explicit pattern for BROKEN messages.") changed to a manual scan of
logs rather than using is_in_log, so it needs to manually refresh,
otherwise we miss final log messages.
This causes us to often miss memleak messages, which are printed
only in the exit path!
Reported-by: Lagrang3
Fixes: https://github.com/ElementsProject/lightning/issues/7565
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Mainly, it just causes complaints if we're running a lightningd which is installed
(in which case, my_path is not stolen, unlike running locally installed).
Reported-by: daywalker90
Fixes: https://github.com/ElementsProject/lightning/issues/7569
Changelog-None: regression this release
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Our hash tables allow duplicate keys, and we use that in a few places.
However, the get() function only returns the first, so it's not a good
idea with such hash tables.
Another patch fixes this at a deeper level (using different hash table
types depending on whether this table can have duplicates), but this
is the minimal fix for existing code.
This may be the cause behind us occasionally missing onchain events:
Fixes: https://github.com/ElementsProject/lightning/issues/7460
Fixes: https://github.com/ElementsProject/lightning/issues/7377
Fixes: https://github.com/ElementsProject/lightning/issues/7118
Fixes: https://github.com/ElementsProject/lightning/issues/6951
This fixes them in future: fixing them now will require something else.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: lightningd: occasionally we could miss transaction outputs (not telling gossipd, or even onchaind)
msggen cannot handle the complex type in renepay-exclude,
therefore I added a rule override for it, just like pay-exclude.
```
msggen cln-grpc/proto/node.proto
Traceback (most recent call last):
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/__main__.py", line 131, in <module>
main()
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/__main__.py", line 115, in main
run()
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/__main__.py", line 72, in run
service = load_jsonrpc_service(
^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/utils/utils.py", line 241, in load_jsonrpc_service
methods = [load_jsonrpc_method(name) for name in grpc_method_names]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/utils/utils.py", line 209, in load_jsonrpc_method
request = CompositeField.from_js(schema["methods"][rpc_name]['request'], path=name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/model.py", line 297, in from_js
field = ArrayField.from_js(fpath, ftype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/model.py", line 464, in from_js
itemtype = UnionField.from_js(child_js, path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/model.py", line 393, in from_js
itemtype = PrimitiveField(
^^^^^^^^^^^^^^^
TypeError: PrimitiveField.__init__() missing 2 required positional arguments: 'added' and 'deprecated'
```
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Changelog-EXPERIMENTAL: renepay: add cli option "exclude" to manually disable channels and nodes.
We add a channel filtering paymod that disables channels that have very
low max_htlc. It can be expanded to consider other properties as well,
for instance high base fee, low capacity or high latency.
Changelog-Added: renepay: prune the network by disabling channels we
don't like, eg. very low max_htlc.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Expecting to have more than just a bunch of disabled channels if we
prune the lightning network heavily I am changing the internal data
structure of disabledmap from a simple array to a hashtable.
Have a finer control over disabled channels by targeting
short_channel_id_dir instead of short_channel_id,
ie. we can disable a single direction of a channel.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
If the rpc plugin is run while an older version of reckless is found on
PATH, it produces an error:
2024-08-01T18:32:00.849Z DEBUG plugin-recklessrpc: reckless-stderr:usage: reckless [-h] [-d RECKLESS_DIR] [-l LIGHTNING] [-c CONF] [-r] [--network NETWORK] [-v]
{install,uninstall,search,enable,disable,source,help} ...
reckless: error: unrecognized arguments: --json
Catch this and don't try to parse the output as json.
This allows generic subcommands to be passed to reckless-rpc along with
the target to search/install/etc.. These commands are unvalidated so
far and may crash the reckless process.
Changelog-Added: reckless-rpc plugin: issue commands to reckless over rpc.
This is actually totally fair, and LND, which does not support it without an option, rejects.
Thanks to Vincenzo and roasbeef for debugging this for me!
Fixes: https://github.com/ElementsProject/lightning/issues/7221
Changelog-Fixed: Protocol: we can now open unannounced channels with LND nodes again.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The documentation and the implementation of inform-channel have
different orders for the arguments minimum_msat and maximum_msat.
We change the order in the documentation to match that of the
implementation: minimum_msat comes before maximum_msat.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Often I find this in the logs:
DEBUG plugin-cln-renepay: gossmap ignored 94692259263600 channel updates
Apparently gossmap_refresh does not initialize the input variable num_channel_updates_rejected.
Fix this by explicitely initializing it before calling gossmap_refresh.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
routefail object allocation was linked to route,
we had a crash of the plugin with the following error:
0x561f424fc07a send_backtrace
common/daemon.c:33
0x561f424fc102 crashdump
common/daemon.c:75
0x7f5b0e7dc04f ???
???:0
0x7f5b0e82ae2c ???
???:0
0x7f5b0e7dbfb1 ???
???:0
0x7f5b0e7c6471 ???
???:0
0x561f4252581f call_error
ccan/ccan/tal/tal.c:95
0x561f425258c8 check_bounds
ccan/ccan/tal/tal.c:169
0x561f425258f9 to_tal_hdr
ccan/ccan/tal/tal.c:179
0x561f42526283 tal_free
ccan/ccan/tal/tal.c:525
0x561f424e5379 routefail_end
plugins/renepay/routefail.c:52
0x561f424e557b handle_failure
plugins/renepay/routefail.c:431
apparently there was a race condition for which the route was first
freed before we arrived to routefail_end where we manually free
routefail. I don't see how this could have happened, but anyways
this subtle bug can be avoided by linking the routefail to the payment.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Changelog-Fixed: renepay: finalized routes have to be processed and
determine the payment status even after the payment goes into the
background (no current command active). Not doing so leads to finalized
routes getting stuck in the payment internal data structure and their
associated HTLCs in the uncertainty network don't get released.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
We rather have a payment hash table to look up payments based on the
payment hash than having a list.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Use success_data_from_listsendpays to check if there are "complete"
sendpays instead of imposing the presence of "complete" sendpays as a
precondition for success_data_from_listsendpays.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Simply move the "computed routes" array from the payment to the
routetracker. It makes sense to put all temporary stages of routing into
a single data structure: the routetracker.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
There were some dummy rpc calls to waitblockheight in the payment workflow
to allow the function stack to clear. But it is better if some steps in
the payment are executed "atomically" to avoid strange race conditions.
For example: all steps between getting pending sendpays, computing new
routes and sending those routes.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
It feels unsafe to rely on the internal state of the plugin's database
to tell how many pending sendpays there are for the current payment.
The safest way is to assume lightningd knows and thus use listsendpay
before computing routes.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
- remove payment pointer from routetracker, fetch payment if necessary
from payment_hash;
- "have results" condition as a function call to routetracker.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
The time of decay "TIMER_FORGET_SEC" was set to 1 hour,
which is very low, it would make the plugin try depleted channels after
just a couple of seconds for very small amounts.
I set it to 1 week ~ 6e5 seconds.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
When you have *lots* of events in your bkpr database looking up a
specific event via calling bkpr-listaccountevents and using jq or
grep to filter gets very slow (and wasteful of CPU and disk resources).
This commit adds the paremeter payment_id to the call to filter for a
specific payment id via a where clause in the request to the database of bkpr.
Changelog-Added: Plugins: Add payment_id parameter to bkpr-listaccountevents to filter events.
When a peer connects, we always send all our own gossip (even if they
had set the timestamp filters to filter it out). But we weren't
forcing it out to them when it changed, so this logic only applied to
unstable or frequently-restarting nodes.
So now, we tell all the peers whenever we tell gossipd about our new
gossip.
Fixes: #7276
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: We now send current peers our changed gossip (even if they set timestamp_filter otherwise), not just on reconnect.
We didn't actually check that we *send* the refreshed gossip, just
that we print the message saying we're going to.
So check that everyone received updated gossip when this happens.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
