Of course, we still will, since spendable is for a single HTLC, but
this also shows why we should treat *minimum* as the incorrect answer
if they cross, too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/7563
It seems we didn't handle it correctly: we need to cap the first
segment as well as the others, as far as I can tell.
Also, it can be less than the maximum capacity.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Conversion is lossy, and we don't want to spend more than the channel,
so it's conservative to round down here.
This doesn't actually help our test though!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
You need to know it to make an onion, and in theory if we decided to
fuzz it could be different for different paths.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Channel filter must apply to the modified gossmap+localmods,
otherwise we disable local channels with htlcmax=0.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Add a little bit of uncertainty to the local channels to avoid
consuming precisely all spendable_msat on our side, which leads to
temporary channel failure if the spendable_msat changes during the
course of the payment.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Refresh gossmap once before we read the routehints.
Make sure that if channels in the routehints are public,
we retrieve their capacities from gossip.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
- add more checks
- add more error messages
- compute probabilities without fees during MCF
- compute probabilities with fees during get_routes
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
In theory we should not have htlc_total<=known_max.
But for some strange race condition we do sometimes.
Until we find a solution to ensure the correct state
of the uncertainty network we remove the assertion.
Thanks to signed arithmetic and MIN guards, the rest
of the code in linearize_channel can handle the
weird cases with known_max<htlc_total.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
The probability for a success forward of x amount on a channel is now
computed as:
P_success(x) = b * P_rene(x)
where P_rene(x) is the probability distribution proposed by Rene
Pickhardt with a uniform distribution on the known liquidity of the
channel and 'b' is a new parameter we add in this PR, by the name of
"base probability". The "base probability" represents the probability
for a channel in the network choosen at random to be able to forward at
least 1msat, ie. of being alive, non-depleted and with at least one HTLC
slot. We don't know the value of 'b', but for the moment we assume that
it is 0.98 and use that as default.
As a consequence the probability cost becomes non-linear and non-convex
because of the additional constant term:
Cost(x) = - log P_success(x)
= - log b - - log P_rene(x)
= - log b + Cost_rene(x)
We currently don't handle well base fees and neither this additional
"base probability" but we can as a first approximation linearize the
cost function in this way:
Cost_linear(x) = (- log b)*x + Cost_rene_linear(x)
Changelog-Added: renepay: Add a dev parameter to renepay that represents
a constant probability of availability for all channels in the network.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
msggen cannot handle the complex type in renepay-exclude,
therefore I added a rule override for it, just like pay-exclude.
```
msggen cln-grpc/proto/node.proto
Traceback (most recent call last):
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/__main__.py", line 131, in <module>
main()
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/__main__.py", line 115, in main
run()
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/__main__.py", line 72, in run
service = load_jsonrpc_service(
^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/utils/utils.py", line 241, in load_jsonrpc_service
methods = [load_jsonrpc_method(name) for name in grpc_method_names]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/utils/utils.py", line 209, in load_jsonrpc_method
request = CompositeField.from_js(schema["methods"][rpc_name]['request'], path=name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/model.py", line 297, in from_js
field = ArrayField.from_js(fpath, ftype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/model.py", line 464, in from_js
itemtype = UnionField.from_js(child_js, path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/model.py", line 393, in from_js
itemtype = PrimitiveField(
^^^^^^^^^^^^^^^
TypeError: PrimitiveField.__init__() missing 2 required positional arguments: 'added' and 'deprecated'
```
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Changelog-EXPERIMENTAL: renepay: add cli option "exclude" to manually disable channels and nodes.
We add a channel filtering paymod that disables channels that have very
low max_htlc. It can be expanded to consider other properties as well,
for instance high base fee, low capacity or high latency.
Changelog-Added: renepay: prune the network by disabling channels we
don't like, eg. very low max_htlc.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Expecting to have more than just a bunch of disabled channels if we
prune the lightning network heavily I am changing the internal data
structure of disabledmap from a simple array to a hashtable.
Have a finer control over disabled channels by targeting
short_channel_id_dir instead of short_channel_id,
ie. we can disable a single direction of a channel.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
If the rpc plugin is run while an older version of reckless is found on
PATH, it produces an error:
2024-08-01T18:32:00.849Z DEBUG plugin-recklessrpc: reckless-stderr:usage: reckless [-h] [-d RECKLESS_DIR] [-l LIGHTNING] [-c CONF] [-r] [--network NETWORK] [-v]
{install,uninstall,search,enable,disable,source,help} ...
reckless: error: unrecognized arguments: --json
Catch this and don't try to parse the output as json.
This allows generic subcommands to be passed to reckless-rpc along with
the target to search/install/etc.. These commands are unvalidated so
far and may crash the reckless process.
Changelog-Added: reckless-rpc plugin: issue commands to reckless over rpc.
Often I find this in the logs:
DEBUG plugin-cln-renepay: gossmap ignored 94692259263600 channel updates
Apparently gossmap_refresh does not initialize the input variable num_channel_updates_rejected.
Fix this by explicitely initializing it before calling gossmap_refresh.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
routefail object allocation was linked to route,
we had a crash of the plugin with the following error:
0x561f424fc07a send_backtrace
common/daemon.c:33
0x561f424fc102 crashdump
common/daemon.c:75
0x7f5b0e7dc04f ???
???:0
0x7f5b0e82ae2c ???
???:0
0x7f5b0e7dbfb1 ???
???:0
0x7f5b0e7c6471 ???
???:0
0x561f4252581f call_error
ccan/ccan/tal/tal.c:95
0x561f425258c8 check_bounds
ccan/ccan/tal/tal.c:169
0x561f425258f9 to_tal_hdr
ccan/ccan/tal/tal.c:179
0x561f42526283 tal_free
ccan/ccan/tal/tal.c:525
0x561f424e5379 routefail_end
plugins/renepay/routefail.c:52
0x561f424e557b handle_failure
plugins/renepay/routefail.c:431
apparently there was a race condition for which the route was first
freed before we arrived to routefail_end where we manually free
routefail. I don't see how this could have happened, but anyways
this subtle bug can be avoided by linking the routefail to the payment.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Changelog-Fixed: renepay: finalized routes have to be processed and
determine the payment status even after the payment goes into the
background (no current command active). Not doing so leads to finalized
routes getting stuck in the payment internal data structure and their
associated HTLCs in the uncertainty network don't get released.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
We rather have a payment hash table to look up payments based on the
payment hash than having a list.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Use success_data_from_listsendpays to check if there are "complete"
sendpays instead of imposing the presence of "complete" sendpays as a
precondition for success_data_from_listsendpays.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Simply move the "computed routes" array from the payment to the
routetracker. It makes sense to put all temporary stages of routing into
a single data structure: the routetracker.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
There were some dummy rpc calls to waitblockheight in the payment workflow
to allow the function stack to clear. But it is better if some steps in
the payment are executed "atomically" to avoid strange race conditions.
For example: all steps between getting pending sendpays, computing new
routes and sending those routes.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
It feels unsafe to rely on the internal state of the plugin's database
to tell how many pending sendpays there are for the current payment.
The safest way is to assume lightningd knows and thus use listsendpay
before computing routes.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
- remove payment pointer from routetracker, fetch payment if necessary
from payment_hash;
- "have results" condition as a function call to routetracker.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
The time of decay "TIMER_FORGET_SEC" was set to 1 hour,
which is very low, it would make the plugin try depleted channels after
just a couple of seconds for very small amounts.
I set it to 1 week ~ 6e5 seconds.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
When you have *lots* of events in your bkpr database looking up a
specific event via calling bkpr-listaccountevents and using jq or
grep to filter gets very slow (and wasteful of CPU and disk resources).
This commit adds the paremeter payment_id to the call to filter for a
specific payment id via a where clause in the request to the database of bkpr.
Changelog-Added: Plugins: Add payment_id parameter to bkpr-listaccountevents to filter events.
Did a sweep to find any others, give this from sanitizer:
```
2024-08-09T18:06:45.1729472Z plugins/bkpr/recorder.c:2057:23: runtime error: load of value 190, which is not a valid value for type 'bool'
2024-08-09T18:06:45.1729877Z SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior plugins/bkpr/recorder.c:2057:23 in
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
