Without knowing what method was called, we can't have useful general logging
methods, so go through the pain of adding "const char *method" everywhere,
and add:
1. ignore_and_complete - we're done when jsonrpc returned
2. log_broken_and_complete - we're done, but emit BROKEN log.
3. plugin_broken_cb - if this happens, fail the plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When we used to allow cmd to be NULL, we had to hand the plugin
everywhere. We no longer do.
1. Various jsonrpc_ functions no longer need the plugin arg.
2. send_outreq no longer needs a plugin arg.
3. The init function takes a command, not a plugin.
4. Remove command_deprecated_in_nocmd_ok.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is cleaner: everything can now be associated with a command
context.
You're supposed to eventually dispose of it using timer_complete().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This remove an unnecessary check for existing description field if the
description_hash is provided in the invoice. The bolt11_decode function
already checks the description against the hash if both are provided.
Changelog-Fix: renepay: allow to pay BOLT11 invoices with description_hash, the description field is made optional
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
This is obsolete (since modern onions) and so removed from spec.
We should not set it, and don't need to handle it specially.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It was weird not to have a capacity associated with localmods channels, and
fixing it has some very nice side effects.
Now the gossmap_chan_get_capacity() call never fails (we prevented reading
of channels from gossmap in the partially-written case already), so we
make it return the capacity. We do this in msat, because that's what
all the callers want.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is actually what we want in several places: to only override one or
two fields in a channel_update.
We add a gossmap_local_setchan() with a similar API to the old
gossmap_local_updatechan(), for the case where we want to set every
field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I used `amount_msat_eq(x, AMOUNT_MSAT(0))` because I forgot this
function existed. I probably missed it because the name is surprising,
so add "is" in there to make it clear it's a boolean function.
You'll note almost all the places which did use it are Eduardo's and
Lisa's code, so maybe it's just me.
Fix up a few places which I could use it, too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Channel filter must apply to the modified gossmap+localmods,
otherwise we disable local channels with htlcmax=0.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Add a little bit of uncertainty to the local channels to avoid
consuming precisely all spendable_msat on our side, which leads to
temporary channel failure if the spendable_msat changes during the
course of the payment.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Refresh gossmap once before we read the routehints.
Make sure that if channels in the routehints are public,
we retrieve their capacities from gossip.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
- add more checks
- add more error messages
- compute probabilities without fees during MCF
- compute probabilities with fees during get_routes
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
In theory we should not have htlc_total<=known_max.
But for some strange race condition we do sometimes.
Until we find a solution to ensure the correct state
of the uncertainty network we remove the assertion.
Thanks to signed arithmetic and MIN guards, the rest
of the code in linearize_channel can handle the
weird cases with known_max<htlc_total.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
The probability for a success forward of x amount on a channel is now
computed as:
P_success(x) = b * P_rene(x)
where P_rene(x) is the probability distribution proposed by Rene
Pickhardt with a uniform distribution on the known liquidity of the
channel and 'b' is a new parameter we add in this PR, by the name of
"base probability". The "base probability" represents the probability
for a channel in the network choosen at random to be able to forward at
least 1msat, ie. of being alive, non-depleted and with at least one HTLC
slot. We don't know the value of 'b', but for the moment we assume that
it is 0.98 and use that as default.
As a consequence the probability cost becomes non-linear and non-convex
because of the additional constant term:
Cost(x) = - log P_success(x)
= - log b - - log P_rene(x)
= - log b + Cost_rene(x)
We currently don't handle well base fees and neither this additional
"base probability" but we can as a first approximation linearize the
cost function in this way:
Cost_linear(x) = (- log b)*x + Cost_rene_linear(x)
Changelog-Added: renepay: Add a dev parameter to renepay that represents
a constant probability of availability for all channels in the network.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
msggen cannot handle the complex type in renepay-exclude,
therefore I added a rule override for it, just like pay-exclude.
```
msggen cln-grpc/proto/node.proto
Traceback (most recent call last):
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/__main__.py", line 131, in <module>
main()
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/__main__.py", line 115, in main
run()
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/__main__.py", line 72, in run
service = load_jsonrpc_service(
^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/utils/utils.py", line 241, in load_jsonrpc_service
methods = [load_jsonrpc_method(name) for name in grpc_method_names]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/utils/utils.py", line 209, in load_jsonrpc_method
request = CompositeField.from_js(schema["methods"][rpc_name]['request'], path=name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/model.py", line 297, in from_js
field = ArrayField.from_js(fpath, ftype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/model.py", line 464, in from_js
itemtype = UnionField.from_js(child_js, path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/model.py", line 393, in from_js
itemtype = PrimitiveField(
^^^^^^^^^^^^^^^
TypeError: PrimitiveField.__init__() missing 2 required positional arguments: 'added' and 'deprecated'
```
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Changelog-EXPERIMENTAL: renepay: add cli option "exclude" to manually disable channels and nodes.
We add a channel filtering paymod that disables channels that have very
low max_htlc. It can be expanded to consider other properties as well,
for instance high base fee, low capacity or high latency.
Changelog-Added: renepay: prune the network by disabling channels we
don't like, eg. very low max_htlc.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Expecting to have more than just a bunch of disabled channels if we
prune the lightning network heavily I am changing the internal data
structure of disabledmap from a simple array to a hashtable.
Have a finer control over disabled channels by targeting
short_channel_id_dir instead of short_channel_id,
ie. we can disable a single direction of a channel.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Often I find this in the logs:
DEBUG plugin-cln-renepay: gossmap ignored 94692259263600 channel updates
Apparently gossmap_refresh does not initialize the input variable num_channel_updates_rejected.
Fix this by explicitely initializing it before calling gossmap_refresh.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
routefail object allocation was linked to route,
we had a crash of the plugin with the following error:
0x561f424fc07a send_backtrace
common/daemon.c:33
0x561f424fc102 crashdump
common/daemon.c:75
0x7f5b0e7dc04f ???
???:0
0x7f5b0e82ae2c ???
???:0
0x7f5b0e7dbfb1 ???
???:0
0x7f5b0e7c6471 ???
???:0
0x561f4252581f call_error
ccan/ccan/tal/tal.c:95
0x561f425258c8 check_bounds
ccan/ccan/tal/tal.c:169
0x561f425258f9 to_tal_hdr
ccan/ccan/tal/tal.c:179
0x561f42526283 tal_free
ccan/ccan/tal/tal.c:525
0x561f424e5379 routefail_end
plugins/renepay/routefail.c:52
0x561f424e557b handle_failure
plugins/renepay/routefail.c:431
apparently there was a race condition for which the route was first
freed before we arrived to routefail_end where we manually free
routefail. I don't see how this could have happened, but anyways
this subtle bug can be avoided by linking the routefail to the payment.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Changelog-Fixed: renepay: finalized routes have to be processed and
determine the payment status even after the payment goes into the
background (no current command active). Not doing so leads to finalized
routes getting stuck in the payment internal data structure and their
associated HTLCs in the uncertainty network don't get released.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
We rather have a payment hash table to look up payments based on the
payment hash than having a list.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Use success_data_from_listsendpays to check if there are "complete"
sendpays instead of imposing the presence of "complete" sendpays as a
precondition for success_data_from_listsendpays.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Simply move the "computed routes" array from the payment to the
routetracker. It makes sense to put all temporary stages of routing into
a single data structure: the routetracker.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
There were some dummy rpc calls to waitblockheight in the payment workflow
to allow the function stack to clear. But it is better if some steps in
the payment are executed "atomically" to avoid strange race conditions.
For example: all steps between getting pending sendpays, computing new
routes and sending those routes.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
It feels unsafe to rely on the internal state of the plugin's database
to tell how many pending sendpays there are for the current payment.
The safest way is to assume lightningd knows and thus use listsendpay
before computing routes.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
- remove payment pointer from routetracker, fetch payment if necessary
from payment_hash;
- "have results" condition as a function call to routetracker.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
The time of decay "TIMER_FORGET_SEC" was set to 1 hour,
which is very low, it would make the plugin try depleted channels after
just a couple of seconds for very small amounts.
I set it to 1 week ~ 6e5 seconds.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
When renepay starts, one of the first operations it does is to check for
pending sendpays for the same invoice. Those are added up to an internal
database to keep track of. Also the amount they deliver to the
destination is computed so that the current payment rpc call would try
to complete the payment for whatever amount remains to pay.
For an incomplete payment after calling the RPC renepay I found this in
the logs:
2024-05-22T19:44:19.853Z DEBUG plugin-cln-renepay: There are pending sendpays to this invoice. groupid = 6 delivering = 0msat, last_partid = 10
Where delivering should be the sum of the amounts delivered by pending routes.
This avoids globals (and means memleak traverses the variables!): we
only change over the test plugin though, to avoid unnecessary churn.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Add a test that checks `get_route` and `uncertainty_update` behaviour's
handling of missing channel capacities.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
```
plugins/renepay/mods.c: In function 'payment_continue':
plugins/renepay/mods.c:63:7: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
63 | (u64)payment_virtual_program[payment->exec_state++];
| ^
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
