Commit Graph

17 Commits

Author SHA1 Message Date
Lagrang3
b0054aad46 renepay: accomodate fees in the payment flow
Min. Cost Flow does not take into account fees when computing a flow
    with liquidity constraints.
    This is a work-around solution that reduces the amount on every route to
    respect the liquidity bound. The deficity in the delivered amount is
    solved by running MCF once again.

    Changes:

    1. the function `flow_complete` allocates amounts to send over the set of routes
       computed by the MCF algorithm, but it does not allocate more than liquidity
       bound of the route. For this reason `minflow` returns a set of routes that
       satisfy the liquidity bounds but it is not guaranteed that the total payment
       reaches the destination therefore there could a deficit in the delivery:
       `deficit = amount_to_deliver - delivering`.

    2. in the function `add_payflows` after `minflow` returns a set of routes we
       call `flows_fit_amount` that tries to a allocate the `deficit` in the routes
       that the MCF have computed.

    3. if the resulting flows pass all payment constraints then we update
        `amount_to_deliver = amount_to_deliver - delivering`, and the loop
        repeats as long as `amount_to_deliver` is not zero.

    In other words, the excess amount, beyond the liquidity bound,
    in the routes is removed and then we try to allocate it
    into known routes, otherwise we do a whole MCF again just for the
    remaining amount.

    Fixes issue #6599
2024-01-29 10:48:24 +10:30
Lagrang3
7c7d13d6dc renepay: add self-pay feature 2024-01-24 14:42:45 +10:30
Rusty Russell
8b224b6ec2 pytest: don't assume that msat fields get turned into Millisatoshi.
Since the class interacts with int very well now, we don't have to
make explicit accesses, so it's easy to write code which works with
Millisatoshi or int.

Also, don't access rpc.decoder in one test, it's unnecessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-16 10:22:01 +01:00
Rusty Russell
6c15ea44dd pytest: use --developer instead of environment variable.
And we always enable it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell
2bcae6fc76 plugins/renepay: remove DEVELOPER build-time, in favor of runtime.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell
8be888e63f pytest: fix renepay test_shadow flake.
Sometimes we don't shadow because it would break our budget:

```
    def test_shadow(node_factory):
        '''Make sure we shadow correctly.'''
        l1, l2, l3 = node_factory.line_graph(3,
                                             wait_for_announce=True,
                                             opts=[{},
                                                   {'fee-base': 2000, 'fee-per-satoshi': 20, 'cltv-delta': 20},
                                                   {'fee-base': 3000, 'fee-per-satoshi': 30, 'cltv-delta': 30}])
    
        # Shadow doesn't always happen (50% chance)!
        for i in range(20):
            inv = l2.rpc.invoice(123000, f'test_renepay{i}', 'description')['bolt11']
            details = l1.rpc.call('renepay', {'invstring': inv})
            assert details['status'] == 'complete'
            assert details['amount_msat'] == Millisatoshi(123000)
            assert details['destination'] == l2.info['id']
    
>           line = l1.daemon.wait_for_log("No MPP, so added .*msat shadow fee")

tests/test_renepay.py:51:
```

```
lightningd-1 2023-09-12T05:08:39.699Z UNUSUAL plugin-cln-renepay: No shadow fee for flow 0/1: fee would add 5005msat to 123000msat, exceeding budget 128000msat.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-19 10:20:24 +09:30
Rusty Russell
f4f4ab34f3 renepay: fix case where shadow increases amount past payment amount.
Without this, sendpay will refuse to pay.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-31 16:13:21 +09:30
Rusty Russell
a234888209 pytest: test for shadow route additions.
This fails currently, so next commits fix it up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-31 16:13:21 +09:30
Rusty Russell
7a92eb2d89 renepay: calculate fee and delay accumulation correctly.
1. When we add a shadow amount, we were using the wrong channel for
   the fee calculation.
2. Similarly, when calculating the delay amount.

The result is that we can get WIRE_INCORRECT_CLTV_EXPIRY repeatedly
from nodes.

Reported-by: https://github.com/Sjors
Fixes: #6620
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changlog-Experimental: Fixed: `renepay` handles ctlv correctly when it varies along a path.
2023-08-31 16:13:21 +09:30
Rusty Russell
0135d563a0 pytest: write test to ensure renepay the correct channel's fee/cltv info.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-31 16:13:21 +09:30
Rusty Russell
efc0264ba2 renepay: add command notifications
These show that we should clean up our notes.  Here's the result from test_hardmpp:

# we have computed a set of 1 flows with probability 0.328, fees 0msat and delay 23
# No MPP, so added 0msat shadow fee
# Shadow route on flow 0/1 added 0 block delay. now 5
# sendpay flow groupid=1, partid=1, delivering=1800000000msat, probability=0.328
# Update chan knowledge scid=103x2x0, dir=0: [0msat,1799999999msat]
# onion error WIRE_TEMPORARY_CHANNEL_FAILURE from node #1 103x2x0: failed: WIRE_TEMPORARY_CHANNEL_FAILURE (reply from remote)
# we have computed a set of 2 flows with probability 0.115, fees 0msat and delay 23
# Shadow route on flow 0/2 added 0 block delay. now 5
# Shadow route on flow 1/2 added 0 block delay. now 5
# sendpay flow groupid=1, partid=3, delivering=500000000msat, probability=0.475
# sendpay flow groupid=1, partid=2, delivering=1300000000msat, probability=0.242
# Update chan knowledge scid=103x2x0, dir=0: [0msat,1299999999msat]
# onion error WIRE_TEMPORARY_CHANNEL_FAILURE from node #1 103x2x0: failed: WIRE_TEMPORARY_CHANNEL_FAILURE (reply from remote)
# we have computed a set of 2 flows with probability 0.084, fees 0msat and delay 23
# Shadow route on flow 0/2 added 0 block delay. now 5
# Shadow route on flow 1/2 added 0 block delay. now 5
# sendpay flow groupid=1, partid=5, delivering=260000000msat, probability=0.467
# sendpay flow groupid=1, partid=4, delivering=1040000000msat, probability=0.179
# Update chan knowledge scid=103x2x0, dir=0: [0msat,1039999999msat]
# onion error WIRE_TEMPORARY_CHANNEL_FAILURE from node #1 103x2x0: failed: WIRE_TEMPORARY_CHANNEL_FAILURE (reply from remote)
# we have computed a set of 2 flows with probability 0.052, fees 0msat and delay 23
# Shadow route on flow 0/2 added 0 block delay. now 5
# Shadow route on flow 1/2 added 0 block delay. now 5
# sendpay flow groupid=1, partid=7, delivering=120000000msat, probability=0.494
# sendpay flow groupid=1, partid=6, delivering=920000000msat, probability=0.105

Ideally it would look something like:

# Computed 1 flows, probability=0.328:
#  Flow 1: 103x2x0 1800000000msat fee=0msat probability=0.328 shadow=+0msat/0blocks
#  Flow 1: FAIL: TEMPORARY_CHANNEL_FAILURE for 103x2x0.
# Computed 2 flows, probability=0.115:
#  Flow 2: XXX->XXX 1300000000msat fee=XXX, probability=0.475 shadow=+0msat/0blocks
#  Flow 3: XXX->XXX 500000000msat fee=XXX, probability=0.475 shadow=+0msat/0blocks
#  Flow 2: FAIL: TEMPORARY_CHANNEL_FAILURE from node #1 103x2x0
# Computed 2 flows (3 total), probability=0.084, fee=0msat, delay=23
...
#  Flow 4: SUCCESS, 2 in progress should succeed soon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
d7b3bdd420 renepay: drive *all* progress from termination of struct pay_flow.
The main function here is payment_reconsider:
* Each payment has a list of pay_flow.
* This is populated in try_paying(), calling add_payflows & sendpay_new_flows.
* When we get a notification, we resolve a pay_flow using one of the pay_flow_failedxxx
  or pay_flow_succeeded functions.
* They call payment_reconsider() which cleans up finished flows decides what to do:
  often calling try_paying again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
01a29d8453 renepay: fix up handling of errors from final node.
Treat it just like "PAY_TRY_OTHER_ROUTE", except it is from the final node:
this means we correctly process that it "succeeded".

Add a test: this crashes sometimes, but it's cleaned up soon...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Lagrang3
e5695b3c23 renepay: add test
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
2023-07-31 12:58:56 +09:30
Lagrang3
3024afe524 renepay: more cleanups
- adopt "const <type> *"convention
- remove use_shadow option for some pyln tests
- show prob. information of flows into paynotes
- show prob. of success of entire payment flow in paynotes
- minflow: We were not releasing the memory of flow arrays when replacing
  them with a new canditate.
- use memleak_scan_obj in memleak_check
- replace u64 with size_t

Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
2023-07-31 12:58:56 +09:30
Lagrang3
79486c1e3b renepay: fixups after comments
- remove internal gheap checks
- add check for arc_t.chanidx overflow
- remove outdated comments
- check the delta flow bounds before augmenting along a path
- get_flow_paths uses a dynamic tal array instead of a list.
- fix a unit test that depended on the order of returned flows
- fix bug: lightnind doesn't like if I reuse the partid of a failed
  flow, therefore use a higher partid than any of the previous attempts.
- plugin_err instead of LOG_BROKEN if sendpay fails and we cannot get a
  an error code.
- fix wrong comments.
- remove the background timer.
- This is a bugfix. Previous to this the MCF network was built using the
knowledge of the min and max liquidity but it didn't take into account
pending HTLCs.
- Also remove the min_prob_success option but hardcode a 90% value.

Removing some options that are not relevant to the user, they're kept
for developer mode only:
- base_fee_penalty
- min_prob_success
- prob_cost_factor
- remove heap.h, not used

Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
2023-07-31 12:58:56 +09:30
Rusty Russell
b8ca193606 renepay: an experimental payment plugin
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
Changelog-Added: Plugins: `renepay`: an experimental pay plugin implementing Pickhardt payments (`renepay` and `renepaystatus`).
2023-07-31 12:58:56 +09:30