Commit graph

211 commits

Author SHA1 Message Date
Rusty Russell
37971fb61f plugins/pay: fix capacity bias.
With the warning that we were trying to put "inf" into a u64, we can
see that this calculation was wrong to use integers!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-05 06:12:24 +09:30
Rusty Russell
9366e6b39f cleanup: rename json_add_amount_msat_only to json_add_amount_msat
Now there's no compat variant, we can rename this function.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-18 15:55:49 +10:30
Rusty Russell
780f32dfc6 global: remove deprecated non-msat-named msat fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: all the non-msat-named millisatoshi fields deprecated in v0.12.0.
2023-03-18 15:55:49 +10:30
niftynei
ff0a63a0d7 valgrind-fix: patch valgrind error on log statement in pay plugin
The htlc_budget only exists iff the hint is a 'local' one; we were
failing to write to the htlc_budget field for non-local cases.

To avoid this, we make `local` into a struct that contains the fields
that pertain to local-only payments (in this case, `htlc_budget`).

Valgrind error file: valgrind-errors.1813487
==1813487== Conditional jump or move depends on uninitialised value(s)
==1813487==    at 0x4A9C958: __vfprintf_internal (vfprintf-internal.c:1687)
==1813487==    by 0x4AB0F99: __vsnprintf_internal (vsnprintf.c:114)
==1813487==    by 0x1D2EF9: do_vfmt (str.c:66)
==1813487==    by 0x1D3006: tal_vfmt_ (str.c:92)
==1813487==    by 0x11A60A: paymod_log (libplugin-pay.c:167)
==1813487==    by 0x11B749: payment_chanhints_apply_route (libplugin-pay.c:534)
==1813487==    by 0x11EB36: payment_compute_onion_payloads (libplugin-pay.c:1707)
==1813487==    by 0x12000F: payment_continue (libplugin-pay.c:2135)
==1813487==    by 0x1245B9: adaptive_splitter_cb (libplugin-pay.c:3800)
==1813487==    by 0x11FFB6: payment_continue (libplugin-pay.c:2123)
==1813487==    by 0x1206BC: retry_step_cb (libplugin-pay.c:2301)
==1813487==    by 0x11FFB6: payment_continue (libplugin-pay.c:2123)
==1813487==
{
   <insert_a_suppression_name_here>
   Memcheck:Cond
   fun:__vfprintf_internal
   fun:__vsnprintf_internal
   fun:do_vfmt
   fun:tal_vfmt_
   fun:paymod_log
   fun:payment_chanhints_apply_route
   fun:payment_compute_onion_payloads
   fun:payment_continue
   fun:adaptive_splitter_cb
   fun:payment_continue
   fun:retry_step_cb
[sesh] 0:[tmux]*Z

Suggested-By: @nothingmuch
2023-02-02 18:08:27 -06:00
Rusty Russell
9e9686df20 pay: specify the channel to send the first hop.
If we only specify the node_id, we get the "first" channel.

Closes: #5803
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: `pay` uses the correct local channel for payments when there are multiple available (not just always the first!)
2023-02-03 09:17:33 +10:30
Rusty Russell
ec8aab7cb2 libplugin-pay: fix (transitory) memleak which memleak detection complains about.
We assign this in the loop without freeing it first.

```
 plugin-pay: MEMLEAK: 0x55792b155e18
 plugin-pay:   label=plugins/libplugin-pay.c:3274:struct short_channel_id_dir
 plugin-pay:   backtrace:
 plugin-pay:     ccan/ccan/tal/tal.c:442 (tal_alloc_)
 plugin-pay:     plugins/libplugin-pay.c:3274 (direct_pay_listpeerchannels)
 plugin-pay:     plugins/libplugin.c:860 (handle_rpc_reply)
 plugin-pay:     plugins/libplugin.c:1036 (rpc_read_response_one)
 plugin-pay:     plugins/libplugin.c:1060 (rpc_conn_read_response)
 plugin-pay:     ccan/ccan/io/io.c:59 (next_plan)
 plugin-pay:     ccan/ccan/io/io.c:407 (do_plan)
 plugin-pay:     ccan/ccan/io/io.c:417 (io_ready)
 plugin-pay:     ccan/ccan/io/poll.c:453 (io_loop)
 plugin-pay:     plugins/libplugin.c:1893 (plugin_main)
 plugin-pay:     plugins/pay.c:1294 (main)
 plugin-pay:     ../sysdeps/nptl/libc_start_call_main.h:58 (__libc_start_call_main)
 plugin-pay:     ../csu/libc-start.c:381 (__libc_start_main_impl)
 plugin-pay:   parents:
 plugin-pay:     plugins/libplugin-pay.c:3308:struct direct_pay_data
 plugin-pay:     plugins/libplugin.c:1775:struct plugin
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 09:17:33 +10:30
Rusty Russell
6176912683 plugins/pay: fix htlc_budget calc when we get temporary_channel_failure
Valgrind correctly reports it as uninitialized for this log message, and
the only way this can happen is channel_hints_update() when we receive a
temporary_channel_failure.  Put a dummy value here in this case.

```
Valgrind error file: valgrind-errors.23404
==23404== Conditional jump or move depends on uninitialised value(s)
==23404==    at 0x49E4B56: __vfprintf_internal (vfprintf-internal.c:1516)
==23404==    by 0x49F6519: __vsnprintf_internal (vsnprintf.c:114)
==23404==    by 0x1EBCEB: do_vfmt (str.c:66)
==23404==    by 0x1EBDF8: tal_vfmt_ (str.c:92)
==23404==    by 0x11A336: paymod_log (libplugin-pay.c:167)
==23404==    by 0x11B4B2: payment_chanhints_apply_route (libplugin-pay.c:534)
==23404==    by 0x11E999: payment_compute_onion_payloads (libplugin-pay.c:1707)
==23404==    by 0x11FF4C: payment_continue (libplugin-pay.c:2135)
==23404==    by 0x1245C0: adaptive_splitter_cb (libplugin-pay.c:3800)
==23404==    by 0x11FEF3: payment_continue (libplugin-pay.c:2123)
==23404==    by 0x1205FE: retry_step_cb (libplugin-pay.c:2301)
==23404==    by 0x11FEF3: payment_continue (libplugin-pay.c:2123)
==23404==
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-02 10:11:11 +10:30
Rusty Russell
e9eb5f493b common: update to latest route-blinding spec.
```
make check-source-bolt CHECK_BOLT_PREFIX="--prefix=BOLT-route-blinding" BOLTVERSION=guilt/offers
```

Other than textual changes, this does:

1. Ensures we put total_amount_msat in onion final hop (reported by @t-bast).
2. Require that they put total_amount_msat in onion final hop.
3. Return `invalid_onion_blinding` exactly as defined by the spec (i.e. less
   aggressive when we're the final hop) (also reported by @t-bast, but I already
   knew).

See: #5823
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: `offers` breaking blinded payments change (total_amount_sat required, Eclair compat)
2023-01-30 13:24:29 +10:30
Ken Sedgwick
f29343d740 hsmd: add hsmd_preapprove_invoice and check_preapproveinvoice pay modifier
Changelog-added: hsmd: A new message `hsmd_preapprove_invoice` is added.
Changelog-added: JSON-RPC: A new command `preapproveinvoice` is added.
2023-01-26 21:10:15 -06:00
Rusty Russell
a56c890ae5 plugins: use listpeerchannels instead of listpeers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Rusty Russell
ff2d7e6833 pay: use json_to_listpeers_channels() for local_channel_hints.
Don't parse the listpeers.channels output ourselves: with two extra fields
we can simply reuse json_to_listpeers_channels().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Rusty Russell
5d5b9c6812 libplugin: don't return unopened channels from json_to_listpeers_channels().
This way we always have an SCID and a direction.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Rusty Russell
57dcf68c0b plugins/libplugin: flatten return from json_to_listpeers_result.
Instead of returning a peers -> channels heirarchy, return (as callers
want!) a flat array of channels.

This is actually most of the transition work to make them work with
listpeerchannels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Rusty Russell
02d7454226 db: add invoicerequests table.
We no longer use offers for "I want to send you money", but we'll use
invoice_requests directly.  Create a new table for them, and
associated functions.

The "localofferid" for "pay" and "sendpay" is now "localinvreqid".
This is an experimental-only option, so document the change under
experimental only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `pay` and `sendpay` `localofferid` is now `localinvreqid`.
2022-11-09 15:08:03 +01:00
Rusty Russell
01a47720c3 plugins/libplugin-pay: hack in blinded path support.
We simply take the first one, and route to the start of that.  Then we
append the blinded path to the onion construction.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
426886ff9b lightningd: return invalid_onon_blinding for any blinded payment error.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Christian Decker
759fcb64a8 pay: If the channel_hint matches our allocation allow it
It means we consume the channel completely to the best of our
knowledge, so let that through.

Changelog-Fixed: pay: Squeezed out the last `msat` from our local view of the network
2022-09-21 11:25:47 +02:00
Christian Decker
2def843dce pay: Allow using a channel on equality of estimated capacity
In the case of the local channel we set the estimation to the exact
value spendable, which is important when we want to drain a channel,
because there we actually want to get the last msat.
2022-09-21 11:25:47 +02:00
Rusty Russell
6fe570820e Remove general shadowed variables.
We shadow local variables in several places: generally, these changes
make the code clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-31 12:18:28 +03:00
Christian Decker
2632cddfe4 pay: Fix a memory leak when retrying getroute 2022-08-04 11:14:45 -05:00
Christian Decker
7930e34da3 pay: Populate the channel hints with either the scid or the alias
We'll use one of the two, and we reuse the `scid` field, since we
don't really care that much which one it is.
2022-07-04 22:14:06 +02:00
Christian Decker
1ae3dba529 invoice: Consider aliases too when selecting routehints 2022-07-04 22:14:06 +02:00
Rusty Russell
993f44f289 libplugin: don't be so strict on msat fields.
Let the parsing handle if they're the wrong type (soon they'll be u64).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell
01411d70be common: enforce that msat fields are called "xxx_msat".
Now we've fixed them all, make sure no new ones slip in!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell
cd7e784d6f lightningd: change msatoshi args to amount_msat.
This is consistent with our output changes, and increases consistency.
It also keeps future sanity checks happy, that we only use JSON msat
helpers with '_msat' fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `invoice`, `sendonion`, `sendpay`, `pay`, `keysend`, `fetchinvoice`, `sendinvoice`: `msatoshi` argument is now called `amount_msat` to match other fields.
Changelog-Deprecated: JSON-RPC: `invoice`, `sendonion`, `sendpay`, `pay`, `keysend`, `fetchinvoice`, `sendinvoice` `msatoshi` (use `amount_msat`)
2022-06-21 06:52:35 +09:30
Rusty Russell
f51a3d8ef7 plugins/pay: always include bolt11 (and description) in listpays.
We were setting it on the root, but that doesn't get handed to
sendpay.  Our schema doesn't *require* bolt11, either, so this was
missed (there could be a *bolt12* instead).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `listpays` always includes `bolt11` or `bolt12` field.
2022-04-04 20:26:04 +09:30
Rusty Russell
e47786da04 pay/sendpay: also store description in case bolt11 uses description_hash.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-04 20:26:04 +09:30
Rusty Russell
1c685ce3e8 plugins/pay: send payment_metadata if provided in invoice.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: `pay` (and decode, etc) supports bolt11 payment_metadata a-la https://github.com/lightning/bolts/pull/912
2022-04-02 09:40:18 +10:30
Rusty Russell
dbc77bcbc4 pay: fix leak detect on shadow route.
Short term leak, but leak-detect is right: it's dumb code!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

```
plugin-pay: MEMLEAK: 0x2505318
plugin-pay:   label=common/utils.c:134:char[]
plugin-pay:   backtrace:
plugin-pay:     ccan/ccan/tal/tal.c:442 (tal_alloc_)
plugin-pay:     ccan/ccan/tal/tal.c:471 (tal_alloc_arr_)
plugin-pay:     common/utils.c:134 (tal_hexstr)
plugin-pay:     common/node_id.c:48 (node_id_to_hexstr)
plugin-pay:     common/node_id.c:50 (fmt_node_id_)
plugin-pay:     common/type_to_string.c:32 (type_to_string_)
plugin-pay:     plugins/libplugin-pay.c:2970 (shadow_route_extend)
plugin-pay:     plugins/libplugin-pay.c:3113 (shadow_route_listchannels)
plugin-pay:     plugins/libplugin.c:563 (handle_rpc_reply)
plugin-pay:     plugins/libplugin.c:731 (rpc_read_response_one)
plugin-pay:     plugins/libplugin.c:751 (rpc_conn_read_response)
plugin-pay:     ccan/ccan/io/io.c:59 (next_plan)
plugin-pay:     ccan/ccan/io/io.c:407 (do_plan)
plugin-pay:     ccan/ccan/io/io.c:417 (io_ready)
plugin-pay:     ccan/ccan/io/poll.c:453 (io_loop)
plugin-pay:     plugins/libplugin.c:1565 (plugin_main)
plugin-pay:     plugins/pay.c:2569 (main)
plugin-pay:   parents:
plugin-pay:     plugins/libplugin.c:155:struct out_req
```
2022-03-22 18:45:41 +10:30
Rusty Russell
43a833e405 lightningd: remove support for legacy onion format.
As per proposal in https://github.com/lightning/bolts/pull/962

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: protocol: support for legacy onion format removed, since everyone supports the new one.
2022-03-18 09:20:11 +10:30
Rusty Russell
246b724dbe libplugin-pay: fix spurious leak reports.
1. The dijkstra can be temporary, doesn't need to last as long as pay cmd.
2. We fail multiple times in several places, so don't leak old failreason.
3. Make payments findable by our memleak detector.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Christian Decker
a98ccac777 pay: Disable channels that are not in normal state
Since we have the exact state of the channels from the `listpeers`
response we can filter out the ones that are not yet normal or not
anymore. This is more of a safety net, given that the `gossip_store`
should contain local disables, but better not be racy :-)

Changelog-None
2021-12-14 15:07:45 +10:30
Rusty Russell
4ffda340d3 check: make sure all files outside contrib/ include "config.h" first.
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).

config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Andrew Toth
fa6f01d5b1 Add route exclusion payment modifier and pay rpc exclude arg
Changelog-Added: Add `exclude` option for `pay` command to manually exclude channels or nodes when finding a route.
2021-12-04 21:37:42 +10:30
Simon Vrouwe
e08528b7b5 libplugin-pay: fix valgrind error
for the case rpc "listpeers" returns an error, such as in shutdown
2021-11-30 13:34:44 +10:30
Rene Pickhardt
0ba1bc30fe pay: Use log probability based bias in channel selection
Changelog-Changed: pay: The route selection will now use the log-propability-based channel selection to increase success rate and reduce time to completion
2021-10-22 16:04:37 +02:00
Rusty Russell
233d339061 libplugin-pay: bias towards larger channels.
We bias by channel linearly by capacity, scaled by median fee.
This means that we effectively double the fee if we would use the
entire capacity, and only increase it by 50% if we would only use
1/2 the capacity.

This should drive us towards larger channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `pay` now biases towards larger channels, improving success probability.
2021-10-22 16:04:37 +02:00
Christian Decker
917bea6764 pay: Mark completed payments as such by nullifying cmd
We want to avoid returning duplicate results when cross-completing, so
mark them as completed when we return a result.
2021-10-13 13:41:18 +10:30
Christian Decker
99f6faaabb libplugin: Add callbacks for successful and failed payments
We're about to suspend duplicate calls to `pay` and this will help us
notify them if the original payment completes.
2021-10-13 13:41:18 +10:30
Christian Decker
5f260840ab jsonrpc: Add groupid to waitsendpay 2021-10-13 13:41:18 +10:30
Christian Decker
db84b984e5 pay: Add groupid to the payment struct 2021-10-13 13:41:18 +10:30
Christian Decker
6ad8a8a3c9 pay: Fix wrong result being returned by pay
We were checking against the wrong enum value, resulting in an
erroneous `pending` state as a final result.

Changelog-Fixed: pay: `pay` would sometimes misreport a final state of `pending` instead of `failed`
2021-09-22 09:08:48 +09:30
Jan Sarenik
284ad2bade Fix for Alpine Linux ond OpenBSD
After recent header files clean-up it was not possible to
build c-lightning 7401b2682. This patch fixes it both for
Alpine Linux and OpenBSD.

Proposed-by: nathanael <nathanael@dalliard.ch>

Changelog-None
2021-09-20 14:44:27 +02:00
Rusty Russell
7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
cf1dd779d1 gossmap: don't spew to stderr, include counter for callers.
Fixes: #4722
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: don't drop complaints about silly channels to stderr.
2021-08-24 06:42:32 +09:30
Christian Decker
2510bcad07 pay: Fix another instance of a crash when we can't find ourselves 2021-06-20 14:26:00 +09:30
Christian Decker
eeba75ca7c pay: Do not attempt to filter routehints if we can't find outselves
There are a couple of ways this can happen, but we shouldn't crash.

Fixed #4488
Fixes #4533

Changelog-Fixed: pay: Fixed an issue when filtering routehints when we can't find ourselves in the local network view.
2021-06-05 13:56:01 +09:30
fiatjaf
28953256d0
stop requiring channel and direction on sendonion since they're not used.
Changelog-Changed: `sendonion` no longer requires the gratuitous `direction` and `channel` fields in the `firsthop` parameter.
2021-06-04 10:48:10 +09:30
Christian Decker
5e1fadf799 pay: Skip the presplitter mod if it'd exhaust our HTLC budget
Changelog-Fixed: pay: The presplitter mod will no longer exhaust the HTLC budget.
2021-06-03 16:59:53 +09:30
Christian Decker
803c048a1f pay: Compute the number of HTLCs needed after presplit correctly
We were counting the attempts including the root payment, which
resulted in an off-by-one error with the `test_pay_low_max_htlcs`
test. Counting the children of the root payment after the presplitter
had a go is the correct way to do it, since at that time we only have
one level in the tree, no need to recurse and potentially count
ourselves.
2021-06-03 16:59:53 +09:30