Commit graph

1043 commits

Author SHA1 Message Date
Rusty Russell
35acdc112f offers: fix pay where we are using deprecated apis.
In this case, "fee_base_msat" from `listincoming` has `msat` appended.

Fixes: #5850
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 16:59:51 +01:00
niftynei
ff0a63a0d7 valgrind-fix: patch valgrind error on log statement in pay plugin
The htlc_budget only exists iff the hint is a 'local' one; we were
failing to write to the htlc_budget field for non-local cases.

To avoid this, we make `local` into a struct that contains the fields
that pertain to local-only payments (in this case, `htlc_budget`).

Valgrind error file: valgrind-errors.1813487
==1813487== Conditional jump or move depends on uninitialised value(s)
==1813487==    at 0x4A9C958: __vfprintf_internal (vfprintf-internal.c:1687)
==1813487==    by 0x4AB0F99: __vsnprintf_internal (vsnprintf.c:114)
==1813487==    by 0x1D2EF9: do_vfmt (str.c:66)
==1813487==    by 0x1D3006: tal_vfmt_ (str.c:92)
==1813487==    by 0x11A60A: paymod_log (libplugin-pay.c:167)
==1813487==    by 0x11B749: payment_chanhints_apply_route (libplugin-pay.c:534)
==1813487==    by 0x11EB36: payment_compute_onion_payloads (libplugin-pay.c:1707)
==1813487==    by 0x12000F: payment_continue (libplugin-pay.c:2135)
==1813487==    by 0x1245B9: adaptive_splitter_cb (libplugin-pay.c:3800)
==1813487==    by 0x11FFB6: payment_continue (libplugin-pay.c:2123)
==1813487==    by 0x1206BC: retry_step_cb (libplugin-pay.c:2301)
==1813487==    by 0x11FFB6: payment_continue (libplugin-pay.c:2123)
==1813487==
{
   <insert_a_suppression_name_here>
   Memcheck:Cond
   fun:__vfprintf_internal
   fun:__vsnprintf_internal
   fun:do_vfmt
   fun:tal_vfmt_
   fun:paymod_log
   fun:payment_chanhints_apply_route
   fun:payment_compute_onion_payloads
   fun:payment_continue
   fun:adaptive_splitter_cb
   fun:payment_continue
   fun:retry_step_cb
[sesh] 0:[tmux]*Z

Suggested-By: @nothingmuch
2023-02-02 18:08:27 -06:00
Rusty Russell
9e9686df20 pay: specify the channel to send the first hop.
If we only specify the node_id, we get the "first" channel.

Closes: #5803
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: `pay` uses the correct local channel for payments when there are multiple available (not just always the first!)
2023-02-03 09:17:33 +10:30
Rusty Russell
ec8aab7cb2 libplugin-pay: fix (transitory) memleak which memleak detection complains about.
We assign this in the loop without freeing it first.

```
 plugin-pay: MEMLEAK: 0x55792b155e18
 plugin-pay:   label=plugins/libplugin-pay.c:3274:struct short_channel_id_dir
 plugin-pay:   backtrace:
 plugin-pay:     ccan/ccan/tal/tal.c:442 (tal_alloc_)
 plugin-pay:     plugins/libplugin-pay.c:3274 (direct_pay_listpeerchannels)
 plugin-pay:     plugins/libplugin.c:860 (handle_rpc_reply)
 plugin-pay:     plugins/libplugin.c:1036 (rpc_read_response_one)
 plugin-pay:     plugins/libplugin.c:1060 (rpc_conn_read_response)
 plugin-pay:     ccan/ccan/io/io.c:59 (next_plan)
 plugin-pay:     ccan/ccan/io/io.c:407 (do_plan)
 plugin-pay:     ccan/ccan/io/io.c:417 (io_ready)
 plugin-pay:     ccan/ccan/io/poll.c:453 (io_loop)
 plugin-pay:     plugins/libplugin.c:1893 (plugin_main)
 plugin-pay:     plugins/pay.c:1294 (main)
 plugin-pay:     ../sysdeps/nptl/libc_start_call_main.h:58 (__libc_start_call_main)
 plugin-pay:     ../csu/libc-start.c:381 (__libc_start_main_impl)
 plugin-pay:   parents:
 plugin-pay:     plugins/libplugin-pay.c:3308:struct direct_pay_data
 plugin-pay:     plugins/libplugin.c:1775:struct plugin
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 09:17:33 +10:30
Rusty Russell
f87c7ed439 plugins/sql: fix foreign keys.
I noticed that our subtables were not being cleaned, despite being "ON
DELETE CASCADE".  This is because foreign keys were not enabled, but
then I got foreign key errors: rowid cannot be a foreign key anyway!

So create a real "rowid" column.  We want "ON DELETE CASCADE" for
nodes and channels (and other tables in future) where we update
partially.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-02 14:56:04 -06:00
Rusty Russell
6176912683 plugins/pay: fix htlc_budget calc when we get temporary_channel_failure
Valgrind correctly reports it as uninitialized for this log message, and
the only way this can happen is channel_hints_update() when we receive a
temporary_channel_failure.  Put a dummy value here in this case.

```
Valgrind error file: valgrind-errors.23404
==23404== Conditional jump or move depends on uninitialised value(s)
==23404==    at 0x49E4B56: __vfprintf_internal (vfprintf-internal.c:1516)
==23404==    by 0x49F6519: __vsnprintf_internal (vsnprintf.c:114)
==23404==    by 0x1EBCEB: do_vfmt (str.c:66)
==23404==    by 0x1EBDF8: tal_vfmt_ (str.c:92)
==23404==    by 0x11A336: paymod_log (libplugin-pay.c:167)
==23404==    by 0x11B4B2: payment_chanhints_apply_route (libplugin-pay.c:534)
==23404==    by 0x11E999: payment_compute_onion_payloads (libplugin-pay.c:1707)
==23404==    by 0x11FF4C: payment_continue (libplugin-pay.c:2135)
==23404==    by 0x1245C0: adaptive_splitter_cb (libplugin-pay.c:3800)
==23404==    by 0x11FEF3: payment_continue (libplugin-pay.c:2123)
==23404==    by 0x1205FE: retry_step_cb (libplugin-pay.c:2301)
==23404==    by 0x11FEF3: payment_continue (libplugin-pay.c:2123)
==23404==
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-02 10:11:11 +10:30
Rusty Russell
0240c24936 plugins/sql: listsqlschemas command to retrieve schemas.
Good for detection of what fields are present.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
d8320f015f plugins/sql: add bkpr-listaccountevents and bkpr-listincome support.
This *would* be a 1-line change (add it to Makefile) except that we
previously assumed a "list" prefix on commands.

These use the default refreshing, but they could be done better using
the time-range parameters.

Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
9a591277f5 plugins/sql: allow some simple functions.
And document them!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
14aac0769c doc: document the sql command.
In particular, we generate the schema part from the plugin itself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `sql` plugin command to perform server-side complex queries.
2023-01-30 15:15:41 -06:00
Rusty Russell
adb8de3e07 plugins/sql: print out part of man page referring to schemas.
We now add tables to the strmap as we allocate them, since we don't
want to call "finish_td" when we're merely invoked for the
documentation, and don't need a database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
20654ebd49 plugins/sql: pay attention to deprecated in schema.
For now, we ignore every deprecated field, but put in the logic so
that future deprecations will work as expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
9b08c4f25a plugins/sql: refresh listnodes and listchannels by monitoring the gossip_store.
It's quite a lot of code, but these are the most expensive commands we
do so it's worth it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
aa3a1131aa plugins/sql: include the obvious indexes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
8a0ee5f56e plugins/sql: add listpeerchannels support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
51ae7118f1 plugins/sql: make tables for non-object arrays.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
c230291141 plugins/sql: rework to parse schemas.
This requires us to rename "index" fields, rename fields if we have a
sub-object, and create sub-tables if we have an array, and handle the
fact that some listX commands don't contain array X (listsendpays
contains "payments").

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
260643157d plugins/sql: create struct column to encode column details.
Rather than two arrays "columns" (for names) and "fieldtypes" (for
types), use a struct.  This makes additions easier for successive
patches.

Also pull process_json_obj() out of the loop in list_done().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
24d86a85c3 plugins/sql: initial commit of new plugin.
This is designed to allow you to perform complex server-side
queries.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
0274d88bad common/gossip_store: clean up header.
It's actually two separate u16 fields, so actually treat it as
such!

Cleans up zombie handling code a bit too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
9ab488fc41 plugins/topology: add direction field to listchannels.
It's a core concept in the spec which isn't directly exposed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listchannels` added a `direction` field (0 or 1) as per gossip specification.
2023-01-30 15:15:41 -06:00
Rusty Russell
2dec805465 decode: fix handling of blinded_payinfo.
Our pay code handles this correctly, but decode was still using an old model
where there was a payinfo per hop, not per path.

Reported-by: @t-bast
See: #5823
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Rusty Russell
6eb7a4cbf2 plugins: update to match latest offers text.
```
make check-source-bolt CHECK_BOLT_PREFIX="--prefix=BOLT-offers" BOLTVERSION=guilt/offers
```

In this case, only trivial mods.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 13:24:29 +10:30
Rusty Russell
e9eb5f493b common: update to latest route-blinding spec.
```
make check-source-bolt CHECK_BOLT_PREFIX="--prefix=BOLT-route-blinding" BOLTVERSION=guilt/offers
```

Other than textual changes, this does:

1. Ensures we put total_amount_msat in onion final hop (reported by @t-bast).
2. Require that they put total_amount_msat in onion final hop.
3. Return `invalid_onion_blinding` exactly as defined by the spec (i.e. less
   aggressive when we're the final hop) (also reported by @t-bast, but I already
   knew).

See: #5823
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: `offers` breaking blinded payments change (total_amount_sat required, Eclair compat)
2023-01-30 13:24:29 +10:30
Ken Sedgwick
a4dc714cdc hsmd: add hsmd_preapprove_keysend and check_preapprovekeysend pay modifier
Changelog-added: hsmd: A new message `hsmd_preapprove_keysend` is added.
Changelog-added: JSON-RPC: A new command `preapprovekeysend` is added.
2023-01-26 21:10:15 -06:00
Ken Sedgwick
f29343d740 hsmd: add hsmd_preapprove_invoice and check_preapproveinvoice pay modifier
Changelog-added: hsmd: A new message `hsmd_preapprove_invoice` is added.
Changelog-added: JSON-RPC: A new command `preapproveinvoice` is added.
2023-01-26 21:10:15 -06:00
Rusty Russell
d9fed06b90 common/bolt11: const cleanup, fix parsing errors.
Also, we don't need to pass the total length to the field parsers,
just the length for this field (confusingly, this was called
"data_length").

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-25 13:59:34 -06:00
Rusty Russell
6a95d3a25e common: expose node_id_hash functions.
They're used in several places, and we're about to add more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-21 08:05:31 -06:00
Christian Decker
71b581da4d make: Clean up duplicate cargo examples build rules 2023-01-16 13:53:48 +10:30
Vincenzo Palazzo
c488561282 plugins/topology: use listpeerchannels. 2023-01-13 10:42:42 +10:30
Rusty Russell
a56c890ae5 plugins: use listpeerchannels instead of listpeers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Rusty Russell
ff2d7e6833 pay: use json_to_listpeers_channels() for local_channel_hints.
Don't parse the listpeers.channels output ourselves: with two extra fields
we can simply reuse json_to_listpeers_channels().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Rusty Russell
5d5b9c6812 libplugin: don't return unopened channels from json_to_listpeers_channels().
This way we always have an SCID and a direction.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Rusty Russell
57dcf68c0b plugins/libplugin: flatten return from json_to_listpeers_result.
Instead of returning a peers -> channels heirarchy, return (as callers
want!) a flat array of channels.

This is actually most of the transition work to make them work with
listpeerchannels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Vincenzo Palazzo
cb5ee7e49c plugins: make bookkeeper use the new listpeerchannels command. 2023-01-13 10:42:42 +10:30
Rusty Russell
5dfcd15782 all: no longer need to call htable_clear to free htable contents.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell
0d93841cc7 plugins/command: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell
3c4ce9e448 plugins/pay: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell
3f0c5b985b commando: add filtering support.
1. When we receive a commando command from a remote using the `filter`
   field, use it.
2. Add a `filter` parameter to `commando` to send it: this is usually
   more efficient than using filtering locally.

Of course, older remote nodes will ignore the filter, but that's
harmless.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `commando` now supports `filter` as a parameter (for send and receive).
2023-01-11 11:13:27 +10:30
Rusty Russell
1250806060 commando: correctly replace the id field in responses.
This was reported a while ago: now do it properly.

Fixes: #5637
Changelog-Fixed: Plugins: `commando` now responds to remote JSON calls with the correct JSON `id` field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Rusty Russell
b75ada7017 commando: track incoming and outgoing JSON IDs. Get upset if they don't match!
They currently don't, so we get some BROKEN messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Rusty Russell
0201e6977f commando: build ID of command based on the id they give us.
We change the libplugin API so commando can provide its own ID base.

This id chaining enables much nicer diagnostics!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Rusty Russell
b3fa4b932e commando: send id inside JSON request.
All JSON-RPC calls should have one!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Rusty Russell
19db6a25e4 commando: require that we have an id field in JSON request.
We don't do this yet, so we add deprecated to those test (until next
patch!).

Changelog-Deprecated: plugins: `commando` JSON commands without an `id` (see doc/lightningd-rpc.7.md for how to construct a good id field).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Rusty Russell
2d8fff6b57 libplugin: don't turn non-string JSON ids into strings.
When called with `"id": 1` we replied with `"id": "1"`.  lightningd doesn't
actually care, but it's weird.

Copy the entire token: this way we don't have to special case anything.

Also, remove the doubled test in json_add_jsonstr.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Matt Whitlock
9d5eab1b69 topology: fix memleak in listchannels
local_connected allocates a struct node_map off of the struct
listchannels_opts but fails to ever release the internal table, so those
table allocations hang around forever. Add node_map_clear as a
destructor for the struct node_map to ensure that the internal table is
freed when the enclosing struct is freed.

Changelog-Fixed: topology: Fixed memleak in `listchannels`
Signed-off-by: Matt Whitlock <c-lightning@mattwhitlock.name>
2023-01-03 15:09:04 +10:30
Christian Decker
5a4c8402a7 rs: Add cln-plugin metadata 2023-01-02 14:33:02 +01:00
Christian Decker
77ad5525f5 cln-plugin: Re-export anyhow::anyhow macro
For now this makes reporting an error from plugins much simpler for
now.

Changelog-None
2022-12-24 11:32:22 +01:00
Vincenzo Palazzo
6c1e589ee8 cln-plugin: make available the configuration in plugin
There are several cases where you want to access to the configuration,
and given the nature of the rust API, there is no way to access to
the `configuration` field at any point after the configuration logic.

Suggested-by: Sergi Delgado Segura <@sr-gi>

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-12-24 11:32:22 +01:00
Christian Decker
465d9121bf rs: Bump crate versions for publication 2022-12-22 15:42:12 +01:00