Commit Graph

115 Commits

Author SHA1 Message Date
Rusty Russell
2da5244e83 jsonrpc: make error codes an enum.
This allows GDB to print values, but also allows us to use them in
'case' statements.  This wasn't allowed before because they're not
constant terms.

This also made it clear there's a clash between two error codes,
so move one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: Error code from bcli plugin changed from 400 to 500.
2022-09-19 10:18:55 +09:30
Rusty Russell
42c9aa1a5f libplugin: forget pending requests if associated command freed.
This is usually fine, but without this, commando (another branch!) has
a race:

1. A command has multiple parts.
2. We start sending them out.
3. We get a response, which completes the cmd.
4. We go to send the next one out, but it's been freed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
bf54d6dcf5 libplugin: use proper JSON id for rpc_scan().
And fold start_json_request() and start_json_rpc() into the core
function since it's the only caller.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
d360075d22 libplugin: use string ids correctly.
Build them from the command which caused them, and take plugin name
as basename with extension stripped.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
db89a34135 libplugin: allow lightningd to give us non-numeric ids.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
b3b5b74069 libplugin: allow NULL calllbacks for jsonrpc_set_datastore.
You often don't care about the reply, so this is quite convenient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 14:00:41 +02:00
Rusty Russell
6fe570820e Remove general shadowed variables.
We shadow local variables in several places: generally, these changes
make the code clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-31 12:18:28 +03:00
Rusty Russell
721ceb7519 patch db-fatal-plugin_err.patch 2022-07-28 12:08:18 +09:30
niftynei
d3ba017672 valgrind: rm ref to cmd when cmd is free'd
We were cmd was getting free'd but holding on to reference of the
thing was causing problems.

==523280== Invalid read of size 8
==523280==    at 0x1B3E14: del_notifier_property (tal.c:326)
==523280==    by 0x1B3E14: tal_del_notifier_ (tal.c:569)
==523280==    by 0x1123E7: handle_rpc_reply (libplugin.c:671)
==523280==    by 0x1123E7: rpc_read_response_one (libplugin.c:866)
==523280==    by 0x1123E7: rpc_conn_read_response (libplugin.c:886)
==523280==    by 0x1A7B53: next_plan (io.c:59)
==523280==    by 0x1A7B53: do_plan (io.c:407)
==523280==    by 0x1A7B53: io_ready (io.c:417)
==523280==    by 0x1A9BDB: io_loop (poll.c:453)
==523280==    by 0x1141D0: plugin_main (libplugin.c:1708)
==523280==    by 0x10D7E4: main (commando.c:937)
==523280==  Address 0x52de928 is 8 bytes inside a block of size 40 free'd
==523280==    at 0x483F0C3: free (vg_replace_malloc.c:872)
==523280==    by 0x1B2CDD: del_tree (tal.c:419)
==523280==    by 0x1B37BB: tal_free (tal.c:486)
==523280==    by 0x1B37BB: tal_free (tal.c:474)
==523280==    by 0x110CB2: command_complete (libplugin.c:255)
==523280==    by 0x110CB2: command_done_err (libplugin.c:390)
==523280==    by 0x10F511: handle_reply (commando.c:560)
==523280==    by 0x10F511: handle_custommsg (commando.c:609)
==523280==    by 0x113877: ld_command_handle (libplugin.c:1441)
==523280==    by 0x113877: ld_read_json_one (libplugin.c:1491)
==523280==    by 0x113877: ld_read_json (libplugin.c:1511)
==523280==    by 0x1A7B53: next_plan (io.c:59)
==523280==    by 0x1A7B53: do_plan (io.c:407)
==523280==    by 0x1A7B53: io_ready (io.c:417)
==523280==    by 0x1A9BDB: io_loop (poll.c:453)
==523280==    by 0x1141D0: plugin_main (libplugin.c:1708)
==523280==    by 0x10D7E4: main (commando.c:937)
==523280==  Block was alloc'd at
==523280==    at 0x483C855: malloc (vg_replace_malloc.c:381)
==523280==    by 0x1B3BBD: allocate (tal.c:250)
==523280==    by 0x1B3BBD: add_notifier_property (tal.c:303)
==523280==    by 0x1B3BBD: tal_add_destructor2_ (tal.c:529)
==523280==    by 0x110725: jsonrpc_request_start_ (libplugin.c:181)
==523280==    by 0x10E0EA: send_more_cmd (commando.c:643)
==523280==    by 0x11243C: handle_rpc_reply (libplugin.c:696)
==523280==    by 0x11243C: rpc_read_response_one (libplugin.c:866)
==523280==    by 0x11243C: rpc_conn_read_response (libplugin.c:886)
==523280==    by 0x1A7B53: next_plan (io.c:59)
==523280==    by 0x1A7B53: do_plan (io.c:407)
==523280==    by 0x1A7B53: io_ready (io.c:417)
==523280==    by 0x1A9BDB: io_loop (poll.c:453)
==523280==    by 0x1141D0: plugin_main (libplugin.c:1708)
==523280==    by 0x10D7E4: main (commando.c:937)
==523280==
{
   <insert_a_suppression_name_here>
2022-07-26 15:11:30 -07:00
Rusty Russell
43e5ef3cc4 libplugin: don't call callbacks if cmd completed before response.
This can particularly happen with commando:

```
 commando: FATAL SIGNAL 11 (version 06b36d3)
0x55609e953d51 send_backtrace
	common/daemon.c:33
0x55609e953dfb crashdump
	common/daemon.c:46
0x7f665e3b908f ???
	/build/glibc-SzIz7B/glibc-2.31/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0
0x55609e9387a3 send_more_cmd
	plugins/commando.c:632
0x55609e93b270 handle_rpc_reply
	plugins/libplugin.c:669
0x55609e93bd50 rpc_read_response_one
	plugins/libplugin.c:842
0x55609e93be86 rpc_conn_read_response
	plugins/libplugin.c:862
0x55609e9f4f68 next_plan
	ccan/ccan/io/io.c:59
0x55609e9f5b70 do_plan
	ccan/ccan/io/io.c:407
0x55609e9f5bb2 io_ready
	ccan/ccan/io/io.c:417
0x55609e9f7ea5 io_loop
	ccan/ccan/io/poll.c:453
0x55609e93eb20 plugin_main
	plugins/libplugin.c:1676
0x55609e9397ab main
	plugins/commando.c:922
0x7f665e39a082 __libc_start_main
	../csu/libc-start.c:308
0x55609e93677d ???
	???:0
0xffffffffffffffff ???
	???:0
```

Reported-by: @adi2011
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-21 15:37:05 -05:00
Rusty Russell
3eccf16f98 libplugin: datastore helpers.
Plugins are supposed to store their data in the datastore, and commando does so:
let's make it easier for them by providing convenience APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-17 08:51:02 +09:30
Rusty Russell
d3e64c3970 libplugin: jsonrpc_request_whole_object_start() for more custom request handling.
commando wants to see the whole reply object, and also not to assume params is
an object.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-17 08:51:02 +09:30
Rusty Russell
dbae5ae569 common/json_stream.c: provide explicit json_add_primitive_fmt and json_add_str_fmt routines.
Rather than a generic "add member", provide two routines: one which
doesn't quote, and one which does.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Rusty Russell
0236d4e4da common/json_stream: remove useless attempt at oom handling.
We tell membuf to use tal, and tal never returns NULL, so this
code can never be triggered.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Christian Decker
0ce68b26c6 jsonrpc: Include the direction also if we have an alias
The direction only depends on the ordering between node_ids, not the
short_channel_id, so we can include it and it won't change. This was
causing some trouble loading the `channel_hints` in the `pay` plugin.
2022-07-04 22:14:06 +02:00
Christian Decker
a4e6b58fa4 ld: Consider local aliases when forwarding 2022-07-04 22:14:06 +02:00
Christian Decker
1ae3dba529 invoice: Consider aliases too when selecting routehints 2022-07-04 22:14:06 +02:00
Rusty Russell
993f44f289 libplugin: don't be so strict on msat fields.
Let the parsing handle if they're the wrong type (soon they'll be u64).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell
43a833e405 lightningd: remove support for legacy onion format.
As per proposal in https://github.com/lightning/bolts/pull/962

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: protocol: support for legacy onion format removed, since everyone supports the new one.
2022-03-18 09:20:11 +10:30
Rusty Russell
d5064ff56c libplugin: make memleak see current requests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
b3338a174e plugins/keysend: prevent leak detecting getting upset.
1. We don't keep a pointer to payments (unlike pay, which keeps a
   linked list), so mark it notleak.
2. plugin->our_features is overloaded for "features we want to set" (used by keysend)
   and then "features we have".  Create a new field, which is cleaner.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
4ffda340d3 check: make sure all files outside contrib/ include "config.h" first.
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).

config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
ZmnSCPxj jxPCSnmZ
d4690358d9 plugins/libplugin.c: Allow freeing notification struct command *.
We always allocate a new `struct command` when we get a full JSON
object from stdin:

b2df01dc73/plugins/libplugin.c (L1229-L1233)

If it happens to be a notification, we pass the `struct command` to
the handler, and not free it ourselves:

b2df01dc73/plugins/libplugin.c (L1270-L1275)

There are only nine points in `plugins/libplugin.c` where we `tal_free`
anything, and only one of them frees a `struct command`:

b2df01dc73/plugins/libplugin.c (L224-L234)

The above function `command_complete` is not appropriate for
notification handlers; the above function sends out a response
to our stdout, which a notification handler should not do.

However, as-is, it does mean that notification handling leaks
`struct command` objects, which can be problematic if we ever
have future built-in plugins which are significantly more
dependent on notifications.

This commit changes notification handlers to return
`struct command_result *`, because possibly in the future
notification handlers may want to perform `send_outreq`, so we
might as well use our standard convention for callbacks, and
to encourage future developers to check how to properly
terminate notification handlers (and free up the
`struct command`).

We also now provide a `notification_handled` function which a
notification handler must eventually call, as well as a
`notification_handler_pending` which is just a snowclone of
`command_still_pending`.
2021-10-08 14:40:04 +10:30
Rusty Russell
7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
ea30c34d82 cleanup: remove unneeded includes in header files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
31f439760f libplugin: make leaks log at LOG_BROKEN so they break CI.
Now we've fixed them, this makes sure CI notices if new leaks appear.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell
14002915a1 plugins/libplugin: mark timers as not-a-leak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell
4f4c49c88a libplugin: fix leak of struct command when we don't read it all at once.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: C plugins would could leak memory on every command (esp. seen when hammering topology's listchannels).
2021-09-08 19:11:47 +02:00
Rusty Russell
b82e5be6e1 libplugin: let plugins add annotations to memleak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
mar
2021-09-08 19:11:47 +02:00
Rusty Russell
dd65a9150a libplugin: if DEVELOPER set, do memleak detection on shutdown.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell
bab7778b5b libplugin: allow take() for commands, notif_subs, hook_subs, notif_topics.
Useful for plugins which dynamically generate them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell
952471e7a3 libplugin: allow stealing of feature sets, make keysend do that.
Otherwise the NULL parents look like a memleak.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell
2063049559 libplugin: plugin_exit helper which flushes stdout.
We weren't actually getting the last log out; this does that.

We have to fix test_bitcoin_failure which now notices the BROKEN
log message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: libplugin: Fatal error messages from plugin_exit() now logged in lightningd.
2021-09-05 15:16:56 +02:00
niftynei
c934fe095b libplugin: add u16_option parsing
A couple of the fields for liquidity_ads are u16
2021-07-20 13:28:38 -04:00
Rusty Russell
38fafc7d6d libplugin: create debug message for notifications
Makes it easier to post-mortem in the logs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-03 12:13:45 +09:30
Rusty Russell
a4c7ff0f10 libplugin: remove command_success_str function.
As mentioned in previous commits: "result" must be an object,
and anything else is an antipattern.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-27 20:28:49 +09:30
Rusty Russell
2bb365a931 common/route: route_from_dijkstra returns route_hop array.
This is what (most) callers actually want, so unify it into one place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
e531a38963 gossipd / plugin: clean up names in struct route_hop.
We're going to unify them, but the names are not the normal ones.

Fix that first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
52e5ca3c11 plugins/libplugin: make headers update-mocks friendly.
In particular, it expects return type at start of line.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Christian Decker
c8c0c4dc99 libplugin: Add functions to start and end notifications 2021-05-03 11:20:15 +09:30
Christian Decker
c1f08a42fe libplugin: Add notification topics to the manifest response 2021-05-03 11:20:15 +09:30
Christian Decker
f963a6a551 libplugin: Add notification topics to plugin_main 2021-05-03 11:20:15 +09:30
Christian Decker
f08ae49134 plugin: Restrict plugin notifications only to announced topics
We want to have well-behaved notifications that are clearly announced
during the initialization, kill plugins that don't behave.
2021-05-03 11:20:15 +09:30
niftynei
15e83925e8 libplugin: add no-op command complete function
When an RPC originates from a plugin, and there's no associated command,
it's useful to be able to signal that you're finished.
2021-05-03 11:06:10 +09:30
Jan Sarenik
1b02d15695 typo: information is an uncountable mass noun
See https://en.wikipedia.org/wiki/Information

In libplugin.c also the word "details" was added (without removing
the 'information').

Changelog-None
2021-03-16 10:45:40 +10:30
Rusty Russell
27c006f7aa libplugin: make init return a string.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: libplugin: init can return a non-NULL string to disable the plugin.
2021-01-13 14:45:36 +01:00
Rusty Russell
3b7d0e7a62 common/json: make json_scan return an error string.
This makes for more useful errors.  It prints where it was up to in
the guide, but doesn't print the entire JSON it's scanning.

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell
09b18bf64f libplugin: replace rpc_delve with rpc_scan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell
b61da8c5a9 plugins: use json_scan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell
9361b62e3a libplugin: add command_hook_success helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-02 10:38:04 +10:30