Commit graph

182 commits

Author SHA1 Message Date
Rusty Russell
757e6f8a9b libplugin: allow display of default values.
This means we can see the values in listconfigs, even if we haven't set
them yet.

In particular, we now see the following:

* autoclean-cycle.value_int=3600
* bitcoin-rpcclienttimeout.value_int=60
* bitcoin-retry-timeout.value_int=60
* funder-max-their-funding.value_str=4294967295sat
* funder-per-channel-min.value_str=10000sat
* funder-reserve-tank.value_str=0sat
* funder-fund-probability.value_int=100

Changelog-Changed: plugins: libplugin now shows plugin option default values (where they're non-trivial)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-15 15:50:54 -05:00
Rusty Russell
6af32885fa libplugin: add test for dynamic setting values.
Serves as an example, but also shows a quirk which we fix in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-15 15:50:54 -05:00
Lagrang3
7b18056b09 renepay: fix sources 2024-05-08 22:32:13 -05:00
Lagrang3
16d00e09ef renepay: bug fix, local channel information
Listpeerchannels would update the local channel information setting the
liquidity in the outgoing channel to known_min=known_max=capacity,
when in fact it should be known_min=known_max=spendable.
2024-05-08 22:32:13 -05:00
Rusty Russell
03db143216 pytest: test the check command in libplugin.
For extra points, we use an async command which calls out to listdatastore
before returning.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-06 20:51:19 -05:00
Alex Myers
56d5d45448 pytest: test pay during channel open
Notably this parses the listpeerchannels output while the state is
still uncommitted, i.e., state = OPENINGD.
2024-04-30 14:52:22 -05:00
Rusty Russell
6a2d949250 lightningd: remove failure_code from invoice hook and htlc_accepted hook.
These were deprecated in v22.08 (invoice hook) and v0.8 (htlc_accepted hook), and
marked EOL in v23.02.

*PLEASE* complain if this breaks things for you: it's kind of a test canary of
the deprecation system!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Plugins: `invoice_payment` and `htlc_accepted` hook `failure_code` response (derepcated v22.08 and v0.8, EOL v23.02)
2024-03-25 15:02:35 +10:30
Rusty Russell
277ed2ccbf libplugin: support version strings for deprecations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Erik De Smedt
375985bf6c Implement tests for notifications on custommsg 2023-12-16 11:36:42 +10:30
Erik De Smedt
557b5fee46 Allow dynamic option in python plugin 2023-12-14 16:48:02 +01:00
Rusty Russell
eae0d1da1e lightningd: fix up deprecated rest-port, rest-protocol, rest-host and rest-certs option if we would otherwise fail.
Since these worked in v23.08, we can't just rename them.  So if they are
used and unclaimed, we should rename them internally (if they're claimed,
it's probably clightning-rest, and we should *NOT* touch them!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Plugins: `clnrest` parameters `rest-port`, `rest-protocol`, `rest-host` and `rest-certs`: prefix `cln` to them
2023-11-17 10:51:41 +01:00
Rusty Russell
dfe207d503 pytest: test for accepting even unknown messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 11:50:57 +10:30
Christian Decker
4121941d92 pytest: Fix elementsd tests 2023-09-26 19:21:05 +02:00
Rusty Russell
de81a59b1e lightningd: mark explicitly what commands are dev-only.
And require --developer to use them.

Also refuse redirection to deprecated APIs if deprecated APIs are disabled! 

Changelog-Removed: `dev-sendcustommsg` (use `sendcustommsg`, which was added in v0.10.1)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell
67a391f8d0 lightningd: pass --developer down to plugins, subdaemons when it set.
They might need to know this before they do anything, so cmdline seems most
natural.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell
bc36a4580c pytest: test (failing) for not crashing if plugin fails to exec via plugin startdir
It hits the new assert(): before that (tested!) it actually kills everything and logs you out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 13:18:49 +09:30
Rusty Russell
be2cbe784f pytest: test for filtering bug.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-06 14:37:49 +09:30
Rusty Russell
8de3b89aac pytest: test for bad estimatefees results.
See: #6183
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-16 12:57:27 +09:30
Rusty Russell
4d507065f3 libplugin: support wildcard subscriptions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-13 18:59:37 +09:30
Rusty Russell
279b3aa7e8 pyln: add ability to subscribe to all notifications.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-13 18:59:37 +09:30
Rusty Russell
99b885c84c plugins: update for new deprecation API where all fields are nicely wrapped.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-10 14:51:11 +09:30
Shahana Farooqui
4e5e38f4b0 tests: notification response fixes
tests: notification response fixes
2023-07-10 14:51:11 +09:30
Rusty Russell
aa5c7e763f libplugin: make set callback for options take plugin ptr, check correct type.
I added a plugin arg and was surprised that compile didn't break.
This is because typesafe_cb et al are conditional casts: if the type
isn't as expected it has no effect, but we're passing plugin_option() through
varargs, so everything is accepted!

Add a noop inline to check type, and fix up the two cases where we
used `const char *` instead of `char *`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-20 20:08:25 +09:30
Rusty Russell
70aee52903 libplugin: don't spew datastore errors to LOG_DEBUG.
People get upset, especially as our "not found" error can be a bit
hard to read!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
See-also: #5990
2023-02-11 12:22:40 -06:00
Rusty Russell
ff1d537b87 pytest: neaten test_cln_sendpay_weirdness, rename.
1. Allow 'any' as an option to zeroconf-selective.py plugin so we can use
   it in line_graph where we don't know ids yet.
2. Use modern helpers like line_graph and remove debugging statement.
3. Don't use listchannels(): it's true that it shows local channels as well,
   but that's a quirk I'd like to remove.
4. Make flake8 happy.
5. Rename to be more specific now it's a more narrow test.

I manually tested that the test still failed with the fixes removed, too,
so it is still the same test!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 09:17:33 +10:30
Rusty Russell
d1b5943e90 tests: make test_libplugin use the datastore.
It's a bit convoluted, but it exercises all our datastore fetch APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-12-22 12:04:30 +01:00
Rusty Russell
6e86fa9220 lightningd: figure out optimal channel *before* forward_htlc hook.
Otherwise what the hook sees is actually a lie, and if it sets it
we might override it.

The side effect is that we add an explicit "forward_to" field, and
allow hooks to override it.  This lets a *hook* control channel
choice explicitly.

Changelod-Added: Plugins: `htlc_accepted_hook` return can specify what channel to forward htlc to.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-26 13:52:04 +02:00
Christian Decker
c5b2aee5c6 pytest: Add a zeroreserve test 2022-09-21 11:25:47 +02:00
Rusty Russell
3380f559f9 memleak: simplify API.
Mainly renaming.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 11:34:42 +09:30
fiatjaf
1ef8fb7ef8 rename block_processed to block_added 2022-09-14 13:50:38 -05:00
fiatjaf
9b33a921f0 Add plugin notification topic "block_processed".
Changelog-Added: Plugins: Added notification topic "block_processed".
2022-09-14 13:50:38 -05:00
Rusty Russell
3e672b784d Makefile: use a library archive for CCAN
The linker discards whole files in an archive if it doesn't need them,
so saves a bit of space (and time).  Also allows us to add more niche
things to CCAN (e.g. runes support!) without bloating all the binaries.

We also had many places which depended on $(CCAN_FILES), but that was
already a dependent of $(ALL_PROGRAMS) and $(ALL_TEST_PROGRAMS).

Before:

```
$ size lightningd/lightning*d
   text	   data	    bss	    dec	    hex	filename
2247683	   8696	  39008	2295387	 23065b	lightningd/lightning_channeld
2086607	   7432	  38880	2132919	 208bb7	lightningd/lightning_closingd
2227916	   8056	  39200	2275172	 22b764	lightningd/lightning_connectd
3369236	 119288	  39240	3527764	 35d454	lightningd/lightningd
2183551	   8352	  38880	2230783	 2209ff	lightningd/lightning_dualopend
2196389	   8024	  39136	2243549	 223bdd	lightningd/lightning_gossipd
2086216	   7488	  39264	2132968	 208be8	lightningd/lightning_hsmd
2134396	   8136	  39424	2181956	 214b44	lightningd/lightning_onchaind
2133391	   8352	  38880	2180623	 21460f	lightningd/lightning_openingd
1512168	   2136	  34384	1548688	 17a190	lightningd/lightning_websocketd
```

After:
```
text	   data	    bss	    dec	    hex	filename
2192065	   8488	  38912	2239465	 222be9	lightningd/lightning_channeld
2030957	   7224	  38816	2076997	 1fb145	lightningd/lightning_closingd
2179571	   7968	  39104	2226643	 21f9d3	lightningd/lightning_connectd
3354296	 119288	  39208	3512792	 3599d8	lightningd/lightningd
2127933	   8144	  38816	2174893	 212fad	lightningd/lightning_dualopend
2141699	   7856	  39072	2188627	 216553	lightningd/lightning_gossipd
2024482	   7288	   5240	2037010	 1f1512	lightningd/lightning_hsmd
2072074	   7920	   5400	2085394	 1fd212	lightningd/lightning_onchaind
2077773	   8144	  38816	2124733	 206bbd	lightningd/lightning_openingd
1408958	   1752	    344	1411054	 1587ee	lightningd/lightning_websocketd
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-17 08:51:02 +09:30
Rusty Russell
401f1debc5 common: clean up json routine locations.
We have them split over common/param.c, common/json.c,
common/json_helpers.c, common/json_tok.c and common/json_stream.c.

Change that to:
* common/json_parse (all the json_to_xxx routines)
* common/json_parse_simple (simplest the json parsing routines, for cli too)
* common/json_stream (all the json_add_xxx routines)
* common/json_param (all the param and param_xxx routines)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Simon Vrouwe
7115611249 pytest: test plugin does not register same option "name" 2022-07-10 21:09:41 -05:00
Christian Decker
8609f9e00d pytest: Test the mindepth customizations of fundchannel and hook
Just test that we can customize, and we'll add mindepth=0 support in
the next couple of commits.
2022-07-04 22:14:06 +02:00
Rusty Russell
36a2491a89 json: fix up msat amounts in non-_msat fields.
We had json_add_amount_msat_only(), which was designed to be used to
print out msat fields, if we had sats.

However, we misused it, so split it into the three different cases:
1. json_add_amount_sat_msat: We are using it correctly, with a field called
   xxx_msat.
2. json_add_amount_sats_deprecated: We were using it wrong, so deprecate
   the old field and create a new one which does end in _msat.
3. json_add_sats: we were using it to hand sats as a JSON parameter to an
   interface, where "XXXsat".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Plugins: `rbf_channel` and `openchannel2` hooks `their_funding` (use `their_funding_msat`)
Changelog-Deprecated: Plugins: `openchannel2` hook `dust_limit_satoshis` (use `dust_limit_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `funding_satoshis` (use `funding_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `dust_limit_satoshis` (use `dust_limit_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `channel_reserve_satoshis` (use `channel_reserve_msat`)
Changelog-Deprecated: Plugins: `channel_opened` notification `amount` (use `funding_msat`)
Changelog-Deprecated: JSON-RPC: `listtransactions` `msat` (use `amount_msat`)
Changelog-Deprecated: Plugins: `htlc_accepted` `forward_amount` (use `forward_msat`)
2022-06-21 06:52:35 +09:30
Michael Schmoock
6db97b4235 pytest: check for remote_addr 2022-02-22 05:45:47 +10:30
niftynei
70a73928cb balance-snaps: add a balance_snapshot event; fires after first catchup
Fire off a snapshot of current account balances (node wallet + every
'active' channel) after we've caught up to the chain tip for the *first*
time (in other words, on start).
2021-12-28 04:42:42 +10:30
niftynei
f169597a02 test plugins/coin_moves: clean up/tidy coin moves plugin logic
removes unused in-memory stash of coin-moves; don't write the entire set
of coinmoves out on every update.
2021-12-28 04:42:42 +10:30
niftynei
d2c4d4aec2 coin_mvts: rewrite how onchain events are recorded, update tests
The old model of coin movements attempted to compute fees etc and log
amounts, not utxos. This is not as robust, as multi-party opens and dual
funded channels make it hard to account for fees etc correctly.

Instead, we move towards a 'utxo' view of the onchain events. Every
event is either the creation or 'destruction' of a utxo. For cases where
the value of the utxo is not (fully) debited/credited to our account, we
also record the output_value. E.g. channel closings spend a utxo who's
entire value we may not own.

Since we're now tracking UTXOs onchain, we can now do more complex
assertions about the onchain footprint of them. The integration tests
have been updated to now use more 'chain aware' assertions about the
ending state.
2021-12-28 04:42:42 +10:30
Simon Vrouwe
28816c387c testing: remove test_htlc_accepted_hook_shutdown, move relevant parts
into test_plugin_shutdown

The fact that plugins are kept alive untill the very end also ensures
their hooks are not silently unregistered.
2021-12-14 09:33:10 +10:30
Rusty Russell
4ffda340d3 check: make sure all files outside contrib/ include "config.h" first.
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).

config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Simon Vrouwe
89cbdf4dce testing: test that RPC calls made in shutdown fail and receive error code -5
and the two conditions in which plugins can receive shutdown notification
2021-11-30 13:34:44 +10:30
Simon Vrouwe
5fb3674233 testing: test hook semantics is preserved in shutdown
plugins expect their hooks to work also in shutdown, see issue #4883
2021-11-30 13:34:44 +10:30
Rusty Russell
68043c2e8c common: clean up autodata in common_shutdown().
valgrind locally complains about the allocations in autodata leaking:

```
==138200== 16 bytes in 1 blocks are still reachable in loss record 1 of 2
==138200==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==138200==    by 0x10D41A: autodata_register_ (autodata.c:20)
==138200==    by 0x10E7B8: register_autotype_type_to_string (type_to_string.h:79)
==138200==    by 0x10F5CA: register_one_type_to_string0 (block.c:259)
==138200==    by 0x19734C: __libc_csu_init (in /home/rusty/devel/cvs/lightning/common/test/run-route-specific)
==138200==    by 0x4A3D03F: (below main) (libc-start.c:264)
==138200== 
==138200== 176 bytes in 1 blocks are still reachable in loss record 2 of 2
==138200==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==138200==    by 0x10D472: autodata_register_ (autodata.c:26)
==138200==    by 0x122D37: register_autotype_type_to_string (type_to_string.h:79)
==138200==    by 0x122F1F: register_one_type_to_string0 (node_id.c:50)
==138200==    by 0x19734C: __libc_csu_init (in /home/rusty/devel/cvs/lightning/common/test/run-route-specific)
==138200==    by 0x4A3D03F: (below main) (libc-start.c:264)
==138200== 
make: *** [Makefile:638: unittest/common/test/run-route-specific] Error 7
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
ZmnSCPxj jxPCSnmZ
d4690358d9 plugins/libplugin.c: Allow freeing notification struct command *.
We always allocate a new `struct command` when we get a full JSON
object from stdin:

b2df01dc73/plugins/libplugin.c (L1229-L1233)

If it happens to be a notification, we pass the `struct command` to
the handler, and not free it ourselves:

b2df01dc73/plugins/libplugin.c (L1270-L1275)

There are only nine points in `plugins/libplugin.c` where we `tal_free`
anything, and only one of them frees a `struct command`:

b2df01dc73/plugins/libplugin.c (L224-L234)

The above function `command_complete` is not appropriate for
notification handlers; the above function sends out a response
to our stdout, which a notification handler should not do.

However, as-is, it does mean that notification handling leaks
`struct command` objects, which can be problematic if we ever
have future built-in plugins which are significantly more
dependent on notifications.

This commit changes notification handlers to return
`struct command_result *`, because possibly in the future
notification handlers may want to perform `send_outreq`, so we
might as well use our standard convention for callbacks, and
to encourage future developers to check how to properly
terminate notification handlers (and free up the
`struct command`).

We also now provide a `notification_handled` function which a
notification handler must eventually call, as well as a
`notification_handler_pending` which is just a snowclone of
`command_still_pending`.
2021-10-08 14:40:04 +10:30
Rusty Russell
37ddf2e829 lightningd: rename sendonionmessage to sendobsonionmessage.
sendonionmessage is going to be the new one, and do much *less*.

As this is an internal experimental-only API, no deprecation cycle
required.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell
7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
ea30c34d82 cleanup: remove unneeded includes in header files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
a0d531d515 pytest: annotate name_option against leak detection in test_libplugin
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00