Commit Graph

234 Commits

Author SHA1 Message Date
Simon Vrouwe
28816c387c testing: remove test_htlc_accepted_hook_shutdown, move relevant parts
into test_plugin_shutdown

The fact that plugins are kept alive untill the very end also ensures
their hooks are not silently unregistered.
2021-12-14 09:33:10 +10:30
Michael Schmoock
bad09887e0 pytest: fix test_pluginconnected_hook_chaining
The last line of the testcase was checking on the wrong node l3
instead of l1. l3 didn't had the plugins configured that would
produce the log entry we were looking for not to be present.

Changelog-None
2021-12-06 14:53:37 +01:00
Andrew Toth
69bc1191cb tests: add pay test for exclude arg 2021-12-04 21:37:42 +10:30
Simon Vrouwe
89cbdf4dce testing: test that RPC calls made in shutdown fail and receive error code -5
and the two conditions in which plugins can receive shutdown notification
2021-11-30 13:34:44 +10:30
Simon Vrouwe
5fb3674233 testing: test hook semantics is preserved in shutdown
plugins expect their hooks to work also in shutdown, see issue #4883
2021-11-30 13:34:44 +10:30
Rusty Russell
7a6fd70078 pytest: Reduce memory consumption by test_plugin_disable
It runs 6 nodes: under valgrind this ends up consuming 5.3 GB RSS.  By
stopping nodes between, we peak about 1G RSS.

Measured using:
	(while true; do echo $(for i in 4 5 6; do ps uh | tr -s ' ' | cut -d\  -f$i | tally; done); sleep 5; done)&

(Which measures my other processes as well, but that's only about 100M).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-27 12:41:37 +00:00
Rusty Russell
65bb989cf1 pytest: don't checksum plugins on startup in VALGRIND developer mode.
This loads up 20MB of plugins temporarily; we seem to be getting OOM
killed under CI and I wonder if this is contributing.

Doesn't significantly reduce runtime here, but I have lots of memory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-14 18:49:46 +01:00
Rusty Russell
516c38a750 lightningd: call shutdown plugin when we dynamic shutdown a single one.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-05 15:16:56 +02:00
Michael Schmoock
f6709a7a90 pytest: test new commitment_revocation hook values 2021-09-04 11:54:33 +09:30
Rusty Russell
f5e08c3dae pyln-client: document and test that init can self-disable.
mypy says it returns None, but it actually doesn't have to!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-14 12:52:19 +09:30
Michael Schmoock
896da4fc65 pytest: rescan restarts plugin on update 2021-07-15 13:26:05 -04:00
Rusty Russell
4e881e56ce pytest: always provide payment_secret when making payments.
They're about to become compulsory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-14 14:38:00 -05:00
niftynei
e4e1396447 tests: update coin movements for dual-funded mutual closes
This bug originated in 8f33f46960,
which updates the closing fee calculations.
2021-07-11 10:41:08 +09:30
Rusty Russell
8f33f46960 closingd: use a more accurate fee for closing fee negotiation.
We were actually using the last commit tx's size, since we were
setting it in lightningd.  Instead, hand the min and desired feerates
to closingd, and (as it knows the weight of the closing tx), and have
it start negotiation from there.

This can be significantly less when anchor outputs are enabled: for
example in test_closing.py, the commit tx weight is 1124 Sipa, the
close is 672 Sipa!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: Use a more accurate fee for mutual close negotiation.
2021-06-30 14:12:24 +09:30
Rusty Russell
1a181a3240 doc/schemas: listfunds, listinvoices, listnodes, listoffers
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-25 09:49:33 +09:30
Rusty Russell
d8136d856b listpeers: clean up JSON output.
In general, it's better to omit a field than put in a 'null', and
putting variable-named fields in an object is also a bad idea.

This is reflected in how hard this is to express in JSON schema, too.

Others:
1. Remove the obsolete "funding": "LOCAL" from unopened channels, but add
   "opener": "local" as used in normal channels.
2. htlc cltv_expiry is a u16.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `listfunds` `channels` `funding_allocation_msat` and `funding_msat`: use `funding`.
Changelog-Deprecated: JSON-RPC: `listfunds` `channels` `last_tx_fee`: use `last_tx_fee_msat`.
Changelog-Deprecated: JSON-RPC: `listfunds` `channels` `closer` is now omitted if it does not apply, not JSON `null`.
2021-06-25 09:49:33 +09:30
Christian Decker
610d8a0a74 plugin: Add a status field to htlcs in listpeers
Annotating the htlc in `listpeers` with their current status, and
which plugin is currently holding on to them with their
`htlc_accepted` hook can help us debug where plugins may go wrong.

Changelog-Added: jsonrpc: HTLCs in `listpeers` are now annotated with a status if they are waiting on an `htlc_accepted` hook of a plugin.
2021-06-05 17:47:32 +09:30
Rusty Russell
129d3f65e7 pytest: don't use command_success_str in test_libplugin.c
result should *always* be an object.  This allows it to add fields
without breaking the API.  A command which returns "result" as a
string is living in sin.

This changes one of the two callers of "command_success_str".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-27 20:28:49 +09:30
niftynei
e375932108 df-tests: test_coin_movement_notices, make work?
No idea how this slipped past the first time
2021-05-12 11:25:41 +09:30
niftynei
6d3fb11bc6 df-tests: patch for state == AWAITING_UNILATERAL problem
Found on CI where DEVELOPER=0 EXPERIMENTAL_DUAL_FUND=1,
as we turn off automatic reconnects when DEVELOPER=1

This test has been modified to make the error happen every time, and
then fixed.

lightningd-2: 2021-05-07T20:12:03.790Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Peer has reconnected, state AWAITING_UNILATERAL
lightningd-2: 2021-05-07T20:12:03.812Z **BROKEN** lightningd: FATAL SIGNAL 6 (version e8b3f78)
lightningd-2: 2021-05-07T20:12:03.812Z **BROKEN** lightningd: backtrace: common/daemon.c:44 (send_backtrace) 0x56384ee072e9
lightningd-2: 2021-05-07T20:12:03.813Z **BROKEN** lightningd: backtrace: common/daemon.c:52 (crashdump) 0x56384ee0733b

----------------------------- Captured stderr call -----------------------------
lightningd: lightningd/peer_control.c:1100: peer_connected_hook_final: Assertion `channel->state == DUALOPEND_OPEN_INIT || channel->state == DUALOPEND_AWAITING_LOCKIN' failed.
lightningd: FATAL SIGNAL 6 (version e8b3f78)
0x56384ee072a1 send_backtrace
	common/daemon.c:39
0x56384ee0733b crashdump
	common/daemon.c:52
0x7f88486a020f ???
	???:0
0x7f88486a018b ???
	???:0
0x7f884867f858 ???
	???:0
0x7f884867f728 ???
	???:0
0x7f8848690f35 ???
	???:0
0x56384eddc94e peer_connected_hook_final
	lightningd/peer_control.c:1100
0x56384edea2ed plugin_hook_call_
	lightningd/plugin_hook.c:275
0x56384eddfeb8 plugin_hook_call_peer_connected
	lightningd/peer_control.c:1156
0x56384eddfeb8 peer_connected
	lightningd/peer_control.c:1209
0x56384edc30cd connectd_msg
	lightningd/connect_control.c:332
0x56384edebe6f sd_msg_read
	lightningd/subd.c:509
0x56384edebfb1 read_fds
	lightningd/subd.c:310
0x56384ee483b0 next_plan
	ccan/ccan/io/io.c:59
0x56384ee4885b do_plan
	ccan/ccan/io/io.c:407
0x56384ee488f8 io_ready
	ccan/ccan/io/io.c:417
0x56384ee4a23c io_loop
	ccan/ccan/io/poll.c:445
0x56384edcabda io_loop_with_timers
	lightningd/io_loop_with_timers.c:24
0x56384edce826 main
	lightningd/lightningd.c:1111
0x7f88486810b2 ???
	???:0
0x56384edb52ad ???
	???:0
0xffffffffffffffff ???
	???:0
2021-05-12 11:25:41 +09:30
niftynei
558abe288a tests: explicitly mark tests to run both as v1+v2
By default, tests only run as v1 unless marked as v2.

These tests we want to run as both v1+v2

Includes fixes to have tests pass
2021-05-12 11:25:41 +09:30
niftynei
d0bbf07655 tests: not DEVELOPER -> mark.developer
Nicer syntaxtic sugar for marking pytests as 'developer required'
2021-05-12 11:25:41 +09:30
niftynei
b4e24ac8ba df: anchor outputs are on if EXP_DF
So we should treat it the same as EXPERIMENTAL_FEATURES
2021-05-12 11:25:41 +09:30
Christian Decker
c6fd849aa3 pay: Add notification for pay_success 2021-05-03 11:20:15 +09:30
Christian Decker
98aa3c3da7 plugin: Make unannounced notification topics no longer fatal
Since plugins will start sending them soon, and they are likely to get
it wrong sometimes, be a bit more lenient, warn them in the logs
instead and then make sure it doesn't accidentally work anyway.
2021-05-03 11:20:15 +09:30
Christian Decker
62e3358a5b plugin: Wrap custom notifications in a dict with additional origin
This should allow us to differentiate the origin of the notification,
and further prevent plugins from spoofing native notifications.
2021-05-03 11:20:15 +09:30
Christian Decker
083b41f090 plugin: Add a list of notification topics registered by plugin
We will eventually start emitting and dispatching custom notifications
from plugins just like we dispatch internal notifications. In order to
get reasonable error messages we need to make sure that the topics
plugins are asking for were correctly registered. When doing this we
don't really care about whether the plugin that registered the
notification is still alive or not (it might have died, but
subscribers should stay up and running), so we keep a list of all
topics attached to the `struct plugins` which gathers global plugin
information.
2021-05-03 11:20:15 +09:30
niftynei
7c76363e20 openchannel2: add channel_max_msat to openchannel2 hook payload
Changelog-Added: Plugins: add a `channel_max_msat` value to the `openchannel2` hook. Tells you the total max funding this channel is allowed to have.
2021-05-03 11:06:10 +09:30
niftynei
69cc9201de plugin-tests: cleanup outdated openchannel2 hook fields
We removed/changed the fields on the openchannel2 hook but never
updated this test (which doesn't run with the CI for #reasons)
2021-05-03 11:06:10 +09:30
Michael Schmoock
d753ee27a2 pytest: test rpc_command hook chain 2021-03-03 09:18:53 +10:30
Rusty Russell
171af7742a plugin: log self-disable at debug level, other manifest fails at unusual.
This avoids spamming the logs.  We also remove the duplicate debug
logs on self-disable (plugin_kill logs it for us).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-01 12:16:42 +01:00
Rusty Russell
ee5da52677 lightningd: differentiate cases of plugin death correctly.
If a plugin died due to connection close, we'd always say
"Plugin exited before completing handshake.", which was often
wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-01 12:16:42 +01:00
Rusty Russell
c6bd87ccab pytest: fix "bad gossip" flake in test_forward_event_notification
If not all nodes are up-to-date with the new blocks, they can reject
announcements:

```
lightningd-4: 2021-02-23T02:02:47.832Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-gossipd: Ignoring future channel_announcment for 124x1x0 (current block 123)
lightningd-4: 2021-02-23T02:02:47.848Z DEBUG   lightningd: Adding block 133: 2d950451211398de9c10bf9df7eb53b385390eca31e306bc8fc1387b53d9f9a2
lightningd-4: 2021-02-23T02:02:47.865Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-gossipd: Bad gossip order: WIRE_CHANNEL_UPDATE before announcement 124x1x0/0
lightningd-4: 2021-02-23T02:02:47.866Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-gossipd: Bad gossip order: WIRE_CHANNEL_UPDATE before announcement 124x1x0/1
```

Technically, this change is not sufficient either, since *gossipd* might
not know about new block yet.  But it makes this case less likely.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-25 14:44:03 +10:30
Christian Decker
5a066d8536 pytest: Stabilize test_forward_event_notification
LightningNode.join_nodes and LightningNode.openchannel internally
generate blocks, which causes nodes to be out of sync, and ignore
"future" announcements, because they haven't seen that block yet.
2021-02-11 09:48:55 +01:00
Rusty Russell
f3159ec4ac pytest: detect warnings, too.
Since we turned many errors into warnings, we want our tests to fail
when they happen unexpectedly.  We make WARNING clear in the strings
we print, too, to help out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-04 12:02:52 +10:30
Rusty Russell
6b11cc8b8c common: disallow NULL channel_id to peer_failed_err.
No more sending "all-channel" errors; in particular, gossipd now only
sends warnings (which make us hang up), not errors, and peer_connected
rejections are warnings (and disconnect), not errors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `peer_connected` rejections now send a warning, not an error, to the peer.
2021-02-04 12:02:52 +10:30
Michael Schmoock
bc40287ade pytest: peer_connected chainable tests 2021-02-01 09:57:15 +10:30
Michael Schmoock
bdf0d60fd6 chore: fix typo in openchannel hook log
Nit: The underscore in "openchannel_hook" is wrong, bcause the name of
the hook is just "openchannel". The "_hook" implied this to be part of
the name.

Changelog-None
2021-02-01 09:57:15 +10:30
Michael Schmoock
3a0b1c5b1d pytest: improve test_openchannel_hook_chaining
The current test was not checking for the output of the first plugin in
the chain.
2021-01-29 13:37:42 +10:30
Christian Decker
2834aaced0 gci: Stabilize test_forward_event_notification 2021-01-29 10:29:09 +10:30
Christian Decker
7962db821c pytest: Stabilize test_channel_state_changed_bilateral 2021-01-29 10:29:09 +10:30
Rusty Russell
001b5d6416 offers: make it a runtime option.
The fetchinvoice and offers plugins disable themselves if the option
isn't enabled (it's enabled by default on EXPERIMENTAL_FEATURES).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-offers` enables fetch, payment and creation of (early draft) offers.
2021-01-13 14:45:36 +01:00
Rusty Russell
27c006f7aa libplugin: make init return a string.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: libplugin: init can return a non-NULL string to disable the plugin.
2021-01-13 14:45:36 +01:00
Rusty Russell
529ae0d766 plugins: allow plugins to disable themselves at startup.
By returning 'disable: <reason>' inside getmanifest or init result.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: plugins: plugins can now disable themselves by returning `disable`, even if marked important.
2021-01-13 14:45:36 +01:00
niftynei
609d509ca5 df-tests: test_channel_state_changed_unilateral 2021-01-10 13:44:04 +01:00
niftynei
ac88c72f1e df-tests: test_channel_state_changed_bilateral 2021-01-10 13:44:04 +01:00
niftynei
450a094d26 df-tests: test_coin_movement_notices 2021-01-10 13:44:04 +01:00
niftynei
923e1f5c80 df-tests: test_feature_set 2021-01-10 13:44:04 +01:00
niftynei
db65771f38 df-tests: test_plugin_feature_announce 2021-01-10 13:44:04 +01:00
niftynei
21b9bec93b df-tests: test_channel_state_change_history 2021-01-10 13:44:04 +01:00