Commit Graph

76 Commits

Author SHA1 Message Date
Rusty Russell
30c8db148d lightningd: make sure rpc_command replacement is well-formed.
In particular:
1. It must redirect to an existing command.
2. It must contain method, params and id.

And update the docs to show the id, which is vital.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-07 21:26:31 +01:00
Rusty Russell
e572b487d3 pytest: demonstrate failing rpc_command hook rewrite to plugin.
lightningd: lightningd/plugin.c:648: plugin_rpcmethod_dispatch: Assertion `idtok != NULL' failed.
lightningd: FATAL SIGNAL 6 (version v0.7.3-234-gf3aeb51-modded)
0x557652363eb1 send_backtrace
	common/daemon.c:41
0x557652363f5b crashdump
	common/daemon.c:54
0x7f5b065a346f ???
	???:0
0x7f5b065a33eb ???
	???:0
0x7f5b06582898 ???
	???:0
0x7f5b06582768 ???
	???:0
0x7f5b06594005 ???
	???:0
0x557652355901 plugin_rpcmethod_dispatch
	lightningd/plugin.c:648
0x557652331836 command_exec
	lightningd/jsonrpc.c:588
0x557652331bad rpc_command_hook_callback
	lightningd/jsonrpc.c:655
0x557652358745 plugin_hook_callback
	lightningd/plugin_hook.c:90
0x55765235486a plugin_response_handle
	lightningd/plugin.c:258
0x557652354a16 plugin_read_json_one
	lightningd/plugin.c:356
0x557652354b6d plugin_read_json
	lightningd/plugin.c:388
0x5576523b5e6c next_plan
	ccan/ccan/io/io.c:59
0x5576523b6a51 do_plan
	ccan/ccan/io/io.c:407
0x5576523b6a93 io_ready
	ccan/ccan/io/io.c:417
0x5576523b8d2b io_loop
	ccan/ccan/io/poll.c:445
0x55765232dc6f io_loop_with_timers
	lightningd/io_loop_with_timers.c:24
0x557652334a6e main
	lightningd/lightningd.c:848
0x7f5b065841e2 ???
	???:0
0x5576523193ed ???
	???:0
0xffffffffffffffff ???
	???:0
Log dumped in crash.log.20191206093336

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-07 21:26:31 +01:00
Rusty Russell
f6ed7f2e89 plugin: handle corner case where rpc_command is to stop the plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-06 16:52:16 +01:00
Rusty Russell
d8fc5332c3 pytest: test rpc_command hook when it handles command to stop itself.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-06 16:52:16 +01:00
Rusty Russell
aab83e729b lightningd: change config-dir from plugin / wallet / hsm POV into <network> subdir
Changelog-changed: .lightningd plugins and files moved into <network>/ subdir
Changelog-changed: WARNING: If you don't have a config file, you now may need to specify the network to lightning-cli
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Christian Decker
5a233a1b27 pytest: Fix TLV hook tests in experimental mode
I missed this test breaking with `EXPERIMENTAL_FEATURES` in #3261
2019-11-22 03:48:54 +00:00
Christian Decker
dda792c766 plugin: Pass the full raw_payload including realm to htlc_accepted
So far we've only handled legacy payloads, which meant we could drop the realm
byte since it was always 0x00. Once we start handling TLV payloads the first
byte, i.e., the former realm byte, is important since it gives us the length
of the payload. This is a breaking change, however I don't think there's
anyone using the `raw_payload` as of yet.

Changelog-Changed: JSON-RPC: the `raw_payload` now includes the first byte, i.e., the realm byte, of the payload as well. This allows correct decoding of a TLV payload in the plugins.
2019-11-18 12:00:15 +01:00
Rusty Russell
709c98f539 logging: remove spaces from subsystem names.
Spaces just make life a little harder for everyone.

(Plus, fix documentation: it's 'jsonrpc' not 'json' subsystem).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
ef7a820ab1 log: make formatting more consistent.
1. Printed form is always "[<nodeid>-]<prefix>: <string>"
2. "jcon fd %i" becomes "jsonrpc #%i".
3. "jsonrpc" log is only used once, and is removed.
4. "database" log prefix is use for db accesses.
5. "lightningd(%i)" becomes simply "lightningd" without the pid.
6. The "lightningd_" prefix is stripped from subd log prefixes, and pid removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Logging: formatting made uniform: [NODEID-]SUBSYSTEM: MESSAGE
Changelog-removed: `lightning_` prefixes removed from subdaemon names, including in listpeers `owner` field.
2019-11-18 04:50:22 +00:00
Rusty Russell
e433d4ddc1 lightningd: have logging include an optional node_id for each entry.
A log can have a default node_id, which can be overridden on a per-entry
basis.  This changes the format of logging, so some tests need rework.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
bb06bec891 features: set OPT_VAR_ONION (bit 9) iff EXPERIMENTAL_FEATURES
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Rusty Russell
2a2259083a lightningd: handle tlv-style payloads.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: JSON API: `htlc_accepted` hook has `type` (currently `legacy` or `tlv`) and other fields directly inside `onion`.
Changelog-deprecated: JSON API: `htlc_accepted` hook `per_hop_v0` object deprecated, as is `short_channel_id` for the final hop.
2019-11-14 10:15:33 +01:00
darosior
95c6513ff3 pytest: test the 'rpc_command' hook 2019-11-11 22:30:01 +01:00
darosior
7dac7a8c52 pytest: deactivate shadow routing for some tests that use 'pay'
So that we can assert payment values
2019-11-08 03:27:58 +00:00
darosior
f3d304901b pay: add a dev-only parameter to deactivate shadow routing
Had to make a special pylightning function to avoid rewriting all calls
to 'pay()' with 'rpc.call()' in the next commit..
2019-11-08 03:27:58 +00:00
darosior
c0ae3cb8c4 pytest: make test_plugin_slowinit a test again 2019-10-11 10:05:08 -05:00
darosior
32d98f0a87 plugins: refactor the tests for dynamic plugins
This adapts the test to the new 'plugin' command: no more sleeping,
since we are synchronous !

This tests the timeout by increasing the 'slowinit' plugin sleep
duration at init reception.

This adds a broken plugin to make sure we won't crash because of a
misbehaving plugin (unmet dependency is the most common case).
2019-09-30 00:20:16 +00:00
Christian Decker
efc4aa94a3 pytest: Skip some tests that assume we have a sqlite3 db on postgres
These will not work since they touch the DB file itself.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
trueptolemy
a3a0f651e3 pytest: Let fail_htlcs.py plugin generates TEMPORARY_NODE_FAILURE error 2019-09-16 12:22:06 +08:00
trueptolemy
619362ae19 pytest: Add a test for 'sendpay_success' and 'sendpay_failure' 2019-09-11 00:57:39 +00:00
Christian Decker
492d77f213 db: Add setup and teardown function to DB
These are used to do one-time initializations and wait for pending statements
before closing.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Rusty Russell
049529542a lightningd: delay reprocessing of incoming htlcs at startup until plugins ready.
Fixes: #2923
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-01 16:55:32 +02:00
Rusty Russell
f0ab4d72da pytest: add test that htlc_accepted hook only gets called after init.
We fail this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-01 16:55:32 +02:00
Rusty Russell
20a2bf9547 pytest: make test_forward_event_notification more explicit.
We seem to be getting intermittant failures, but it's hard
to disgnose.  Simplify it by moving all the test logic into
the test itself, and making the plugin dumber.  This means we'll
see exactly what the differences are if it fails again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Simon Vrouwe
5b5d70d640 lightningd/plugin: extend plugins->startup window to include plugins_config
Fixes incorrect configuration[`startup`] in plugin `init`, modified tests
to test this.
2019-08-27 00:02:20 +00:00
Rusty Russell
63103db3f3 pytest: disable some v. slow tests when !DEVELOPER.
We've gone over 50 minutes with valgrind, and my measurements show
these are the slowest.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-14 05:53:36 +00:00
Rusty Russell
118150227e cli: restore 0.7.0-style whitespace printing.
@renepickhardt has a shell script we broke.  While we still produce
perfectly valid JSON, we should not gratuitously change tool output.

Plus, I prefer the missing space before the ':'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-08 18:16:48 +08:00
Christian Decker
e2f9a410f4 pytest: Run plugin --help tests in the test directory
This is a followup to #2892. Since we now attempt to lock the PID file before
starting plugins we need to make sure that we actually use a unique lightning
directory for anything that attempts to call `--help`. If not we may be
conflicting with a `lightningd` that is running against that directory.

Notice that this still means that we will be unable to call `--help` on
`lightningd` if we have a running instance, but isolation in this case is
good, otherwise we'd be reading the default config anyway.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-06 13:10:32 +08:00
Rusty Russell
b460590278 plugins: detect and fixup old relative paths.
Note that we move adding the plugin to the plugins list to the end, otherwise
the hook from logging can examine the (uninitialized) plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell
4fcd94a091 pytest: use absolute paths for plugin arguments.
We're going to change this in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
trueptolemy
5ac02907bb pytest: Add the test for 'forward_event' in test_plugin 2019-08-01 18:49:25 +08:00
Rusty Russell
79d32ec2f2 plugin: notice when plugin has *started* configuring.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 08:44:01 +08:00
Rusty Russell
df8a6f615b pytest: test for crash when we do a plugin op and a plugin has not finished init
We try to start it twice, with predictable hilarious results:

   DEBUG plugin-manager started(20701) /home/rusty/devel/cvs/lightning/tests/plugins/slow_init.py
   DEBUG lightningd(20670): Adding block 101: 0f26301caaae390d1b956c3eb73827d1f87a2af306561348045bf1313b7551b0
   INFO plugin-slow_init.py slow_init.py initializing
   DEBUG plugin-manager started(20704) /home/rusty/devel/cvs/lightning/tests/plugins/slow_init.py
   **BROKEN** lightningd(20670): FATAL SIGNAL 6 (version v0.7.1-146-g7d3b352-modded)
   **BROKEN** lightningd(20670): backtrace: common/daemon.c:45 (send_backtrace) 0x55a29c3ec539
   **BROKEN** lightningd(20670): backtrace: common/daemon.c:53 (crashdump) 0x55a29c3ec589
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281ebcf5f
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281ebced7
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281e9e534
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281e9e40e
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281eae011
   **BROKEN** lightningd(20670): backtrace: lightningd/json_stream.c:225 (json_stream_output_) 0x55a29c3bff5f
   **BROKEN** lightningd(20670): backtrace: lightningd/plugin.c:383 (plugin_write_json) 0x55a29c3e103f
   **BROKEN** lightningd(20670): backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x55a29c43316b
   **BROKEN** lightningd(20670): backtrace: ccan/ccan/io/io.c:435 (io_do_always) 0x55a29c433df1
   **BROKEN** lightningd(20670): backtrace: ccan/ccan/io/poll.c:300 (handle_always) 0x55a29c435865
   **BROKEN** lightningd(20670): backtrace: ccan/ccan/io/poll.c:377 (io_loop) 0x55a29c435b3d
   **BROKEN** lightningd(20670): backtrace: lightningd/io_loop_with_timers.c:24 (io_loop_with_timers) 0x55a29c3bdfcf
   **BROKEN** lightningd(20670): backtrace: lightningd/lightningd.c:830 (main) 0x55a29c3c44b6
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281e9fb6a
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x55a29c3ac0f9
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0xffffffffffffffff

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 08:44:01 +08:00
darosior
307fb0708e lightningd/plugin_control: don't control non-dynamic plugins 2019-07-28 07:24:04 +00:00
darosior
9ddc2e0334 Add the 'plugin' subcommands to pylightning and add a test for them 2019-07-28 07:24:04 +00:00
darosior
2683170da8 Test and document 'channel_opened' notification 2019-07-27 12:18:25 +02:00
darosior
34533ead68 Plugins: Add a test for the 'invoice_payment' notfication 2019-07-25 11:19:47 +08:00
Christian Decker
53488e5739 pytest: Strengthen the htlc_accepted tests
We were having a few issues with malformed data in the past, so this time we
really check that stuff.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-13 14:44:12 +02:00
Rusty Russell
54ce4ed1cf pytest: fail tests if we get any LOG_BROKEN level messages, unless flagged.
And clean up some dev ones which actually happen (mainly by calling
channel_fail_permanent which logs UNUSUAL, rather than
channel_internal_error which logs BROKEN).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-02 03:26:10 +00:00
Rusty Russell
5f02294b5b lightningd/log: prefix log messages with level.
In particular, this lets us spot UNUSUAL and BROKEN messages easily.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-02 03:26:10 +00:00
Rusty Russell
380c044668 tests: add plugin for simple invoice delay.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:19:11 +00:00
trueptolemy
4d08ed2fa6 pytest: Add a test for the 'warning' subscription and notification
1. Create a plugin: ./lightning/tests/plugins/pretend_badlog.py
This plugin subscribes 'warning' notification and log the payload of
'warning';
2. Add a new test: tests/test_plugin.py::test_warning_notification
This test runs the plugin-pretend_badlog.py and check if 'warning'
notification can be normal triggered and subscribed.
2019-06-07 01:23:51 +00:00
Michael Schmoock
c40250ad0b test: adds wait mempool to flaky tests 2019-06-06 00:04:59 +00:00
Christian Decker
dd26a01c54 pytest: Add a test for htlc_accepted hook replay on startup 2019-06-04 00:27:15 +00:00
Christian Decker
9fd8be6463 pytest: Add tests for htlc_accepted_hook
Two tests: one for failures and one for in-path resolution.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-04 00:27:15 +00:00
Christian Decker
b54577041a pytest: Add a simple test for the hooks
This uses the `htlc_accepted` hook to delay payment acceptance.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-04 00:27:15 +00:00
Michael Schmoock
107381aca2 test: fix flakyness of funding_reorg_private openchannel_hook 2019-06-03 19:30:19 +02:00
Christian Decker
ced2ff8aa7 pytest: Mark repeat offenders as flaky
It's always the same few tests that fail, so I'm marking them as flaky and
will do a cleanup round later to individually address them.

I've been tracking the failures on the master branch for a few months now and
these are the worst offenders:

|-----------------------------------------|--------|--------|-------|
| test_name                               | state  | branch | count |
|-----------------------------------------|--------|--------|-------|
| test_pay_direct                         | FAILED | master | 31    |
| test_reconnect_gossiping                | FAILED | master | 20    |
| test_htlc_send_timeout                  | FAILED | master | 15    |
| test_pay_limits                         | FAILED | master | 13    |
| test_permfail                           | FAILED | master | 13    |
| test_funding_reorg_private              | FAILED | master | 12    |
| test_invoice                            | FAILED | master | 12    |
| test_invoice_preimage                   | FAILED | master | 12    |
| test_shutdown                           | FAILED | master | 12    |
| test_withdraw                           | FAILED | master | 12    |
| test_gossip_store_load_v3               | FAILED | master | 11    |
| test_onchain_multihtlc_their_unilateral | FAILED | master | 10    |
| test_opening_tiny_channel               | FAILED | master | 10    |
| test_channel_reenable                   | FAILED | master | 9     |
| test_crashlog                           | FAILED | master | 9     |
| test_gossip_weirdalias                  | FAILED | master | 9     |
| test_invoice_expiry                     | FAILED | master | 9     |
| test_onchain_multihtlc_our_unilateral   | FAILED | master | 9     |
| test_peerinfo                           | FAILED | master | 9     |
| test_private_channel                    | FAILED | master | 9     |
| test_shutdown_reconnect                 | FAILED | master | 9     |
| test_closing                            | FAILED | master | 8     |
| test_closing_different_fees             | FAILED | master | 8     |
| test_closing_while_disconnected         | FAILED | master | 8     |
| test_gossip_timestamp_filter            | FAILED | master | 8     |
| test_gossipwith                         | FAILED | master | 8     |
| test_invoice_routeboost                 | FAILED | master | 8     |
| test_onchain_middleman                  | FAILED | master | 8     |
| test_plugin_notifications               | FAILED | master | 8     |
| test_reconnect_channel_peers            | FAILED | master | 8     |
| test_bad_onion                          | FAILED | master | 7     |
| test_closing_torture                    | FAILED | master | 7     |
| test_fulfill_incoming_first             | FAILED | master | 7     |
| test_funding_reorg_disagree_scid_route  | FAILED | master | 7     |
| test_pay_disconnect                     | FAILED | master | 7     |
| test_balance                            | FAILED | master | 6     |
| test_check_command                      | FAILED | master | 6     |
| test_closing_id                         | FAILED | master | 6     |
| test_gossip_addresses                   | FAILED | master | 6     |
| test_gossip_notices_close               | FAILED | master | 6     |
| test_setchannelfee_usage                | FAILED | master | 6     |
| test_announce_address                   | FAILED | master | 5     |
| test_connect_by_gossip                  | FAILED | master | 5     |
| test_gossip_badsig                      | FAILED | master | 5     |
| test_gossip_disable_channels            | FAILED | master | 5     |
| test_gossip_jsonrpc                     | FAILED | master | 5     |
| test_gossip_persistence                 | FAILED | master | 5     |
| test_gossip_pruning                     | FAILED | master | 5     |
| test_onchain_different_fees             | FAILED | master | 5     |
| test_onchain_dust_out                   | FAILED | master | 5     |
|-----------------------------------------|--------|--------|-------|
2019-05-26 16:19:10 +02:00
Rusty Russell
e5b5f1d7e5 openingd: add openchannel hook.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-20 20:31:07 -04:00
Christian Decker
c96cc8a444 pytest: Fix the flaky test_plugin_connected_hook test
It was a bit eager :-)

Signed-off-by: Christian Decker <@cdecker>
2019-04-30 17:56:09 -05:00