Commit Graph

928 Commits

Author SHA1 Message Date
Christian Decker
5e36257375 pytest: Test that we delete channels opened below our birth height
This was the initial issue that was addressed by #2756 and now we just test
that all is working as expected.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-09 02:31:51 +00:00
Christian Decker
98447e454e gossip: Use the getfilteredblock method to look up scid outputs
Just a tiny shim to reconcile the `get_output` with `getfileteredblock`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-09 02:31:51 +00:00
Christian Decker
5bb411b572 pytest: Add test for the block backfilling trigger by gossip
This is just the test that we use to verify block backfilling below the wallet
birth height is working correctly.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-09 02:31:51 +00:00
Rusty Russell
118150227e cli: restore 0.7.0-style whitespace printing.
@renepickhardt has a shell script we broke.  While we still produce
perfectly valid JSON, we should not gratuitously change tool output.

Plus, I prefer the missing space before the ':'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-08 18:16:48 +08:00
Rusty Russell
b73a85a75e lightningd: don't say 'killing channel' when HTLC times out.
We're actually only killing the connection.  I saw this in my logs,
but it was all OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-07 21:12:52 +08:00
Christian Decker
a85d3dc220 pytest: Make test_funding_cancel_race SLOW_MACHINE compatible
This test is spawning 100 nodes concurrently, which is a lot even when not
running with `valgrind`, especially when executing tests in parallel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-07 01:54:43 +00:00
Christian Decker
e2f9a410f4 pytest: Run plugin --help tests in the test directory
This is a followup to #2892. Since we now attempt to lock the PID file before
starting plugins we need to make sure that we actually use a unique lightning
directory for anything that attempts to call `--help`. If not we may be
conflicting with a `lightningd` that is running against that directory.

Notice that this still means that we will be unable to call `--help` on
`lightningd` if we have a running instance, but isolation in this case is
good, otherwise we'd be reading the default config anyway.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-06 13:10:32 +08:00
Christian Decker
9d4148ce68 pylightning: Warn users of plugins that may break due to extra args
We recently noticed that the way we unpack the call arguments for hooks and
notifications in pylightning breaks pretty quickly once you start changing the
hook and notification params. If you add params they will not get mapped
correctly causing the plugin to error out.

This can be fixed by adding a `VAR_KEYWORD` argument to the calbacks, i.e., by
adding a single `**kwargs` argument at the end of the signature. This commit
adds a check that such a catch-all argument exists, and emits a warning if it
doesn't.

It also fixes up the plugins that we ship ourselves.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-05 00:19:48 +00:00
Rusty Russell
979fbeb3b0 lightningd: simplify --daemon.
Dumb programs which have a --daemon option call fork() early.  This is
terrible UX since startup errors get lost: the program exits with
"success" immediately then you discover via the logs that it didn't
start at all.

However, forking late introduced a heap of problems with changing
pids.  Instead, fork early but keep stderr and the parent around: if
we fail early on, the parent fails with us.  We release our parent
with an explicit action just before the main loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Rusty Russell
fc024f81d6 pytest: check that --daemon still means we recognize our own children.
We create our children then fork, so we're not a parent.  I noticed this
because 'lightning-cli stop' takes a long time: this is because it tries to
wait for them and they don't respond.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Rusty Russell
b460590278 plugins: detect and fixup old relative paths.
Note that we move adding the plugin to the plugins list to the end, otherwise
the hook from logging can examine the (uninitialized) plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell
4fcd94a091 pytest: use absolute paths for plugin arguments.
We're going to change this in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
ZmnSCPxj
3e74ca4b86 gossipd/routing.c: Correctly handle a duplicated entry in exclude of getroute. 2019-08-02 16:06:15 +02:00
ZmnSCPxj
a5fb37298c tests/test_gossip.py: Add test to check that duplicated exclusions in getroute have no lasting effect. 2019-08-02 16:06:15 +02:00
trueptolemy
5ac02907bb pytest: Add the test for 'forward_event' in test_plugin 2019-08-01 18:49:25 +08:00
trueptolemy
d2423c6f9a pytest: Add a simple plugin to test 'forward_event' 2019-08-01 18:49:25 +08:00
Rene Pickhardt
dbc0265a5d added mako dependency to tests/requirements.txt
this should partially fix #2879 

this dependency seems to be needed in `tools/generate-wire.py`
2019-07-31 07:46:49 +08:00
Rusty Russell
79d32ec2f2 plugin: notice when plugin has *started* configuring.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 08:44:01 +08:00
Rusty Russell
df8a6f615b pytest: test for crash when we do a plugin op and a plugin has not finished init
We try to start it twice, with predictable hilarious results:

   DEBUG plugin-manager started(20701) /home/rusty/devel/cvs/lightning/tests/plugins/slow_init.py
   DEBUG lightningd(20670): Adding block 101: 0f26301caaae390d1b956c3eb73827d1f87a2af306561348045bf1313b7551b0
   INFO plugin-slow_init.py slow_init.py initializing
   DEBUG plugin-manager started(20704) /home/rusty/devel/cvs/lightning/tests/plugins/slow_init.py
   **BROKEN** lightningd(20670): FATAL SIGNAL 6 (version v0.7.1-146-g7d3b352-modded)
   **BROKEN** lightningd(20670): backtrace: common/daemon.c:45 (send_backtrace) 0x55a29c3ec539
   **BROKEN** lightningd(20670): backtrace: common/daemon.c:53 (crashdump) 0x55a29c3ec589
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281ebcf5f
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281ebced7
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281e9e534
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281e9e40e
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281eae011
   **BROKEN** lightningd(20670): backtrace: lightningd/json_stream.c:225 (json_stream_output_) 0x55a29c3bff5f
   **BROKEN** lightningd(20670): backtrace: lightningd/plugin.c:383 (plugin_write_json) 0x55a29c3e103f
   **BROKEN** lightningd(20670): backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x55a29c43316b
   **BROKEN** lightningd(20670): backtrace: ccan/ccan/io/io.c:435 (io_do_always) 0x55a29c433df1
   **BROKEN** lightningd(20670): backtrace: ccan/ccan/io/poll.c:300 (handle_always) 0x55a29c435865
   **BROKEN** lightningd(20670): backtrace: ccan/ccan/io/poll.c:377 (io_loop) 0x55a29c435b3d
   **BROKEN** lightningd(20670): backtrace: lightningd/io_loop_with_timers.c:24 (io_loop_with_timers) 0x55a29c3bdfcf
   **BROKEN** lightningd(20670): backtrace: lightningd/lightningd.c:830 (main) 0x55a29c3c44b6
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281e9fb6a
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x55a29c3ac0f9
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0xffffffffffffffff

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 08:44:01 +08:00
darosior
307fb0708e lightningd/plugin_control: don't control non-dynamic plugins 2019-07-28 07:24:04 +00:00
darosior
9ddc2e0334 Add the 'plugin' subcommands to pylightning and add a test for them 2019-07-28 07:24:04 +00:00
darosior
2683170da8 Test and document 'channel_opened' notification 2019-07-27 12:18:25 +02:00
Simon Vrouwe
af7044d09d pytest: test_fee_limits, add case where opening channel is denied when feerate is too low 2019-07-27 05:14:34 +00:00
darosior
34533ead68 Plugins: Add a test for the 'invoice_payment' notfication 2019-07-25 11:19:47 +08:00
Rusty Russell
d59e2b1b4b developer: add --dev-force-bip32-seed to force a specific BIP32 seed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
Rusty Russell
fb6870c139 param: implement helpers for multiplex commands.
Our previous param support was a bit limited in this case.

We create a dev- command multiplexer, so we can exercise it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-16 21:39:14 +00:00
Christian Decker
53488e5739 pytest: Strengthen the htlc_accepted tests
We were having a few issues with malformed data in the past, so this time we
really check that stuff.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-13 14:44:12 +02:00
Rusty Russell
789d14299b pytest: mix up order of cancel vs complete in test_funding_cancel_race
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-08 15:14:31 +00:00
Rusty Russell
ee8edfe6a6 pytest: severely reduce scope of test_funding_cancel_race under valgrind.
Otherwise we get timeouts across the board.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-08 15:14:31 +00:00
Rusty Russell
280bd60988 lightningd: allow multiple cancels on a single fundchannel command.
Instead of taking over the ->cmd pointer, append ourselves to a list
of cancels.  This fixes the test_funding_cancel_race.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-08 15:14:31 +00:00
Rusty Russell
13dbe6e74b pytest: stress test funding_cancel vs funding_complete
This fails, because they fight over the fc->cmd pointer, leaving
others hanging.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-08 15:14:31 +00:00
Rusty Russell
54ce4ed1cf pytest: fail tests if we get any LOG_BROKEN level messages, unless flagged.
And clean up some dev ones which actually happen (mainly by calling
channel_fail_permanent which logs UNUSUAL, rather than
channel_internal_error which logs BROKEN).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-02 03:26:10 +00:00
Rusty Russell
5f02294b5b lightningd/log: prefix log messages with level.
In particular, this lets us spot UNUSUAL and BROKEN messages easily.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-02 03:26:10 +00:00
Christian Decker
915c932e2d pytest: Make sure the channels entries don't get deleted anymore
Signed-off-by: Christian Decker <@cdecker>
2019-06-30 16:41:30 +09:30
Rusty Russell
c303d7d534 gossipd: only do (automatic) store compaction at startup.
Rewriting the gossip_store is much more trivial when we don't have
any pointers into it, so add some simple offline compaction code
and disable the automatic compaction code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-21 20:03:10 -05:00
Rusty Russell
c15d9ed37c gossip_store: make copy of corrupt gossip_store on failure.
This should help debugging vastly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-21 22:03:35 +00:00
Rusty Russell
8928f0b5f9 gossipd: remove gossip entirely if we hit a problem on load.
The crashes in #2750 are mostly caused by us trying to partially truncate
the store.  The simplest fix for release is to discard the whole thing if
we detect a problem.

This is a workaround: it'd be far nicer to try to recover.

Fixes: #2750
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-21 22:03:35 +00:00
Rusty Russell
9bf0467967 pytest: fix test_gossip_store_load_no_channel_update
It wasn't invalid due to a missing channel_update, but in fact was a
bad checksum due to a cut & paste bug.  Fix that, and assert it's not
actually truncating.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-21 22:03:35 +00:00
Rusty Russell
47b5f2e837 gossipd: truncate gossip_store.tmp for compaction.
If something went wrong and there was an old one, we were
appending to it!

Reported-by: @SimonVrouwe
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-20 02:53:52 +00:00
lisa neigut
92bd3a3587 connect test: use wait_for_log in test_funding_external_wallet
test_funding_external_wallet is flakey because we don't wait for
the channel state to update before asserting that it's changed.
2019-06-18 06:54:21 +00:00
Rusty Russell
5e3690b3c5 gossipd: delete channel_amount from the store when we delete channel_announcement.
Otherwise we slowly build up cruft: compaction simply moves them since
they're not deleted.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-15 10:52:05 +02:00
Rusty Russell
10c503b4b4 gossip_store: clean up a truncated store.
We might have channel_announcements which have no channel_update: normally
these don't get written into the store until there is one, but if the
store was truncated it can happen.  We then get upset on compaction, since
we don't have an in-memory representation of the channel_announcement.

Similarly, we leave the node_announcement pending until after that
channel_announcement, leading to a similar case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-15 10:52:05 +02:00
Rusty Russell
adc52b6ee8 pytest: add test for dangling channel_announcement/node_announcement after gossip_store.
This can happen if the store was truncated.

Reported-by: @jb55
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-15 10:52:05 +02:00
Rusty Russell
a35ab51a06 pytest: gossip_store test for channel_amount truncated.
We pass, but this test should have been added a while ago with the
original code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-15 10:52:05 +02:00
Rusty Russell
909f22f117 pytest: gossip_store test for node_announcement before update.
We pass, but this test should have been added a while ago with the fix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-15 10:52:05 +02:00
Rusty Russell
eb5cc47bdd gossipd: count deleted records correctly when loading gossip_store.
The result of an incorrect count was that we failed on next compaction.

Fixes: #2743
Fixes: #2742
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-14 02:17:32 +00:00
Rusty Russell
12a523f7c5 pytest: add (xfail) test for store load miscount.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-14 02:17:32 +00:00
lisa neigut
c00e0d2936 funding: rename fundchannel_continue -> _complete
Renaming. "complete" more accurately describes what we're doing here.
2019-06-12 02:22:43 +00:00
lisa neigut
3ae78a61fa tests: finish up test for external funding flow
Add to test for fundchannel with composing and broadcasting
an external transaction.
2019-06-12 02:22:43 +00:00
lisa neigut
5aad642c59 opening: add fundchannel_cancel command
Provide the option to cancel a funding-opening with a peer.
Must either call `fundchannel_cancel` or `fundchannel_continue`
2019-06-12 02:22:43 +00:00