Commit Graph

457 Commits

Author SHA1 Message Date
Rusty Russell
e4457e59ed pytest: test_gossip_jsonrpc can always test aliases.
The whole test is under DEVELOPER anyway, but even if it weren't it
would work fine.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
383b309a00 utils: make subd_pid return the *last* pid, in case we restarted daemon.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
e9483f12b6 test_lightningd.py: actually make sure we do zlib encoding.
Previous replies weren't large enough; add another channel and then
use IO tracing to make sure the reply is zlib encoded.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
9e51e196c1 gossipd: dev-set-max-scids-encode-size to artificially force "full" replies.
We cap each reply at a single one, which forces the code into our
recursion logic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
118f099dd8 gossip: dev-query-channel-range to test query_channel_range.
We keep a crappy bitmap, and finish when their replies cover
everything we asked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
c34b49c356 gossipd: add dev-send-timestamp-filter command for testing timestamp filtering.
Since we currently only (ab)use it to send everything, we need a way to
generate boutique queries for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
c633cbe2ee tests: add dev-query-scids
And write the test for it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
c2189229ca gossipd: only broadcast channel_announcement once we have a channel_update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
601a855543 tests/test_lightningd.py: test new delayed gossip channel_announce behaviour.
The gossip-query spec enhancements say not to forward an channel_announcement until
you have receive a channel_update.  This test fails for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
498ad91318 tests: new helper to get pid of a particular subdaemon.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Christian Decker
27a186be9c pytest: Stabilize LightningNode.openchannel for multiple channels
When opening a number channels from a single node we could end up not waiting
for the funding tx to make it into the mempool, instead triggering on a previous
`sendrawtransaction` or `CHANNEL_NORMAL` in the logs. This now checks that the
actual funding transaction makes it into the mempool and that we wait for the
depth change for that specific channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker
c17848a3f3 gossip: Disable local channels after loading the gossip_store
We don't have any connection yet, so how could they be active? Disable both
sides to avoid trying to route through them or telling others to use them as
`contact_points` in invoices.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker
202ce5e4ea pytest: Replace wait_for_channels with an actual check
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker
568cd9ff36 pytest: Making test_routing_gossip less flaky
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
ZmnSCPxj
0087105d6c test_lightningd: Test forget-if-fundee behavior. 2018-05-23 14:37:32 -07:00
Christian Decker
f2b89a9965 pytest: Simplify test_routing_gossip
Settling first and then asserting is longer than just waiting for it to succeed.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Christian Decker
1a45efd0a9 pytest: Use utils.TIMEOUT when settling in test_routing_gossip
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Christian Decker
2e8b3066e4 pytest: Filter out some unimportant log lines
Mostly `lightningd` complaining about not being able to estimate fees. Safes us
a lot of log space when some tests time out, and safes us a few context switches
between log appender and log watchers.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Christian Decker
1a9c7783b7 pytest: Yet another instance where we wait only on one side
Waiting on one end and then just asserting on the other is racy.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Christian Decker
f1c64db5e7 pytest: Wait on both sides in test_peerinfo
We were sort of assuming that one side telling it's ok would automagically make
the other side succeed as well.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Christian Decker
eb869cf144 pytest: Make test_sendpay less flaky
While 1 second is very generous, it resolving the HTLCs may take longer, so we
just loop over this instead of making it one-shot

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Christian Decker
a35cc9a79f pytest: Stabilize test_autocleaninvoice
The RPC calls aren't really free, so let's wait for absolute times, computed
from the `start_time` instead.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Christian Decker
dcbd27e7cf pytest: Actually wait for payments to confirm in benchmark
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Christian Decker
126f024cda pytest: Added a few more benchmarks
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Christian Decker
4d40ff37aa pytest: Fix the benchmark script
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Christian Decker
c4ca2ab4b9 pytest: Fix the benchmark script
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Rusty Russell
90dc5173d5 test_lightningd.py: remove 5 slowest tests from non-developer mode.
We can restore them once we get parallel testing on Travis, but meanwhile
we time out because of the 30 seconds bitcoind poll.

Running on my laptop with --duration=5:

=========================== slowest 5 test durations ===========================
184.07s call     tests/test_lightningd.py::LightningDTests::test_multiple_channels
156.66s call     tests/test_lightningd.py::LightningDTests::test_forward
155.77s call     tests/test_lightningd.py::LightningDTests::test_closing
126.83s call     tests/test_lightningd.py::LightningDTests::test_waitinvoice
126.11s call     tests/test_lightningd.py::LightningDTests::test_waitanyinvoice

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
a2dc3d02a8 locktime-blocks: rename to watchtime-blocks.
And clarify the descriptions for end users.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
fd86fbf2fc lightningd: rename --anchor-confirms to --funding-confirms.
That nomenclature won out in Milan.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
8739b4cbe8 lighningd: Remove --debug-subdaemon-io.
We can use SIGUSR1, even in non-developer builds.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
edf1b3cec9 More option cleanups.
Because we have too many which are never used and I don't want to document
them.

1. Remove unused anchor_onchain_wait.  When implemented, it should be
   hardcoded to 100 or more.
2. Remove anchor_confirms_max.  10 always reasonable, and we can readd
   an override option should someone need it.
3. max_htlc_expiry should be the same as locktime_max (which increases
   from 3 to 5 days by default): they're both a limit on how long
   funds can be locked up.
4. channel_update_interval should always be a dev option.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
0aa22741df option cleanup: --dev-override-fee-rates
Make --override-fee-rates a dev option.  We use default-fee-rate in
its place, which (since bitcoind won't give fee estimates in regtest
mode for short chains) gives an effective feerate of 15000/7500/3750.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
c8cc8fd83f option cleanup: --dev-bitcoind-poll
Make --bitcoind-poll a dev option.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
85a062164a test_lightningd: test_closing_torture timeout increase.
I hit this timeout on pytest and valgrind with -n5; use the timeout variable
from utils.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
c546b1bbb6 gossipd: specify origin of updates in errors.
@cdecker points out that in test_forward, where we manually create a route,
we get an error back which contains an update for an unknown channel.

We should still note this, but it's not an error for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
8ee60e2d8e testing: make sure we don't see gossip in bad order.
This is something which generally shouldn't happen, but we didn't
notice it previously.

We ignore this warning in the case where a channel was deleted: this
happens because one side can send an update while the other notices
that the channel is closed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
cdded4f53d tests/utils.py: make default timeout configurable via $TIMEOUT
This is useful when you're expecting a test to fail, you've just added
diagnostics.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell
c33bbb2639 test_lightningd.py: wait longer test_permfail for l2 to notice new blocks.
Reproduced this failure locally: l2 hadn't seen the block yet.  Timeout
was too aggressive.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 06:13:49 +00:00
Rusty Russell
2caeab506b tests/test_lightningd.py: make test_blockchaintrack safe to re-run.
It gets confused if re-run due to flaky, since:
1. Using the same HSM, it generates the same spend and sees a previous one.
2. The block height numbers are off.

Fixes: #1479
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 06:13:49 +00:00
Rusty Russell
ecebbe700b tests/test_lightningd.py: remove unneeded btc variable.
We have a 'bitcoind' global: getting it from inside one of the daemons
was a mistake I've copied widely.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 06:13:49 +00:00
ZmnSCPxj
757c059cd4 test_lightningd: Add closingd torture test. 2018-05-09 04:43:21 +00:00
Christian Decker
783327622f pytest: Mark test_blockchaintrack as flaky
The flakyness is probably due to timing issues when reorging.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Christian Decker
fac1d24ce3 pytest: Use the port from the lightning node
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Christian Decker
5f7b0fdf92 pytest: Marking test_permfail as flaky
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Christian Decker
104645ba3a pytest: Fix flaky test_closing_id test
It simply wasn't waiting for us to register the peer before attempting to open a
connection. Moved into a separate test to be able rerun multiple times.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 05:43:35 +00:00
Rusty Russell
79902b48c7 test_lightningd.py: remove gratuitous 'addr' option from test_connect_by_gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
d40d22b68e gossipd: don't try to connect to non-routable addresses.
Someone could try to announce an internal address, and we might probe
it.

This breaks tests, so we add '--dev-allow-localhost' for our tests, so
we don't eliminate that one.  Of course, now we need to skip some more
tests in non-developer mode.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
af065417e1 gossipd: handle wildcard addresses correctly.
If we're given a wildcard address, we can't announce it like that: we need
to try to turn it into a real address (using guess_address).  Then we
use that address.  As a side-effect of this cleanup, we only announce
*any* '--addr' if it's routable.

This fix means that our tests have to force '--announce-addr' because
otherwise localhost isn't routable.

This means that gossipd really controls the addresses now, and breaks
them into two arrays: what we bind to, and what we announce.  That is
now what we return to the master for json_getinfo(), which prints them
as 'bindings' and 'addresses' respectively.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
52917ff6c9 More flexible address wildcards, only add wildcard if nothing else.
1. Add special option where an empty host means 'wildcard for IPv4 and/or IPv6'
   which means ':1234' can be used to set only the portnum.
2. Only add this protocol wildcard if --autolisten=1 (default)
   and no other addresses specified.
3. Pass it down to gossipd, so it can handle errors correctly: in most cases,
   it's fatal not to be able to bind to a port, but for this case, it's OK
   if we can only bind to one of IPv4/v6 (fatal iff neither).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
fe96fe10c7 Clean up network options.
It's become clear that our network options are insufficient, with the coming
addition of Tor and unix domain support.

Currently:

1. We always bind to local IPv4 and IPv6 sockets, unless --port=0, --offline,
   or any address is specified explicitly.  If they're routable, we announce.
2. --addr is used to announce, but not to control binding.

After this change:

1. --port is deprecated.
2. --addr controls what we bind to and announce.
3. --bind-addr/--announce-addr can be used to control one and not the other.
4. Unless --autolisten=0, we add local IPv4 & IPv6 port 9735 (and announce if they are routable).
5. --offline still overrides listening (though announcing is still the same).

This means we can bind to as many ports/interfaces as we want, and for
special effects we can announce different things (eg. we're sitting
behind a port forward or a proxy).

What remains to implement is semi-automatic binding: we should be able
to say '--addr=0.0.0.0:9999' and have the address resolve at bind
time, or even '--addr=0.0.0.0:0' and have the port autoresolve too
(you could determine what it was from 'lightning-cli getinfo'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00