Commit Graph

487 Commits

Author SHA1 Message Date
Rusty Russell
dba7f9002f gossipd: provide connectd with address resolution.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell
3d3d2ef9af gossipd: remove connectd functionality, enable connectd.
This patch guts gossipd of all peer-related functionality, and hands
all the peer-related requests to channeld instead.

gossipd now gets the final announcable addresses in its init msg, since
it doesn't handle socket binding any more.

lightningd now actually starts connectd, and activates it.  The init
messages for both gossipd and connectd still contain redundant fields
which need cleaning up.

There are shims to handle the fact that connectd's wire messages are
still (mostly) gossipd messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell
f4e2a9c921 pytest: don't expect alias and color in listpeers.
Gossipd combines the information if it knows it, but that's really the
job of 'listnodes'.  More importantly, channeld won't have access to
this information.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell
7986af1b1e pytest: only_one() helper to catch if RPC returns more elements than we expect
I saw an error in test_gossip_weirdalias in Travis, where listnodes(nodeid)
returned *BOTH* nodes; it happened to fail because [0] was the wrong one, but
it would have passed if the order had been different.

This helper asserts that we really do only have one element, and should
catch such bugs faster.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-18 15:11:05 +02:00
Rusty Russell
edf043c89b pytest: make utils.py read config.vars.
I could not figure out why test_announce_address suddenly stopped working:
I had previously been using DEVELOPER=1 on the cmdline for historical
reasons when testing locally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-16 23:16:34 +02:00
Rusty Russell
a37e692af6 pytest: mark test_reconnect_normal flaky for now.
The bad gossip order bug is going to take a while to fix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-16 22:48:41 +02:00
Rusty Russell
e0c21debc2 pytest: fix sleep(1) in test_channel_persistence.
It's obviously not enough under load, as Travis demonstrates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-15 17:37:29 +02:00
Rusty Russell
b1182702ae pytest: increase timeout for test_payment_failed_persistence
This happened under travis.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-15 17:37:29 +02:00
Rusty Russell
dd7afc33ee pytest: fix flaky test in test_htlc_sig_persistence
We can shutdown before the tx hits bitcoind.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:26:23 +02:00
Rusty Russell
ed83bbe623 pytest: fix flaky race in test_gossip_query_channel_range.
We weren't waiting for gossipd to actually process the
dev_set_max_scids_encode_size message, so under Travis it sometimes
split the reply before processing that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:26:23 +02:00
Rusty Russell
55a09d79b9 pytest: make test_gossip_no_empty_announcements robust when update delayed.
In this test we tell l3 to disconnect on sending WIRE_CHANNEL_ANNOUNCEMENT.
This is hit by gossipd (to disconnect from l2) but *also* channeld to
disconnect from l4.  That's OK, because normally by this point l4 has
sent its real channel_update.

However, the next patch introduces a delay in sending channel_updates,
meaning l4 hasn't sent it yet.  If l3 doesn't reconnect to l4, we
never get the channel_update and the test which expects l1 to eventually
see both sides of the channel fails.

So we manually reconnect then.  Note that we remove the redundant
'dev-no-reconnect' option from l2: it's added automatically as it
doesn't set 'may_reconnect'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell
9fa738a741 listpeers: expose peer features as 'local_features' and 'global_features'
For now, just the connected peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell
c46f373205 options: refuse two --announce-addr of the same type.
Gossipd will ignore the second one, but doing it in the front end
gives an explicit error message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 15:03:21 +02:00
Rusty Russell
f67182ff20 gossipd: order node_announcement addresses correctly, remove duplicate types.
Fixes: #1596
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 15:03:21 +02:00
Rusty Russell
e6f31b3c66 pytest: allow duplicated options.
Python dict can't have duplicate entries, but some options can be specified
multiple times.  The easiest way is to put a list in the dict.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 15:03:21 +02:00
Christian Decker
6c82d1b8b0 pylightning: Subclass ValueError to access to the returned error 2018-07-01 04:13:04 +00:00
Rusty Russell
ce5b4bddf2 tests: fix to work with bitcoind master branch.
E               ConnectionRefusedError: [Errno 111] Connection refused

And in debug.log:

2018-05-17T04:06:35Z Warning: Config setting for -rpcport only applied on regtest network when in [regtest] section.

Unfortunately, current versions including 0.16.1 *ignore* the contents of
a '[regtest]' section, so we need it in *both* places.

Also remove the misleading 'rpcport' initialization which we always
override.

Note that we don't fix this message though:

	2018-05-17T04:06:35Z Config options rpcuser and rpcpassword will soon be deprecated. Locally-run instances may remove rpcuser to use cookie-based auth, or may be replaced with rpcauth. Please see share/rpcuser for rpcauth auth generation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-21 13:43:32 +02:00
Rusty Russell
91d62ad697 pytest: create only a single config file.
For some reason, we created a second bitcoin.conf in the regtest/ directory,
which AFAICT nothing uses.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-21 13:43:32 +02:00
Rusty Russell
2375302534 pytest: move regtest argument into config file.
It's not optional for our test setup, and this makes it easier to invoke
bitcoin-cli manually, for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-21 13:43:32 +02:00
Rusty Russell
c78afa9201 onchaind: use lowball fee instead of donating to miners.
As of bitcoind 0.16.1, you can't send a single-input OP_RETURN output,
as you get 'tx-too-small'.

	sendrawtx exit 26, gave error code: -26?error message:?tx-size-small (code 64)?'

So instead we use the minimum fee we can, but otherwise ignore it and
don't wait for it to be mined.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-21 13:43:32 +02:00
Christian Decker
e556bf4f36 pytest: Disable DNS lookups during test runs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-21 11:21:16 +02:00
Mark Beckwith
8f0ef1636f Added wallet related error codes
New codes: FUND_MAX_EXCEEDED, FUND_CANNOT_AFFORD, FUND_DUST_LIMIT_UNMET.

The error message "Cannot afford fee" was not exactly correct because
it would also occur if the amount requested could not be afforded.  So
I changed it to the more generic "Cannot afford transaction".

Other things:

* Fixed off-by-one satoshi in fundchannel manpage.
* Changed 'arror' to 'error' because we are not pirates.
2018-06-18 12:33:25 +02:00
Christian Decker
583c1c50f8 pytest: Explicitly state the max fee multiplier we assumed in tests
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-14 00:59:42 +00:00
Christian Decker
4dca6daf34 opts: Bump locktime_max so we don't disagree as much with lnd
I still believe that 2 weeks is way too much, but we were promised that these
defaults would be slowly reduced to saner values as the stability increases.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-14 00:59:42 +00:00
Rusty Russell
8183590cdb pytest: test_onchaind_replay needs to wait for tx broadcast.
Waiting for 'Disabling channel' is not enough, since it's async to the
actual tx broadcast: I caught a case where the unilateral close wasn't
in the block, and so failed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-13 16:22:23 +02:00
Rusty Russell
965c20caae tests: reenable developer tests.
72d103d6bb deprecated DEVELOPER env var
in favor of config.vars, but didn't update test_closing.py or test_gossip.py.

5d0a54b7f0 then removed the explicit
DEVELOPER= setting from Travis.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-13 16:22:23 +02:00
Christian Decker
135bd962f7 pytest: Mark test_closing_different_fees as flaky
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:39:27 +02:00
Rusty Russell
72d103d6bb Makefile: import config.vars.
We leave VALGRIND env var as an override for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Christian Decker
de3b359782 pytest: Use pytest fixtures for the test directory and clean it up
The modern, pytest based, tests now clean up after themselves by removing
directories of successful tests and the base directory if there was no failure.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:32:27 +00:00
Christian Decker
38b7a0e2d2 pytest: Remove directories of successful tests
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:32:27 +00:00
Rusty Russell
e4457e59ed pytest: test_gossip_jsonrpc can always test aliases.
The whole test is under DEVELOPER anyway, but even if it weren't it
would work fine.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
383b309a00 utils: make subd_pid return the *last* pid, in case we restarted daemon.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
e9483f12b6 test_lightningd.py: actually make sure we do zlib encoding.
Previous replies weren't large enough; add another channel and then
use IO tracing to make sure the reply is zlib encoded.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
9e51e196c1 gossipd: dev-set-max-scids-encode-size to artificially force "full" replies.
We cap each reply at a single one, which forces the code into our
recursion logic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
118f099dd8 gossip: dev-query-channel-range to test query_channel_range.
We keep a crappy bitmap, and finish when their replies cover
everything we asked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
c34b49c356 gossipd: add dev-send-timestamp-filter command for testing timestamp filtering.
Since we currently only (ab)use it to send everything, we need a way to
generate boutique queries for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
c633cbe2ee tests: add dev-query-scids
And write the test for it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
c2189229ca gossipd: only broadcast channel_announcement once we have a channel_update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
601a855543 tests/test_lightningd.py: test new delayed gossip channel_announce behaviour.
The gossip-query spec enhancements say not to forward an channel_announcement until
you have receive a channel_update.  This test fails for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
498ad91318 tests: new helper to get pid of a particular subdaemon.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Christian Decker
27a186be9c pytest: Stabilize LightningNode.openchannel for multiple channels
When opening a number channels from a single node we could end up not waiting
for the funding tx to make it into the mempool, instead triggering on a previous
`sendrawtransaction` or `CHANNEL_NORMAL` in the logs. This now checks that the
actual funding transaction makes it into the mempool and that we wait for the
depth change for that specific channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker
c17848a3f3 gossip: Disable local channels after loading the gossip_store
We don't have any connection yet, so how could they be active? Disable both
sides to avoid trying to route through them or telling others to use them as
`contact_points` in invoices.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker
202ce5e4ea pytest: Replace wait_for_channels with an actual check
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker
568cd9ff36 pytest: Making test_routing_gossip less flaky
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
ZmnSCPxj
0087105d6c test_lightningd: Test forget-if-fundee behavior. 2018-05-23 14:37:32 -07:00
Christian Decker
f2b89a9965 pytest: Simplify test_routing_gossip
Settling first and then asserting is longer than just waiting for it to succeed.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Christian Decker
1a45efd0a9 pytest: Use utils.TIMEOUT when settling in test_routing_gossip
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Christian Decker
2e8b3066e4 pytest: Filter out some unimportant log lines
Mostly `lightningd` complaining about not being able to estimate fees. Safes us
a lot of log space when some tests time out, and safes us a few context switches
between log appender and log watchers.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Christian Decker
1a9c7783b7 pytest: Yet another instance where we wait only on one side
Waiting on one end and then just asserting on the other is racy.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Christian Decker
f1c64db5e7 pytest: Wait on both sides in test_peerinfo
We were sort of assuming that one side telling it's ok would automagically make
the other side succeed as well.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00