Commit Graph

89 Commits

Author SHA1 Message Date
Rusty Russell
383b309a00 utils: make subd_pid return the *last* pid, in case we restarted daemon.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
498ad91318 tests: new helper to get pid of a particular subdaemon.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Christian Decker
27a186be9c pytest: Stabilize LightningNode.openchannel for multiple channels
When opening a number channels from a single node we could end up not waiting
for the funding tx to make it into the mempool, instead triggering on a previous
`sendrawtransaction` or `CHANNEL_NORMAL` in the logs. This now checks that the
actual funding transaction makes it into the mempool and that we wait for the
depth change for that specific channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker
202ce5e4ea pytest: Replace wait_for_channels with an actual check
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker
2e8b3066e4 pytest: Filter out some unimportant log lines
Mostly `lightningd` complaining about not being able to estimate fees. Safes us
a lot of log space when some tests time out, and safes us a few context switches
between log appender and log watchers.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-23 00:20:19 +00:00
Rusty Russell
a2dc3d02a8 locktime-blocks: rename to watchtime-blocks.
And clarify the descriptions for end users.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
0aa22741df option cleanup: --dev-override-fee-rates
Make --override-fee-rates a dev option.  We use default-fee-rate in
its place, which (since bitcoind won't give fee estimates in regtest
mode for short chains) gives an effective feerate of 15000/7500/3750.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
c8cc8fd83f option cleanup: --dev-bitcoind-poll
Make --bitcoind-poll a dev option.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
cdded4f53d tests/utils.py: make default timeout configurable via $TIMEOUT
This is useful when you're expecting a test to fail, you've just added
diagnostics.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell
d40d22b68e gossipd: don't try to connect to non-routable addresses.
Someone could try to announce an internal address, and we might probe
it.

This breaks tests, so we add '--dev-allow-localhost' for our tests, so
we don't eliminate that one.  Of course, now we need to skip some more
tests in non-developer mode.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
af065417e1 gossipd: handle wildcard addresses correctly.
If we're given a wildcard address, we can't announce it like that: we need
to try to turn it into a real address (using guess_address).  Then we
use that address.  As a side-effect of this cleanup, we only announce
*any* '--addr' if it's routable.

This fix means that our tests have to force '--announce-addr' because
otherwise localhost isn't routable.

This means that gossipd really controls the addresses now, and breaks
them into two arrays: what we bind to, and what we announce.  That is
now what we return to the master for json_getinfo(), which prints them
as 'bindings' and 'addresses' respectively.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
fe96fe10c7 Clean up network options.
It's become clear that our network options are insufficient, with the coming
addition of Tor and unix domain support.

Currently:

1. We always bind to local IPv4 and IPv6 sockets, unless --port=0, --offline,
   or any address is specified explicitly.  If they're routable, we announce.
2. --addr is used to announce, but not to control binding.

After this change:

1. --port is deprecated.
2. --addr controls what we bind to and announce.
3. --bind-addr/--announce-addr can be used to control one and not the other.
4. Unless --autolisten=0, we add local IPv4 & IPv6 port 9735 (and announce if they are routable).
5. --offline still overrides listening (though announcing is still the same).

This means we can bind to as many ports/interfaces as we want, and for
special effects we can announce different things (eg. we're sitting
behind a port forward or a proxy).

What remains to implement is semi-automatic binding: we should be able
to say '--addr=0.0.0.0:9999' and have the address resolve at bind
time, or even '--addr=0.0.0.0:0' and have the port autoresolve too
(you could determine what it was from 'lightning-cli getinfo'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
6ba9a7cb57 tests/test_lightningd.py: move port into node.
We're about to change the JSONRPC, so let's put an explicit 'port' into
our node class.

We initialize it at startup time: in future I hope to use ephemeral ports
to make our tests more easily parallelizable.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
7d27501c94 tests/utils.py: remove unused connect routine.
It's broken anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Christian Decker
6231e9969f pytest: Give lightningd nodes a numeric ID to prefix logs
This used to be the port, but since we no longer have fixed ports, and we start
them in random order we can't easily distinguish them by the port anymore. Just
use a numeric ID that matches their lightning-dirs.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
071ef628db pytest: Use random ports for bitcoind and lightningd to allow parallel testing
Adds a new dependency, but totally worth it :-)

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
e3bac12683 pytest: Allow stdout to be dropped for bitcoind
We never really look at the output, and it's rather noisy, so we just stop
writing to the log.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
81b715aae9 pytest: Move wait_for to utils.py and make fund_channel a member
Slowly moving towards a more pythonic approach to the testing code.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Rusty Russell
ac51231166 lightningd: remove --dev-hsm-seed option.
We can create the hsm file from python directly; that works even if we
don't have DEVELOPER set, and is simpler.

We add a test that the aliases are correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-05 17:55:10 +02:00
Christian Decker
ac9e5581c8 pytest: Start nodes with --rescan=1
This shaves off about 15% of our integration testing suite on my machine. It
assumes we never reorg below the first block the node starts with, which is true
for all tests, so it's safe.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Rusty Russell
d0bfd8407a test_lightningd.py: catch unexpected reconnections.
I had a weird failure which was caused by an unexpected disconnect and
reconnecct.  Since we are prersistend and recover from these, they can
slip through our tests; most tests don't involve reconnection, so we
need to catch this explicitly.

For the connect() helper, we always suppress reconnection; tests which
want it all want other options so don't use this helper anyway.  (Actually,
after I said that, test_closing_while_disconnected was added when I
rebased, which did require it, so I had to open-code that one).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-23 20:18:15 +00:00
Christian Decker
4da3d407b4 pytest: Use a dict to pass options to lightningd
This allows us to have some default options that can then be overridden easily
on a per-test basis.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-06 06:52:22 +00:00
Rusty Russell
a077c3defb lightningd: test for weird aliases.
Test has inverted conditions to make it "pass", we fix as we go.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
practicalswift
d2109bfe0c Fix flake8 warnings 2018-02-26 12:38:00 +01:00
practicalswift
ae7d857c44 Make Python code conform to the Python PEP 8 style guide 2018-02-22 10:47:19 +01:00
Rusty Russell
d47d85fa30 wallet: add list of upgrades.
Useful for debugging a db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 01:38:32 +00:00
William Casarin
3d27bbb47d test: test_bech32_funding
1. Test wallet funding to a bech32 p2wpkh address
2. Test channel opening with this address

Signed-off-by: William Casarin <jb55@jb55.com>
2018-02-06 01:37:58 +00:00
cornwarecjp
6112df3a90 Bring back the time-out on the start-up of bitcoind, but make it 60 seconds instead of the original 10 seconds. This is much larger than the normal start-up time of bitcoind (so we should never hit the time-out), but still lets a test terminate in a reasonable time in the unlikely case that bitcoind hangs. 2018-02-01 23:00:43 +01:00
cornwarecjp
896f7ef37d Make it possible to run the test suite in Python 3.4 (bytes.hex was introduced in 3.5) 2018-02-01 23:00:43 +01:00
cornwarecjp
0b0708d86a Bitcoind should always eventually finish starting, but there is no guarantee on how long it takes, so don't apply a time-out. In case bitcoind hangs indefinitely, the test will hang indefinitely too, but this should be solved in bitcoind, not in the test code. 2018-02-01 23:00:43 +01:00
Rusty Russell
0d3eaef945 tests: add argument to is_in_log so we can check only recent log entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-31 11:37:22 +01:00
practicalswift
7e8f1945f2 Adjust tests to new log text 2018-01-29 03:22:27 +00:00
practicalswift
689db5b7c1 Onboarding: Make log output texts friendlier to new users 2018-01-29 03:22:27 +00:00
Christian Decker
45c935ddba pytest: Have the db_query helper work on a copy of the DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-27 02:27:35 +00:00
Christian Decker
dafd000c2a pytest: Check that we disable on closing and permfail.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-26 01:09:44 +00:00
Rusty Russell
4f1dc91ba5 lightningd: rename --deprecated-apis to --allow-deprecated-apis.
Suggested-byL practicalswift
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
78672738ff test_lightningd.py: remove deprecated APIs.
Individual tests can always re-enable them, though.

[ More test fallout fixes by Christian Decker ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
962514c3a0 addfunds: remove.
We now detect funds, so this is just confusing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
15aaf9f49a test_lightning.py: shutdown tail thread before killing.
Seems to avoid the nasty python resource warnings, as well as the
fatal 'ValueError: PyMemoryView_FromBuffer(): info->buf must not be NULL'

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-17 23:55:35 +01:00
Rusty Russell
79dc44713b channeld: --ignore-fee-limits as a hack for fee disparities.
This, of course, should never be used.  But it helps maintain connections
for the moment while we dig deeper into feerates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-16 12:22:06 +01:00
Christian Decker
90527498bc pytest: Do not re-print logs if we failed to find a log entry
CI always runs with TEST_DEBUG=1 which prints logs anyway, and testing
locally should also be done this way, combined with pytest which
captures the logs. No need to duplicate the functionality of pytest.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-10 03:52:46 +00:00
Christian Decker
3e0efb6f8d pytest: Avoid leaking stdout fds 2018-01-08 14:36:33 +01:00
practicalswift
6757255ba4 Use deterministic command-line ordering when executing lightningd from tests 2018-01-07 17:28:34 +01:00
Christian Decker
2a7ba010c3 pytest: Use the bitcoind config file to read RPC params from
With python-bitcoinlib==0.9.0 it appears that the URL based auth
information is no longer used, so we fall back to reading the config
file for the bitcoind daemon instead.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-23 06:15:33 +00:00
Rusty Russell
6b9c525f35 lightningd: use env var not cmdline to suppress backtrace.
We now set it up *before* parsing cmdline, so this is more convenient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Christian Decker
e154f4a019 pytest: Minor cleanup
Now using assertRaisesRegex instead of try-except and added restart to
nodes.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Rusty Russell
70cebb8653 test_addfunds_from_block: fix test flakiness.
If you run locally, it fails occasionally; presumably because it
sees previous funds.  Use a random HSM key for that teste.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:52:18 +01:00
Christian Decker
12944ffc9b pytest: Switch to 'Hello world' as startup marker
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-29 14:39:12 +01:00
Rusty Russell
cfde208e82 tests: prepare for bitcoind 0.16.
I noticed some breakage with git master:
1. getinfo no longer supported (for us, use getblockchaininfo)
2. generate no longer supported (use generatetoaddress)

Both these options are supported at least in 0.15, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-28 12:15:22 +01:00
Rusty Russell
88af0f5bf8 tests: change --override-fee-rate to --override-fee-rates and use in tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00