Commit Graph

82 Commits

Author SHA1 Message Date
Rusty Russell
c8cc8fd83f option cleanup: --dev-bitcoind-poll
Make --bitcoind-poll a dev option.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
cdded4f53d tests/utils.py: make default timeout configurable via $TIMEOUT
This is useful when you're expecting a test to fail, you've just added
diagnostics.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell
d40d22b68e gossipd: don't try to connect to non-routable addresses.
Someone could try to announce an internal address, and we might probe
it.

This breaks tests, so we add '--dev-allow-localhost' for our tests, so
we don't eliminate that one.  Of course, now we need to skip some more
tests in non-developer mode.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
af065417e1 gossipd: handle wildcard addresses correctly.
If we're given a wildcard address, we can't announce it like that: we need
to try to turn it into a real address (using guess_address).  Then we
use that address.  As a side-effect of this cleanup, we only announce
*any* '--addr' if it's routable.

This fix means that our tests have to force '--announce-addr' because
otherwise localhost isn't routable.

This means that gossipd really controls the addresses now, and breaks
them into two arrays: what we bind to, and what we announce.  That is
now what we return to the master for json_getinfo(), which prints them
as 'bindings' and 'addresses' respectively.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
fe96fe10c7 Clean up network options.
It's become clear that our network options are insufficient, with the coming
addition of Tor and unix domain support.

Currently:

1. We always bind to local IPv4 and IPv6 sockets, unless --port=0, --offline,
   or any address is specified explicitly.  If they're routable, we announce.
2. --addr is used to announce, but not to control binding.

After this change:

1. --port is deprecated.
2. --addr controls what we bind to and announce.
3. --bind-addr/--announce-addr can be used to control one and not the other.
4. Unless --autolisten=0, we add local IPv4 & IPv6 port 9735 (and announce if they are routable).
5. --offline still overrides listening (though announcing is still the same).

This means we can bind to as many ports/interfaces as we want, and for
special effects we can announce different things (eg. we're sitting
behind a port forward or a proxy).

What remains to implement is semi-automatic binding: we should be able
to say '--addr=0.0.0.0:9999' and have the address resolve at bind
time, or even '--addr=0.0.0.0:0' and have the port autoresolve too
(you could determine what it was from 'lightning-cli getinfo'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
6ba9a7cb57 tests/test_lightningd.py: move port into node.
We're about to change the JSONRPC, so let's put an explicit 'port' into
our node class.

We initialize it at startup time: in future I hope to use ephemeral ports
to make our tests more easily parallelizable.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
7d27501c94 tests/utils.py: remove unused connect routine.
It's broken anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Christian Decker
6231e9969f pytest: Give lightningd nodes a numeric ID to prefix logs
This used to be the port, but since we no longer have fixed ports, and we start
them in random order we can't easily distinguish them by the port anymore. Just
use a numeric ID that matches their lightning-dirs.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
071ef628db pytest: Use random ports for bitcoind and lightningd to allow parallel testing
Adds a new dependency, but totally worth it :-)

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
e3bac12683 pytest: Allow stdout to be dropped for bitcoind
We never really look at the output, and it's rather noisy, so we just stop
writing to the log.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
81b715aae9 pytest: Move wait_for to utils.py and make fund_channel a member
Slowly moving towards a more pythonic approach to the testing code.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Rusty Russell
ac51231166 lightningd: remove --dev-hsm-seed option.
We can create the hsm file from python directly; that works even if we
don't have DEVELOPER set, and is simpler.

We add a test that the aliases are correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-05 17:55:10 +02:00
Christian Decker
ac9e5581c8 pytest: Start nodes with --rescan=1
This shaves off about 15% of our integration testing suite on my machine. It
assumes we never reorg below the first block the node starts with, which is true
for all tests, so it's safe.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Rusty Russell
d0bfd8407a test_lightningd.py: catch unexpected reconnections.
I had a weird failure which was caused by an unexpected disconnect and
reconnecct.  Since we are prersistend and recover from these, they can
slip through our tests; most tests don't involve reconnection, so we
need to catch this explicitly.

For the connect() helper, we always suppress reconnection; tests which
want it all want other options so don't use this helper anyway.  (Actually,
after I said that, test_closing_while_disconnected was added when I
rebased, which did require it, so I had to open-code that one).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-23 20:18:15 +00:00
Christian Decker
4da3d407b4 pytest: Use a dict to pass options to lightningd
This allows us to have some default options that can then be overridden easily
on a per-test basis.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-06 06:52:22 +00:00
Rusty Russell
a077c3defb lightningd: test for weird aliases.
Test has inverted conditions to make it "pass", we fix as we go.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
practicalswift
d2109bfe0c Fix flake8 warnings 2018-02-26 12:38:00 +01:00
practicalswift
ae7d857c44 Make Python code conform to the Python PEP 8 style guide 2018-02-22 10:47:19 +01:00
Rusty Russell
d47d85fa30 wallet: add list of upgrades.
Useful for debugging a db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 01:38:32 +00:00
William Casarin
3d27bbb47d test: test_bech32_funding
1. Test wallet funding to a bech32 p2wpkh address
2. Test channel opening with this address

Signed-off-by: William Casarin <jb55@jb55.com>
2018-02-06 01:37:58 +00:00
cornwarecjp
6112df3a90 Bring back the time-out on the start-up of bitcoind, but make it 60 seconds instead of the original 10 seconds. This is much larger than the normal start-up time of bitcoind (so we should never hit the time-out), but still lets a test terminate in a reasonable time in the unlikely case that bitcoind hangs. 2018-02-01 23:00:43 +01:00
cornwarecjp
896f7ef37d Make it possible to run the test suite in Python 3.4 (bytes.hex was introduced in 3.5) 2018-02-01 23:00:43 +01:00
cornwarecjp
0b0708d86a Bitcoind should always eventually finish starting, but there is no guarantee on how long it takes, so don't apply a time-out. In case bitcoind hangs indefinitely, the test will hang indefinitely too, but this should be solved in bitcoind, not in the test code. 2018-02-01 23:00:43 +01:00
Rusty Russell
0d3eaef945 tests: add argument to is_in_log so we can check only recent log entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-31 11:37:22 +01:00
practicalswift
7e8f1945f2 Adjust tests to new log text 2018-01-29 03:22:27 +00:00
practicalswift
689db5b7c1 Onboarding: Make log output texts friendlier to new users 2018-01-29 03:22:27 +00:00
Christian Decker
45c935ddba pytest: Have the db_query helper work on a copy of the DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-27 02:27:35 +00:00
Christian Decker
dafd000c2a pytest: Check that we disable on closing and permfail.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-26 01:09:44 +00:00
Rusty Russell
4f1dc91ba5 lightningd: rename --deprecated-apis to --allow-deprecated-apis.
Suggested-byL practicalswift
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
78672738ff test_lightningd.py: remove deprecated APIs.
Individual tests can always re-enable them, though.

[ More test fallout fixes by Christian Decker ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
962514c3a0 addfunds: remove.
We now detect funds, so this is just confusing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
15aaf9f49a test_lightning.py: shutdown tail thread before killing.
Seems to avoid the nasty python resource warnings, as well as the
fatal 'ValueError: PyMemoryView_FromBuffer(): info->buf must not be NULL'

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-17 23:55:35 +01:00
Rusty Russell
79dc44713b channeld: --ignore-fee-limits as a hack for fee disparities.
This, of course, should never be used.  But it helps maintain connections
for the moment while we dig deeper into feerates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-16 12:22:06 +01:00
Christian Decker
90527498bc pytest: Do not re-print logs if we failed to find a log entry
CI always runs with TEST_DEBUG=1 which prints logs anyway, and testing
locally should also be done this way, combined with pytest which
captures the logs. No need to duplicate the functionality of pytest.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-10 03:52:46 +00:00
Christian Decker
3e0efb6f8d pytest: Avoid leaking stdout fds 2018-01-08 14:36:33 +01:00
practicalswift
6757255ba4 Use deterministic command-line ordering when executing lightningd from tests 2018-01-07 17:28:34 +01:00
Christian Decker
2a7ba010c3 pytest: Use the bitcoind config file to read RPC params from
With python-bitcoinlib==0.9.0 it appears that the URL based auth
information is no longer used, so we fall back to reading the config
file for the bitcoind daemon instead.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-23 06:15:33 +00:00
Rusty Russell
6b9c525f35 lightningd: use env var not cmdline to suppress backtrace.
We now set it up *before* parsing cmdline, so this is more convenient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Christian Decker
e154f4a019 pytest: Minor cleanup
Now using assertRaisesRegex instead of try-except and added restart to
nodes.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Rusty Russell
70cebb8653 test_addfunds_from_block: fix test flakiness.
If you run locally, it fails occasionally; presumably because it
sees previous funds.  Use a random HSM key for that teste.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:52:18 +01:00
Christian Decker
12944ffc9b pytest: Switch to 'Hello world' as startup marker
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-29 14:39:12 +01:00
Rusty Russell
cfde208e82 tests: prepare for bitcoind 0.16.
I noticed some breakage with git master:
1. getinfo no longer supported (for us, use getblockchaininfo)
2. generate no longer supported (use generatetoaddress)

Both these options are supported at least in 0.15, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-28 12:15:22 +01:00
Rusty Russell
88af0f5bf8 tests: change --override-fee-rate to --override-fee-rates and use in tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
c14b159166 lightningd: remove --deadline-blocks option.
We will derive it from other factors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-03 05:18:49 +00:00
Rusty Russell
71e794a046 lightningd: split ctlv_expiry and final_cltv.
These need to be different for testing the example in BOLT 11.

We also use the cltv_final instead of deadline_blocks in the final hop:
various tests assumed 5 was OK, so we tweak utils.py.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell
3c6eec87e3 Add DEVELOPER flag, set by default.
This is a bit messier than I'd like, but we want to clearly remove all
dev code (not just have it uncalled), so we remove fields and functions
altogether rather than stub them out.  This means we put #ifdefs in callers
in some places, but at least it's explicit.

We still run tests, but only a subset, and we run with NO_VALGRIND under
Travis to avoid increasing test times too much.

See-also: #176
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell
b6a2b8c58b Add --rgb and --alias options.
And derive random ones from nodeid if they don't choose.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-25 09:16:14 +00:00
Rusty Russell
c3bed51b2d test_lightningd.py: make HSM seeds constant for tests.
Makes it easier to compare before/after failures.  Ideally, we should
run under Travis both with this option and with the seed based on the
entire tmp path (which is still reproducible with determination, but
not fixed every run like this is).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Christian Decker
f5a412d90d pytest: Change the channel persistence test to add inflight HTLCs
Now that we have HTLC persistence we'd also like to test it. This
kills the second node in the middle of an HTLC, it'll recover and
finish the flow.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Rusty Russell
243fc2c05a utils: save log file on failure.
This broke somewhere in the recent changes, because we override
TailalbleProc stop().  Break out log extractor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-03 12:56:08 +02:00