Commit Graph

106 Commits

Author SHA1 Message Date
Rusty Russell
4e8141aa36 features: remove INITIAL_ROUTING_SYNC.
It only had an effect if the peer didn't support option_gossip_queries, but
still, we don't want a gossip blast any more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-25 04:01:56 +00:00
Christian Decker
efc4aa94a3 pytest: Skip some tests that assume we have a sqlite3 db on postgres
These will not work since they touch the DB file itself.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Christian Decker
96a22b4003 pytest: Add db_provider and db instances for configurable backends
We will soon have a postgres backend as well, so we need a way to control the
postgres process and to provision DBs to the nodes. The two interfaces are the
dsn that we pass to the node, and the python query interface needed to query
from tests.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Rusty Russell
27790832a5 gossipd: gossip_queries_ex is not longer experimental.
The master spec has some typos which make it not parse, so I created
a PR and generated the CSV from that:

https://github.com/lightningnetwork/lightning-rfc/pull/673

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell
147eaced2e developer: consolidiate gossip timing options into one --dev-fast-gossip.
It's generally clearer to have simple hardcoded numbers with an
#if DEVELOPER around it, than apparent variables which aren't, really.

Interestingly, our pruning test was always kinda broken: we have to pass
two cycles, since l2 will refresh the channel once to avoid pruning.

Do the more obvious thing, and cut the network in half and check that
l1 and l3 time out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
lisa neigut
904a1384ef tests: de-flake lightningd_still_loading 2019-09-11 23:56:27 +00:00
Rusty Russell
226e2aee48 option_static_remotekey: update to latest draft.
531c8d7d9b

In this one, we always send my_current_per_commitment_point, though it's
ignored.  And we have our official feature numbers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Christian Decker
245dff26e0 pytest: Updated requirements.txt to latest versions
This is an issue that was raised in #2665: some of the dependencies where
causing warnings to be added to the logs about deprecated dependencies. Since
I did not get these warnings I just blanket updated all the dependencies in
the hopes of getting the warnings to resolve.

Signed-off-by: Christian Decker <@cdecker>
2019-09-02 16:31:36 +02:00
Rusty Russell
6901732ee0 lightningd: create --list-features-only which lists what features we support.
This allows the lightning-rfc protocol tests to automatically query what
features we support.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
6310a183af pytest: ignore occasional bad gossip in reorg tests.
It can happen, as expected when we reorg and update for a now-invalid
channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell
854e3c5303 pytest: wait for sync by default when starting lightningd.
Otherwise we get some spurious test failures.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell
ca28c30eff funding: don't allow funding new channels until we're synced.
This is probably worth preventing.

1. Our depth estimate would be inaccurate possibly leading to us
   timing out too early.
2. If we're not up-to-date our onchain funds are unknown.
3. We wouldn't be able to send or receive HTLCs until we're synced anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell
c3a35416da lightningd: don't allow channeld to accept HTLCs if we're not synced.
We want to still allow incoming connections, and reestablishment of
channels, but if one tries to give us an HTLC, stall until we're
synced.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell
6195a878f7 lightningd: don't allow sending of HTLCs while still syncing.
If we don't know block height, we shouldn't be sending HTLCs.  This
stops us forwarding HTLCs as well as new payments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell
3eebd0cc20 lightningd: add flag for whether we're synced, and callback infrastructure.
We consider ourselves synced when bitcoind is synced and we're synced
with that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell
faded9a9cf bitcoind: detect when it's still syncing, add field to getinfo.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell
118150227e cli: restore 0.7.0-style whitespace printing.
@renepickhardt has a shell script we broke.  While we still produce
perfectly valid JSON, we should not gratuitously change tool output.

Plus, I prefer the missing space before the ':'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-08 18:16:48 +08:00
Rusty Russell
b73a85a75e lightningd: don't say 'killing channel' when HTLC times out.
We're actually only killing the connection.  I saw this in my logs,
but it was all OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-07 21:12:52 +08:00
Rusty Russell
979fbeb3b0 lightningd: simplify --daemon.
Dumb programs which have a --daemon option call fork() early.  This is
terrible UX since startup errors get lost: the program exits with
"success" immediately then you discover via the logs that it didn't
start at all.

However, forking late introduced a heap of problems with changing
pids.  Instead, fork early but keep stderr and the parent around: if
we fail early on, the parent fails with us.  We release our parent
with an explicit action just before the main loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Rusty Russell
fc024f81d6 pytest: check that --daemon still means we recognize our own children.
We create our children then fork, so we're not a parent.  I noticed this
because 'lightning-cli stop' takes a long time: this is because it tries to
wait for them and they don't respond.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Rusty Russell
d59e2b1b4b developer: add --dev-force-bip32-seed to force a specific BIP32 seed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
Rusty Russell
fb6870c139 param: implement helpers for multiplex commands.
Our previous param support was a bit limited in this case.

We create a dev- command multiplexer, so we can exercise it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-16 21:39:14 +00:00
Rusty Russell
54ce4ed1cf pytest: fail tests if we get any LOG_BROKEN level messages, unless flagged.
And clean up some dev ones which actually happen (mainly by calling
channel_fail_permanent which logs UNUSUAL, rather than
channel_internal_error which logs BROKEN).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-02 03:26:10 +00:00
Rusty Russell
d817735dc2 lightningd: use ccan/json_out.
This is now a fairly simple transition, which only effects the internals
of json_stream.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell
26cdf9d3dc plugins/pay: don't retry routehint if it contains already-eliminated channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell
260febd88b plugins/pay: fix attempt counter on failure message.
An "attempt" is when we actually try to send, not every route lookup
we do.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Simon Vrouwe
ea8a508a48 pytest: cleanup test_funding_reorg_remote_lags
- mock_rpc function now returns full JSON-RPC response, is much cleaner
- Since reached_announce_depth counting is fixed when starting
  channeld, we don't need the 7th block to tell depth anymore.
2019-06-07 10:34:34 +02:00
Rusty Russell
0543149b89 pytest: move wallet tests into new file tests/test_wallet.py
We're going to add some more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-06 04:47:44 +00:00
Michael Schmoock
c40250ad0b test: adds wait mempool to flaky tests 2019-06-06 00:04:59 +00:00
Rusty Russell
ba036b767f newaddr: fix return value for deprecated "address" field.
It would always return bech32; fix that, and don't bother printing
it if they use the (new) 'all' parameter.

This API was introduced in 3e67c09d5e,
which means it wasn't in a release so no CHANGELOG entry necessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-05 15:38:42 -07:00
Michael Schmoock
107381aca2 test: fix flakyness of funding_reorg_private openchannel_hook 2019-06-03 19:30:19 +02:00
ZmnSCPxj
48df6c8566 lightningd/io_loop_with_timers.c: Move mainloop to its own source file, have chaintopology use it.
Fixes: #2687
2019-05-31 17:57:10 +02:00
Christian Decker
883d4b4e6e pytest: Add a test that reproduces #2687
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-05-31 17:57:10 +02:00
Simon Vrouwe
52c84b2387 pytest: add test_funding_reorg_remote_lags
Nodes may disagree about short_channel_id before channel
announcement.
2019-05-27 13:29:32 +02:00
Simon Vrouwe
5b0c174251 pytest: add test_funding_reorg_private, reorg changes funding height
Reorg changes short_channel_id after lockin of private channel, while
one node restarts.

test that:
- peer->depth_togo in billboard decrements
- reorg and scid change is detected by running node and restarting node
- both `old` and `new` scids are in rtable

Also added a comment to test_blockchaintrack to clarify.
2019-05-27 13:29:32 +02:00
Simon Vrouwe
245807a085 pytest: tighten a log test Owning output... with address type and confirmation
Because the call (wallet_extract_owned outputs) that prints that line can happen
_before_ or _after_ confirmation in block, adding `CONFIRMED` in the later.
2019-05-27 13:29:32 +02:00
Christian Decker
ced2ff8aa7 pytest: Mark repeat offenders as flaky
It's always the same few tests that fail, so I'm marking them as flaky and
will do a cleanup round later to individually address them.

I've been tracking the failures on the master branch for a few months now and
these are the worst offenders:

|-----------------------------------------|--------|--------|-------|
| test_name                               | state  | branch | count |
|-----------------------------------------|--------|--------|-------|
| test_pay_direct                         | FAILED | master | 31    |
| test_reconnect_gossiping                | FAILED | master | 20    |
| test_htlc_send_timeout                  | FAILED | master | 15    |
| test_pay_limits                         | FAILED | master | 13    |
| test_permfail                           | FAILED | master | 13    |
| test_funding_reorg_private              | FAILED | master | 12    |
| test_invoice                            | FAILED | master | 12    |
| test_invoice_preimage                   | FAILED | master | 12    |
| test_shutdown                           | FAILED | master | 12    |
| test_withdraw                           | FAILED | master | 12    |
| test_gossip_store_load_v3               | FAILED | master | 11    |
| test_onchain_multihtlc_their_unilateral | FAILED | master | 10    |
| test_opening_tiny_channel               | FAILED | master | 10    |
| test_channel_reenable                   | FAILED | master | 9     |
| test_crashlog                           | FAILED | master | 9     |
| test_gossip_weirdalias                  | FAILED | master | 9     |
| test_invoice_expiry                     | FAILED | master | 9     |
| test_onchain_multihtlc_our_unilateral   | FAILED | master | 9     |
| test_peerinfo                           | FAILED | master | 9     |
| test_private_channel                    | FAILED | master | 9     |
| test_shutdown_reconnect                 | FAILED | master | 9     |
| test_closing                            | FAILED | master | 8     |
| test_closing_different_fees             | FAILED | master | 8     |
| test_closing_while_disconnected         | FAILED | master | 8     |
| test_gossip_timestamp_filter            | FAILED | master | 8     |
| test_gossipwith                         | FAILED | master | 8     |
| test_invoice_routeboost                 | FAILED | master | 8     |
| test_onchain_middleman                  | FAILED | master | 8     |
| test_plugin_notifications               | FAILED | master | 8     |
| test_reconnect_channel_peers            | FAILED | master | 8     |
| test_bad_onion                          | FAILED | master | 7     |
| test_closing_torture                    | FAILED | master | 7     |
| test_fulfill_incoming_first             | FAILED | master | 7     |
| test_funding_reorg_disagree_scid_route  | FAILED | master | 7     |
| test_pay_disconnect                     | FAILED | master | 7     |
| test_balance                            | FAILED | master | 6     |
| test_check_command                      | FAILED | master | 6     |
| test_closing_id                         | FAILED | master | 6     |
| test_gossip_addresses                   | FAILED | master | 6     |
| test_gossip_notices_close               | FAILED | master | 6     |
| test_setchannelfee_usage                | FAILED | master | 6     |
| test_announce_address                   | FAILED | master | 5     |
| test_connect_by_gossip                  | FAILED | master | 5     |
| test_gossip_badsig                      | FAILED | master | 5     |
| test_gossip_disable_channels            | FAILED | master | 5     |
| test_gossip_jsonrpc                     | FAILED | master | 5     |
| test_gossip_persistence                 | FAILED | master | 5     |
| test_gossip_pruning                     | FAILED | master | 5     |
| test_onchain_different_fees             | FAILED | master | 5     |
| test_onchain_dust_out                   | FAILED | master | 5     |
|-----------------------------------------|--------|--------|-------|
2019-05-26 16:19:10 +02:00
Rusty Russell
3d2af2237c pytest: fix test_htlc_send_timeout now pay doesn't return ROUTE_NOT_FOUND.
This brings up an interesting quirk though, in that we report "3
attempts", where we really should have done one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-22 04:32:34 +00:00
Christian Decker
be853f563a wallet: Clamp maxheight to positive number for large minconf
Fixes #2518

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Changelog-fixed: `minconf` no longer gets wrapped around for large values, which was causing funds with insufficient confirmations to be selected.
2019-04-30 17:52:52 -05:00
Christian Decker
e40b7c5584 pytest: Add test_minconf_withdraw to reproduce issue #2518
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-04-30 17:52:52 -05:00
Rusty Russell
77b859eaec lightning-cli: don't produce bad JSON if fields contain ".
The user can explicitly create such things (within [] or ") as we paste
those cases literally, but not for the simple cases.

Fixes: #2550
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-10 20:03:39 -07:00
Rusty Russell
ede7d0518e connectd: unlink any old socket when we use --bind-addr.
They don't clean up after themselves, so best we do it here (by this
point we've already done the pid check to make sure we're the only
lightningd here anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 22:00:00 +02:00
Rusty Russell
32215177fe pytest: more testing for local socket addresses.
In particular, the assert when `--addr=/sockname` is used, and that it
doesn't clean up on restart, requiring manual deletion of the socket.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 22:00:00 +02:00
Rusty Russell
1069f48082 connectd: fix binding to a UNIX domain socket.
lightning_connectd(19780): STATUS_FAIL_INTERNAL_ERROR: Failed to bind on 2 socket: Address family not supported by protocol

"Untested code is buggy code"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-14 15:05:56 +01:00
Rusty Russell
740f4314ea pytest: test binding lightningd to a UNIX domain socket.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-14 15:05:56 +01:00
Rusty Russell
049f2351d4 newaddr: deprecate 'address' output.
It doesn't make sense with 'all', and it's ok to be explicit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 15:11:20 +01:00
Rusty Russell
3e67c09d5e newaddr: support getting both bech32 and p2sh addresses.
Higher layers consume less addresses this way.

Fixes: #2390
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 15:11:20 +01:00
Christian Decker
72f1c78a1e jsonrpc: Arm the minconf=1 parameter and deal with the fallout
We want to disallow using unconfirmed outputs by default, so making the
default 1 confirmation seems a good idea. This also matches `bitcoind`s
minimum confirmation requirement.

Arming however breaks some of our tests, so I used `minconf=0` for the
breaking tests and added a new test specifically for the `minconf` parameter
for `fundchannel`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 10:40:59 +00:00
Rusty Russell
948ca470ad bitcoin: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
41be796e0e pytest: add simple test for --daemon.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-18 12:01:06 +01:00