Commit Graph

83 Commits

Author SHA1 Message Date
Rusty Russell
d817735dc2 lightningd: use ccan/json_out.
This is now a fairly simple transition, which only effects the internals
of json_stream.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell
26cdf9d3dc plugins/pay: don't retry routehint if it contains already-eliminated channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell
260febd88b plugins/pay: fix attempt counter on failure message.
An "attempt" is when we actually try to send, not every route lookup
we do.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Simon Vrouwe
ea8a508a48 pytest: cleanup test_funding_reorg_remote_lags
- mock_rpc function now returns full JSON-RPC response, is much cleaner
- Since reached_announce_depth counting is fixed when starting
  channeld, we don't need the 7th block to tell depth anymore.
2019-06-07 10:34:34 +02:00
Rusty Russell
0543149b89 pytest: move wallet tests into new file tests/test_wallet.py
We're going to add some more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-06 04:47:44 +00:00
Michael Schmoock
c40250ad0b test: adds wait mempool to flaky tests 2019-06-06 00:04:59 +00:00
Rusty Russell
ba036b767f newaddr: fix return value for deprecated "address" field.
It would always return bech32; fix that, and don't bother printing
it if they use the (new) 'all' parameter.

This API was introduced in 3e67c09d5e,
which means it wasn't in a release so no CHANGELOG entry necessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-05 15:38:42 -07:00
Michael Schmoock
107381aca2 test: fix flakyness of funding_reorg_private openchannel_hook 2019-06-03 19:30:19 +02:00
ZmnSCPxj
48df6c8566 lightningd/io_loop_with_timers.c: Move mainloop to its own source file, have chaintopology use it.
Fixes: #2687
2019-05-31 17:57:10 +02:00
Christian Decker
883d4b4e6e pytest: Add a test that reproduces #2687
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-05-31 17:57:10 +02:00
Simon Vrouwe
52c84b2387 pytest: add test_funding_reorg_remote_lags
Nodes may disagree about short_channel_id before channel
announcement.
2019-05-27 13:29:32 +02:00
Simon Vrouwe
5b0c174251 pytest: add test_funding_reorg_private, reorg changes funding height
Reorg changes short_channel_id after lockin of private channel, while
one node restarts.

test that:
- peer->depth_togo in billboard decrements
- reorg and scid change is detected by running node and restarting node
- both `old` and `new` scids are in rtable

Also added a comment to test_blockchaintrack to clarify.
2019-05-27 13:29:32 +02:00
Simon Vrouwe
245807a085 pytest: tighten a log test Owning output... with address type and confirmation
Because the call (wallet_extract_owned outputs) that prints that line can happen
_before_ or _after_ confirmation in block, adding `CONFIRMED` in the later.
2019-05-27 13:29:32 +02:00
Christian Decker
ced2ff8aa7 pytest: Mark repeat offenders as flaky
It's always the same few tests that fail, so I'm marking them as flaky and
will do a cleanup round later to individually address them.

I've been tracking the failures on the master branch for a few months now and
these are the worst offenders:

|-----------------------------------------|--------|--------|-------|
| test_name                               | state  | branch | count |
|-----------------------------------------|--------|--------|-------|
| test_pay_direct                         | FAILED | master | 31    |
| test_reconnect_gossiping                | FAILED | master | 20    |
| test_htlc_send_timeout                  | FAILED | master | 15    |
| test_pay_limits                         | FAILED | master | 13    |
| test_permfail                           | FAILED | master | 13    |
| test_funding_reorg_private              | FAILED | master | 12    |
| test_invoice                            | FAILED | master | 12    |
| test_invoice_preimage                   | FAILED | master | 12    |
| test_shutdown                           | FAILED | master | 12    |
| test_withdraw                           | FAILED | master | 12    |
| test_gossip_store_load_v3               | FAILED | master | 11    |
| test_onchain_multihtlc_their_unilateral | FAILED | master | 10    |
| test_opening_tiny_channel               | FAILED | master | 10    |
| test_channel_reenable                   | FAILED | master | 9     |
| test_crashlog                           | FAILED | master | 9     |
| test_gossip_weirdalias                  | FAILED | master | 9     |
| test_invoice_expiry                     | FAILED | master | 9     |
| test_onchain_multihtlc_our_unilateral   | FAILED | master | 9     |
| test_peerinfo                           | FAILED | master | 9     |
| test_private_channel                    | FAILED | master | 9     |
| test_shutdown_reconnect                 | FAILED | master | 9     |
| test_closing                            | FAILED | master | 8     |
| test_closing_different_fees             | FAILED | master | 8     |
| test_closing_while_disconnected         | FAILED | master | 8     |
| test_gossip_timestamp_filter            | FAILED | master | 8     |
| test_gossipwith                         | FAILED | master | 8     |
| test_invoice_routeboost                 | FAILED | master | 8     |
| test_onchain_middleman                  | FAILED | master | 8     |
| test_plugin_notifications               | FAILED | master | 8     |
| test_reconnect_channel_peers            | FAILED | master | 8     |
| test_bad_onion                          | FAILED | master | 7     |
| test_closing_torture                    | FAILED | master | 7     |
| test_fulfill_incoming_first             | FAILED | master | 7     |
| test_funding_reorg_disagree_scid_route  | FAILED | master | 7     |
| test_pay_disconnect                     | FAILED | master | 7     |
| test_balance                            | FAILED | master | 6     |
| test_check_command                      | FAILED | master | 6     |
| test_closing_id                         | FAILED | master | 6     |
| test_gossip_addresses                   | FAILED | master | 6     |
| test_gossip_notices_close               | FAILED | master | 6     |
| test_setchannelfee_usage                | FAILED | master | 6     |
| test_announce_address                   | FAILED | master | 5     |
| test_connect_by_gossip                  | FAILED | master | 5     |
| test_gossip_badsig                      | FAILED | master | 5     |
| test_gossip_disable_channels            | FAILED | master | 5     |
| test_gossip_jsonrpc                     | FAILED | master | 5     |
| test_gossip_persistence                 | FAILED | master | 5     |
| test_gossip_pruning                     | FAILED | master | 5     |
| test_onchain_different_fees             | FAILED | master | 5     |
| test_onchain_dust_out                   | FAILED | master | 5     |
|-----------------------------------------|--------|--------|-------|
2019-05-26 16:19:10 +02:00
Rusty Russell
3d2af2237c pytest: fix test_htlc_send_timeout now pay doesn't return ROUTE_NOT_FOUND.
This brings up an interesting quirk though, in that we report "3
attempts", where we really should have done one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-22 04:32:34 +00:00
Christian Decker
be853f563a wallet: Clamp maxheight to positive number for large minconf
Fixes #2518

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Changelog-fixed: `minconf` no longer gets wrapped around for large values, which was causing funds with insufficient confirmations to be selected.
2019-04-30 17:52:52 -05:00
Christian Decker
e40b7c5584 pytest: Add test_minconf_withdraw to reproduce issue #2518
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-04-30 17:52:52 -05:00
Rusty Russell
77b859eaec lightning-cli: don't produce bad JSON if fields contain ".
The user can explicitly create such things (within [] or ") as we paste
those cases literally, but not for the simple cases.

Fixes: #2550
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-10 20:03:39 -07:00
Rusty Russell
ede7d0518e connectd: unlink any old socket when we use --bind-addr.
They don't clean up after themselves, so best we do it here (by this
point we've already done the pid check to make sure we're the only
lightningd here anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 22:00:00 +02:00
Rusty Russell
32215177fe pytest: more testing for local socket addresses.
In particular, the assert when `--addr=/sockname` is used, and that it
doesn't clean up on restart, requiring manual deletion of the socket.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 22:00:00 +02:00
Rusty Russell
1069f48082 connectd: fix binding to a UNIX domain socket.
lightning_connectd(19780): STATUS_FAIL_INTERNAL_ERROR: Failed to bind on 2 socket: Address family not supported by protocol

"Untested code is buggy code"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-14 15:05:56 +01:00
Rusty Russell
740f4314ea pytest: test binding lightningd to a UNIX domain socket.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-14 15:05:56 +01:00
Rusty Russell
049f2351d4 newaddr: deprecate 'address' output.
It doesn't make sense with 'all', and it's ok to be explicit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 15:11:20 +01:00
Rusty Russell
3e67c09d5e newaddr: support getting both bech32 and p2sh addresses.
Higher layers consume less addresses this way.

Fixes: #2390
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 15:11:20 +01:00
Christian Decker
72f1c78a1e jsonrpc: Arm the minconf=1 parameter and deal with the fallout
We want to disallow using unconfirmed outputs by default, so making the
default 1 confirmation seems a good idea. This also matches `bitcoind`s
minimum confirmation requirement.

Arming however breaks some of our tests, so I used `minconf=0` for the
breaking tests and added a new test specifically for the `minconf` parameter
for `fundchannel`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 10:40:59 +00:00
Rusty Russell
948ca470ad bitcoin: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
41be796e0e pytest: add simple test for --daemon.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-18 12:01:06 +01:00
Rusty Russell
da355284de jsonrpc: help, even for a single item, should be in an array.
This is what we do for every other can-be-single JSON API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell
3f8dd7a95f plugins/pay: add paystatus command to get gory details of payments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell
9403df8d0d plugins/pay: add shadow CLTV calculation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell
802b772cad plugins/pay: use final_cltc from bolt11 invoice.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell
d9fa8a3536 plugins/pay: retry on failure in a loop.
We use the 'exclude' option to getroute for successive attempts.  This
is more robust than having gossipd disable for some limited time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell
40637d0017 contrib/pylightning: temporarily convert to use plugin/pay for tests.
That this simply pay plugin passes the tests is a poor reflection on our
test cases, really.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-17 13:02:24 +01:00
Rusty Russell
109c6eb3a3 channeld: include proper sha value in BADONION errors.
Fortunately, we can calculate the sha256 ourselves, so the
outgoing channeld doesn't need to tell us.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-08 19:20:28 +01:00
Rusty Russell
8f8783c0e3 pay: correctly blame the *next* node on BADONION error.
The node which sent the error is doing so because the following
one sent WIRE_UPDATE_FAIL_MALFORMED_HTLC.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-08 19:20:28 +01:00
Rusty Russell
5b6c82384a pytest: test that we get a correct result when peer says onion is bad.
Currently fails: we get a garbage error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-08 19:20:28 +01:00
Mark Beckwith
816840e9c4 rpc: check error now consistent with lightning-cli
We now return JSONRPC2_METHOD_NOT_FOUND if the command is not found,
just like lightning-cli does.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-12-10 09:27:49 +01:00
Rusty Russell
0c5f8a3a15 pytest: add test for check command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-12-10 09:27:49 +01:00
Rusty Russell
aee2197a66 jsonrpc: make sure even errors are valid json.
We often quote their msg in our reply; sanitize it!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-06 23:11:51 +01:00
Christian Decker
4d95bb0539 pytest: Mark test_reserve_enforcement as flaky
It has been causing a lot of Travis failures due to a presumed memory
leak:

```
Exception: Node /tmp/ltests-fhjg26a2/test_reserve_enforcement_1/lightning-2/ has memory leaks: [{'label': 'lightningd/json_stream.c:48:struct json_stream', 'backtrace': ['ccan/ccan/tal/tal.c:435 (tal_alloc_)', 'lightningd/json_stream.c:48 (new_json_stream)', 'lightningd/jsonrpc.c:90 (jcon_new_json_stream)', 'lightningd/jsonrpc.c:444 (attach_json_stream)', 'lightningd/jsonrpc.c:455 (json_start)', 'lightningd/jsonrpc.c:464 (json_stream_success)', 'lightningd/pay.c:932 (json_sendpay_on_resolve)', 'lightningd/pay.c:444 (payment_store)', 'lightningd/pay.c:540 (payment_failed)', 'lightningd/peer_htlcs.c:143 (fail_out_htlc)', 'lightningd/peer_htlcs.c:360 (destroy_hout_subd_died)', 'ccan/ccan/tal/tal.c:235 (notify)', 'ccan/ccan/tal/tal.c:395 (del_tree)', 'ccan/ccan/tal/tal.c:405 (del_tree)', 'ccan/ccan/tal/tal.c:405 (del_tree)', 'ccan/ccan/tal/tal.c:479 (tal_free)', 'ccan/ccan/io/io.c:451 (io_close)', 'lightningd/subd.c:500 (sd_msg_read)', 'lightningd/subd.c:302 (read_fds)', 'ccan/ccan/io/io.c:59 (next_plan)', 'ccan/ccan/io/io.c:395 (do_plan)', 'ccan/ccan/io/io.c:405 (io_ready)', 'ccan/ccan/io/poll.c:310 (io_loop)', 'lightningd/lightningd.c:769 (main)'], 'parents': ['lightningd/jsonrpc.c:681:struct json_connection', 'common/configdir.c:29:char[]'], 'value': '0x125be08'}]
```

Signed-off-by: Christian Decker <@cdecker>
2018-11-29 23:01:11 +00:00
Rusty Russell
2c582cbcfb pytest: test that we don't revert config handling again.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-21 01:08:42 +00:00
Rusty Russell
6f7650e471 pytest: test using malformed JSONRPC commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 16:51:19 +01:00
Rusty Russell
7799368144 pytest: add test for multiplexed RPC output.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 16:51:19 +01:00
Rusty Russell
e0d14bddb9 jsonrpc: allow multiple commands at once.
We now keep multiple commands for a json_connection, and an array of
json_streams.

When a command wants to write something, we allocate a new json_stream
at the end of the array.

We always output from the first available json_stream; once that
command has finished, we free that and move to the next.  Once all are
done, we wake the reader.

This means we won't read a new command if output is still pending, but
as most commands don't start writing until they're ready to write
everything, we still get command parallelism.

In particular, you can now 'waitinvoice' and 'delinvoice' and it will
work even though the 'waitinvoice' blocks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-20 16:51:19 +01:00
Rusty Russell
0c3f85d931 lightning.py: parse multiple JSON RPC commands accurately.
We need to keep the remaining buffer, and we need to try to parse it
before we read the next.  I first tried keeping it in the object, but
its lifetime is that of the *socket*, which we actually reopen for
every command.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-19 21:36:40 +01:00
Rusty Russell
fe11ee5406 pytest: make test_multirpc check that RPC commands complete.
This was hanging sometimes in travis, but actually checking the result
of the commands makes it *always* hang.  We remove the waitinvoice
which will not return.

ZmnSCPxj points out that this behavior, introduced in
ce0bd7abd3, is a regression: it would be
nice to be able to cancel a waitinvoice.  But that fix is more complex,
and will have to be another PR.

This test will now hang, but it's OK: we're about to fix it!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-11-19 21:36:40 +01:00
Christian Decker
46b2e7502c bitcoin: If we fail to estimate the fee in testnet use the minfee
When developing in regtest or testnet it is really inconvenient to
have to fake traffic and generate blocks just to get estimatesmartfee
to return a valid estimate. This just sets the minfee if bitcoind
doesn't return a valid estimate.

Reported-by: Rene Pickhardt <@renepickhardt>
Signed-off-by: Christian Decker <@cdecker>
2018-10-29 03:20:08 +00:00
Rusty Russell
7d614aaf25 pytest: really remove all bitcoin generate RPC calls.
generate was deprecated some time ago, so we added the generate_block()
helper.  But many calls crept back in, and git master refuses it.

(test_blockchaintrack relied on the return value, so make generate_block
return the list of blocks).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-28 16:03:12 +01:00
Rusty Russell
38e6aa66ff python: quieten modern flake8.
After Ubuntu 18.10 upgrade, lots of new flake8 warnings.

$ flake8 --version:
3.5.0 (mccabe: 0.6.1, pycodestyle: 2.4.0, pyflakes: 1.6.0) CPython 3.6.7rc1 on Linux

Note it seems that W503 warned about line breaks before binary
operators, and W504 complains about them after.  I prefer W504, so
disable W503.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-28 16:03:12 +01:00
Rusty Russell
ad2519a6f4 spelling: Check LockTime Verify.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-23 16:55:35 +02:00