Commit Graph

286 Commits

Author SHA1 Message Date
Rusty Russell
7d614aaf25 pytest: really remove all bitcoin generate RPC calls.
generate was deprecated some time ago, so we added the generate_block()
helper.  But many calls crept back in, and git master refuses it.

(test_blockchaintrack relied on the return value, so make generate_block
return the list of blocks).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-28 16:03:12 +01:00
Rusty Russell
38e6aa66ff python: quieten modern flake8.
After Ubuntu 18.10 upgrade, lots of new flake8 warnings.

$ flake8 --version:
3.5.0 (mccabe: 0.6.1, pycodestyle: 2.4.0, pyflakes: 1.6.0) CPython 3.6.7rc1 on Linux

Note it seems that W503 warned about line breaks before binary
operators, and W504 complains about them after.  I prefer W504, so
disable W503.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-28 16:03:12 +01:00
Rusty Russell
ad2519a6f4 spelling: Check LockTime Verify.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-23 16:55:35 +02:00
Rusty Russell
71a40faae7 withdraw: fix incorrect error when we have an empty wallet.
This also highlights the danger of searching the logs: that error
appeared previously in the logs, so we didn't notice that the actual
withdraw call gave a different error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell
84b9e3e72b lightningd: reduce log spam from bitcoin-cli invocations.
During tests, this is half our log!  And Travis truncates it if we get
a failure in test_restart_many_payments.

Interestingly, test_logging had a bug which relied on this spam :)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-10 06:10:42 +00:00
Christian Decker
d6fcfe00c7 pytest: Stabilize test_feerates 2018-09-28 22:45:30 +00:00
Rusty Russell
59bdba6bac pytest: don't test crashing under valgrind at all.
Travis failures:

valgrind: m_scheduler/sema.c:104 (vgModuleLocal_sema_down): Assertion 'sema->owner_lwpid != lwpid' failed.
host stacktrace:
==1296==    at 0x38083F48: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x38084064: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x380841F1: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x38135DAE: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x380D328D: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x3809A4AC: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x3809AE43: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x380988CF: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
sched status:
  running_tid=0
Thread 1: status = VgTs_WaitSys (lwpid 1296)
==1296==    at 0x5729730: __poll_nocancel (syscall-template.S:84)
==1296==    by 0x4348DF: daemon_poll (daemon.c:78)
==1296==    by 0x4169E7: io_poll_lightningd (lightningd.c:543)
==1296==    by 0x471ECD: io_loop (poll.c:282)
==1296==    by 0x416E06: main (lightningd.c:744)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-26 16:32:08 -07:00
Mark Beckwith
cbde3e20f7 cli: help command now also prints usage
The help command now adds command usage to its output by calling each
command handler in CMD_USAGE mode.

Instead of seeing, for example:

	decodepay
	    Decode {bolt11}, using {description} if necessary

we see:

	decodepay bolt11 [description]
	    Decode {bolt11}, using {description} if necessary

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-25 15:11:45 +02:00
Rusty Russell
252bbe1d2d pytest: don't wait for sendrawtx, wait for expected tx.
In particular, test_no_fee_estimate was flaky due to seeing the funding
tx being sent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-19 13:04:01 +02:00
Christian Decker
9e5d7dacb0 pytest: Use the mock bitcoind everywhere 2018-09-16 00:05:34 +02:00
Christian Decker
16869e3fe6 pytest: Use the bitcoind proxy to mock feerates 2018-09-16 00:05:34 +02:00
Christian Decker
e132dffa0b pytest: Add an RPC proxy inbetween bitcoind and bitcoin-cli
This is a simple reverse proxy that `bitcoin-cli` can talk to when invoked by
`lightningd`. It allows us to trace `bitcoin-cli` calls, and intercept calls to
mock the replies, better than the current bash-script based method.
2018-09-16 00:05:34 +02:00
Christian Decker
f1e931f7bb pytest: Fix flaky test_logging
File was rotated away but didn't wait for the first line to be actually written.
2018-09-14 21:19:50 +02:00
Rusty Russell
33c6285787 feerates: turn it into a simple query API, remove setting.
It's probably unnecessary to have this weird way of injecting results
now we have explicit feerate args.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell
e2d4b7cc8d cleanup: extract and formalize feerate conversion.
I didn't want to create a new file for this now, as that would totally
break #1880.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell
af4fa9a359 feerates: rename sipa/bitcoind to perkw/perkb.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-25 00:33:12 +00:00
Rusty Russell
a4b952ebc7 feerate: include rough estimates of actual tx costs.
We could refine this later (based on existing wallet, for example), but
this gives some estimate.

[ Rename onchain_estimates -> onchain_fee_estimates Suggested-by: @SimonVrouwe ]
[ Factor of 1000 fix Reported-by: @SimonVrouwe ]
Suggested-by: @molxyz
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-25 00:33:12 +00:00
Rusty Russell
14294642d2 feerates: consider last three raw values for min/max.
We don't know what our peer is doing, but if we see those values, maybe
they did too, and for longer.  And add the min/max acceptable values
into our JSON API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-25 00:33:12 +00:00
Rusty Russell
c7c5affa3f feerates: new command to inject/query fee estimates.
This is useful mainly in the case where bitcoind is not giving estimates,
but can also be used to bias results if you want.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-25 00:33:12 +00:00
Rusty Russell
7925469f88 pytest: fix flaky assert in test_htlc_send_timeout.
If feerates change, L2 sends L3 a commit for that, which causes us to
fail the assert (which says we won't send a commitment_signed).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 02:17:51 +00:00
Rusty Russell
607d4bf9d2 channel: update fees after lockin.
We don't respond to fee changes until we're locked in: make sure we catch
up at that point.

Note that we use NORMAL fees during opening, but IMMEDIATE after, so
this often sends a fee update.  The tests which break, we set those
feerates to be equal.

This (sometimes) changes the behavior of test_permfail, as we now
get an immediate commit, so that is fixed too so we always wait for
that to complete.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 02:17:51 +00:00
Christian Decker
8f56d64a1f log: Append the current time to the crash log filename
This should make it easier to identify the latest crash file and correlate
crashes with external monitoring tools.
2018-08-23 12:51:08 +02:00
Rusty Russell
00696277d2 logging: always dump a crash log, but make files per-pid.
Someone had a 21GB crash.log, which doesn't help anyone!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 12:51:08 +02:00
Rusty Russell
213be90e77 log: implement reopening log-file on SIGHUP
Closes: #1623
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 12:51:08 +02:00
Rusty Russell
7a77b81dff pytest: always use the fake-bitcoin-cli, name it more appropriately.
We're going to use it to override specific commands.  It's non-valgrinded
already since we use '--trace-children-skip=*bitcoin-cli*' so the overhead
should be minimal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-22 12:13:23 +02:00
Rusty Russell
5e20eedb41 pytest: allow more elaborate bitcoin-cli override.
In particular, this lets us intercept individual commands, such as
estimatesmartfee.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-22 12:13:23 +02:00
Rusty Russell
3454ca57a2 pytest: remove DEVELOPER guard for test_ping
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-15 06:48:55 +00:00
Rusty Russell
9d8b3a070b pytest: make test_htlc_send_timeout more reliable.
Waiting for three node_announcments isn't always enough, since l2 can
publish two of them (an independent bug).  Do the more Right Thing and
just wait for 30 seconds of no input...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell
65c882ca3a Minor cleanups.
1. connect convenience variable for improved readabilty.
2. a comment explaining that timer is on channel, not HTLC.
3. use modern python style in test_htlc_send_timeout

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell
63e4ea17af channeld: don't commit until we've seen recent incoming msg, ping if required.
Now sending a ping makes sense: it should force the other end to send
a reply, unblocking the commitment process.

Note that rather than waiting for a reply, we're actually spinning on
a 100ms loop in this case.  But it's simple and it works.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell
ebd229fb37 pytest: add test for HTLC timeout when a node is unreachable (xfail).
Currently, if we don't realize a TCP connection is down, we almost
certainly don't find out until *after* we're sent the
commitment_signed message, in which case we cannot fail the incoming
HTLC.

This test demonstrates that.  Note the 30 second sleep: we should really
run Travis tests in parallel!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell
71575b2115 ping: no longer a dev_ command.
Fixes: #1407
Suggested-by: conanoc@gmail.com
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell
329270525c pytest: only use dev-allow-localhost when needed.
The next patches get better at reconecting, so if we use dev-allow-localhost
nodes can often find each other and reconnect before shutting down; only
use that option where we actually need it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell
d8a6028214 connectd: fix binding to same port on IPv4 and IPv6.
1. If the IPv6 address was public, that changed the wireaddr and thus the ipv4 bind
   would not be to a wildcard and would fail.
2. Binding two fds to the same port on both wildcard IPv4 and IPv6 succeeds; we only
   fail when we try to listen, so allow error at this point.

For some reason this triggered on my digital ocean machine.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-08 15:04:06 +02:00
Christian Decker
2a14a98ead pytest: Migrate test_withdraw to use LightningNode.db_query
This was causing lock issues in some cases.
2018-08-07 00:54:19 +00:00
Christian Decker
d3731b08b1 pytest: Migrate all remaining tests into test_misc.py 2018-08-07 00:54:19 +00:00