Commit Graph

90 Commits

Author SHA1 Message Date
Christian Decker
cf15670c3f pytest: Disabling test_penalty_outhtlc since it is too flaky
The test fails very often and causes the CI build to die. Needs to be
fixed.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
5029f95066 pytest: Use futures to wait for failing sendpays
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Christian Decker
0b350d2f5f pytest: Integrate with known/allowed failures
Moved the flagging for allowed failures into the factory getter, and
renamed into `may_fail`. Also stopped the teardown of a node from
throwing an exception if we are allowed to exit non-cleanly.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Christian Decker
c1f4c86589 pytest: Make sure to clean up all lightningds after failures
A failed returncode check could result in the cleanup for other
lightningds to be skipped. Now make sure to cleanup all and then
rethrow an exception that contains all returncodes.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Christian Decker
11eaabdbe6 pytest: Stopping daemon cleanly
We used to simply kill the daemon, which in some cases could result in
half-written crashlogs and similar artifacts such as half-completed
RPC calls. Now we ask lightningd to stop nicely, give it some time and
only then kill it. We also return the returncode of the daemon.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Rusty Russell
2fe2a0bcf9 peer_control: don't double-free on permanent fail of non-persistent peer.
peer_fail_permanent() frees peer->owner, but for bad_peer() we're
being called by the sd->badpeercb(), which then goes on to
io_close(conn) which is a child of sd.

We need to detach the two for this case, so neither tries to free the
other.

This leads to a corner case when the subd exits after the peer is gone:
subd->peer is NULL, so we have to handle that too.

Fixes: #282
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 15:37:43 +02:00
Rusty Russell
8741043a0e test_lightningd: rename has_failed to allow_failure.
Note that it should really be a flag to daemon on construction, too,
but that may interfere with another concurrent branch so I've deferred.

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
96a85a786c opening: save the correct commitment tx when we are the fundee.
We were saving *our* commitment tx, not theirs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
aea77653d3 onchaind: update bolt #5, and implement failure of timed-out onchain HTLCs.
We re-use the value for reasonable_depth given by the master, and we
tell it when our timeout transactions reach that depth.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
5666ba551f onchaind: fail htlcs which are not included in commitment tx.
As per update 149cf020d6

Fixes: #249
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
1ac9e0b532 onchain: implement penalty transaction.
Fixes: #242
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
cbe72b658e onchaind: extract payment_preimage from onchain HTLC redemption.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
f96a04e2a6 onchaind: resolve HTLC transactions.
When we sent out an HTLC-Timeout or HTLC-Success tx, we need to spend
it after the timeout so it's safely in our wallet.

We generalize the tx_type OUR_UNILATERAL_TO_US_RETURN_TO_WALLET to
OUR_DELAYED_RETURN_TO_WALLET, since we use it for HTLC transactions too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
a76a53a933 onchaind: fulfill HTLCs onchain.
When we see an offered HTLC onchain, we need to use the preimage if we
know it.  So we dump all the known HTLC preimages at startup, and send
new ones as we discover them.

This doesn't cover preimages we know because we're the final
recipient; that can happen if an HTLC hasn't been irrevocably
committed yet.  We'll do that in a followup patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Christian Decker
081413e247 pytest: Added test to check node_announcement signatures
Reported-by: Rusty Russell
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-16 09:02:06 +09:30
Rusty Russell
584b160cdc lightningd: avoid errors on shutdown.
lightningd can crash on shutdown if it's in the middle of getchaintips;
we free the conn, the finished callback is called (process_chaintips),
and it reports that it received an empty result.

The simplest fix is to set a flag in the struct bitcoind destructor,
and avoid the callback.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 23:00:53 +02:00
Rusty Russell
ec63c0d10b lightningd: give option to crash if a subdaemon fails.
Either when it exits with a signal, or sends an error status message.
Then we make test_lightningd.py use it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 23:00:53 +02:00
Rusty Russell
64a26b06e7 tests/test_lightningd.py: fail on crash.log.
We simply kill lightningd; we should stop it properly and have a timeout
to kill it if that fails.  However, that's beyond my python skills :(

So we just look for crash.log.  Unfortunately, we usually kill
lightningd before it's finished writing it.  So we look for it and
don't kill lightningd, just wait in this case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 23:00:53 +02:00
Rusty Russell
f219955dc6 wallet: fix spending of change.
We recorded our own change as P2SH; it's not.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 22:59:24 +02:00
Christian Decker
fa13190595 wallet: Hook into the hsm_funding_sig to extract change outputs
This is the step where we broadcast the transaction to the network and
a nice place to extract the change from the transaction.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-10 10:48:53 +09:30
Christian Decker
f2d67f88c0 pytest: Run sendpay in a future so it doesn't complain about failing
For the permfail tests the sendpay call is supposed to fail, so this
was printing stacktraces upon success. Running in futures captures any
thrown exceptions and rethrows them when calling `result()`, in our
case we just ignore them.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-08 17:02:04 +09:30
Christian Decker
641a3b64cf channel: Use min_cltv_expiry in the channel announcements
So far we were always using the deadline in the announcements, that's
obviously not good, so this introduces the parameter as per spec.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-08 17:02:04 +09:30
Rusty Russell
b19a4516d4 lightningd: fix crash when old openingd still around.
We weren't killing it.  Eventually it would die, and peer_owner_finished()
would access subd->peer->owner, but that peer was freed already.

Closes: #261
Reported-by: Christian Decker <decker.christian@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-06 19:11:21 +02:00
Rusty Russell
5a5e23c011 test_lightning.py: use NO_VALGRIND instead of NOVALGRIND.
This variable also turns of valgrind on the unit tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell
1cf33eefe2 lightningd: handle case where channeld fails locally-generated HTLC.
jl777 reported a crash when we try to pay past reserve.  Fix that (and
a whole class of related bugs) and add tests.

In test_lightning.py I had to make non-async path for sendpay() non-threaded
to get the exception passed through for testing.

Closes: #236
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-30 11:36:37 +02:00
Rusty Russell
bbed5e3411 Rename subdaemons, move them into top level.
We leave the *build* results in lightningd/ for ease of in-place testing though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
88bb38f63b daemon/lightningd: remove building, and main files
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Christian Decker
4b64b7f2aa lightningd: Fix channel-persistence for channels with commits
I was hoping to defer HTLC updates until we actually store HTLCs, but
we need to flush to DB whenever balances update as well.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-08-26 16:37:53 +09:30
Rusty Russell
0a88948849 test_lightningd: explicitly test the new commit point case.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-23 17:38:22 +02:00
Rusty Russell
e7633613a9 test_lightningd: onchain tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-23 17:38:22 +02:00
Christian Decker
e51d261f51 lightningd: Load persisted channels on startup
This is the big one, and it's completely anticlimactic: it loads all
channels that have reached opening and are not marked as
closingd_complete into memory, that's it.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-08-23 10:23:54 +09:30
Christian Decker
29785d4990 pytest: Added a simple channel-persistence test
This test opens a channel, stops the nodes and the restarts them to
see if we can successfully reload the channel state from the database.
2017-08-23 10:23:54 +09:30
Rusty Russell
fc24c71196 tests/test_lightningd.py: add async option for pay command.
We'll need this for testing nodes going down during payment.

However, there's no good way to silence the threads that I can tell,
so we get a nasty backtrace from it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell
8e0c19c76a lightningd: add dev-fail command to inject permenant failure.
A couple of double-free bugs founnd doing this, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell
1ad960413d test_lightningd: fix valgrind detection on children.
I tracked down a bug, and couldn't figure out why valgrind wasn't
finding it.

From man valgrind:

       --log-file=<filename>
           Specifies that Valgrind should send all of its messages to the
           specified file. If the file name is empty, it causes an abort.
           There are three special format specifiers that can be used in the
           file name.

           %p is replaced with the current process ID. This is very useful for
           program that invoke multiple processes. WARNING: If you use
           --trace-children=yes and your program invokes multiple processes OR
           your program forks without calling exec afterwards, and you don't
           use this specifier (or the %q specifier below), the Valgrind output
           from all those processes will go into one file, possibly jumbled
           up, and possibly incomplete.

"possibly incomplete" indeed!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Christian Decker
9ad8365589 pytest: Split test_disconnect
These were in part reusing some of the nodes, and creating new
instances for other, so I split them to avoid having cross pollution
between the tests.
2017-08-17 10:31:55 +09:30
Christian Decker
943b6526c7 pytest: Configure non-legacy tests correctly and filter legacy 2017-07-12 11:30:23 +09:30
Christian Decker
8e043fc448 pytest: Consolidating sync_blockheight and fixing flaky test 2017-07-12 11:00:26 +09:30
Christian Decker
1a1e29a4bc gossip: Re-initiate the broadcast timer upon reconnect
We weren't registering reconnecting peers for broadcasts. Just
starting a timer is enough. Also added an integration test to check
that the gossip sync is being resumed.
2017-07-12 11:00:26 +09:30
Rusty Russell
f04222cb12 test_lightningd.py: make pay helper use random label if we don't care.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell
87232ec25c lightningd: new peer state CLOSINGD_COMPLETE.
This is a transitional state, while we're waiting to see the
closing tx onchain (which is To Be Implemented).

The simplest way to do re-transmission is to re-use closingd, and just
disallow any updates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell
87f6a57d75 tests/test_lightningd.py: test closing and shutdown reconnection.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell
a827d2b2bb lightningd: add in support for closingd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell
d40334ce18 tests/test_lightningd.py: add pay helper for making payments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell
6fa90c926a channeld: exit after shutdown when no more HTLCs.
Ready for the introduction of closingd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell
4a3c309e84 tests/test_lightningd.py: fix assumption about gossip order
Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-03 19:43:34 +09:30
Rusty Russell
c05f63a4db peer_control: fix double-state transition on reconnect.
We actually don't need to transition if we're reconnecting, and logic
to go to CHANNELD_NORMAL was wrong: we checked that we'd seen funding tx
locked, but not that we'd received a msg from the remote peer.

We need to fix the tests now we no longer double-transition, too.

Fixes: #188
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-03 19:43:34 +09:30
Rusty Russell
5edc14ee1a tests/test_lightningd.py: make test less flaky.
When we drop an HTLC_ADD packet, sometimes the commit timer fires
before we try to read from the fd.  In this case, the payment is
considered committed and we don't fail.

We need manual commit to work around this, and also we'd need to do
the pay command asynchronously from python because it will block.
That's a bit out of scope for now, so just handle either way.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell
866ac79850 channeld: shutdown support.
We keep the scriptpubkey to send until after a commitment_signed (or,
in the corner case, if there's no pending commitment).  When we
receive a shutdown from the peer, we pass it up to the master.

It's up to the master not to add any more HTLCs, which works because
we move from CHANNELD_NORMAL to CHANNELD_SHUTTING_DOWN.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Christian Decker
462d936f25 wallet: Add unit test for withdrawal 2017-06-23 16:02:20 +09:30