Commit Graph

2038 Commits

Author SHA1 Message Date
Christian Decker
c6f210a46c htlcs: Needed to add origin_htlc_id to outgoing HTLCs
While loading HTLCs from the database we might not yet have all the
incoming HTLCs loaded when loading a dependent htlc_out. So we defer
the wiring of the HTLCs until we are sure we have them loaded.

This is also the first step towards keeping that association only in
the database, since otherwise we cannot selectively load channels from
DB.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
eb5ed961ed wallet: Added loading of HTLCs from the database
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
53763ba6a4 wallet: Store HTLCs to the wallet
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
b8fa765c3f wallet: Add primitives to store htlcs in the database
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
3dda72c885 db: Create table for HTLCs
Also added a small warning to one of the used enums not to reorder or
insert values. They'd break the update path.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
a4967d74f1 pytest: Added a simple breakpoint shorthand
I was typing this over and over again, adding this shorthand mainly to
remember it :-)

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
cf15670c3f pytest: Disabling test_penalty_outhtlc since it is too flaky
The test fails very often and causes the CI build to die. Needs to be
fixed.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
249083bcc1 options: Do not schedule a reconnect attempt with --no-reconnect
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:25:08 +10:30
Christian Decker
359c83b855 options: Move non-legacy options into options.c
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:25:08 +10:30
Christian Decker
48796f4f39 cli: Add --no-reconnect cli flag
Especially when testing we might want to disable the automatic
reconnection logic in order not to masquerade bugs that disappear when
reconnecting.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:25:08 +10:30
Rusty Russell
243fc2c05a utils: save log file on failure.
This broke somewhere in the recent changes, because we override
TailalbleProc stop().  Break out log extractor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-03 12:56:08 +02:00
Christian Decker
f35e29607d wire: Fix include order in wire_io.h
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 14:40:34 +02:00
Rusty Russell
29b83aed2a wire: restore BE endian to wire headers for internal messages.
We don't anticipate daemons across machines, but you never know.

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell
3d316518fd wire: use 26-bit lengths for inter-daemon messaging.
Fixes: #289
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell
32631b4278 generate-wire.py: add --bolt arg, use size->type hacks only when that's specified.
For our own internal comms CSVs, we should always name explicit types.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell
1b31d1c5a0 bitcoind: handle up to 60 seconds of outage.
Seems to go out to lunch on reorgs:

   +136792.168286138 lightningd(9465):BROKEN: bitcoin-cli getchaintips exited 28: 'error code: -28
   error message:
   Rewinding blocks...

Closes: #286
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:39:35 +02:00
Rusty Russell
a2d4e09b80 bitcoind: wait for startup.
We don't hit this in testing, since we wait for startup already.  Hacking
tests to avoid that, I tested this code by hand.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:39:35 +02:00
Rusty Russell
66c1da795f lightningd: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
8bb20d127d channeld: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
c850e3c0a4 gossipd: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
1398a208f8 hsmd: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
3404c393b7 common/io_debug: replacement for ccan/io's poll which does sanity checks.
For now we just check for outstanding take() or tal_tmpctx().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
e587ec3bd3 tal_tmpctx: keep information around so we can find leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
7200002773 tests: detect and close leaks in unit tests.
Fixes: #288
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Christian Decker
5029f95066 pytest: Use futures to wait for failing sendpays
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Christian Decker
0b350d2f5f pytest: Integrate with known/allowed failures
Moved the flagging for allowed failures into the factory getter, and
renamed into `may_fail`. Also stopped the teardown of a node from
throwing an exception if we are allowed to exit non-cleanly.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Christian Decker
c1f4c86589 pytest: Make sure to clean up all lightningds after failures
A failed returncode check could result in the cleanup for other
lightningds to be skipped. Now make sure to cleanup all and then
rethrow an exception that contains all returncodes.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Christian Decker
11eaabdbe6 pytest: Stopping daemon cleanly
We used to simply kill the daemon, which in some cases could result in
half-written crashlogs and similar artifacts such as half-completed
RPC calls. Now we ask lightningd to stop nicely, give it some time and
only then kill it. We also return the returncode of the daemon.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Rusty Russell
3f9ec6c2fa jsonrpc: use-after-free bug due to unspecified free behavior 2/2
Using pc after free in the pay_command_destroyed destructor, so
we just steal cmd onto pc so free order is the one we want.

[ Edit: expanded comment, split commit ]
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Christian Decker
eac770760c jsonrpc: Make sure we handle the case where jcon outlives command
So far only happens during normal shutdown, but it may happen in other
cases as well. We simply define a new destructor that unregisters the
`cmd` from the `jcon`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Rusty Russell
3564263e12 jsonrpc: use-after-free bug due to unspecified free behavior 1/2
These were fun to hunt down. The jcon and the conn are allocated off
of ld, so the free order is unspecified and if conn is freed before
conn then the finish_jcon destructor uses conn after free.

[ Edit: split commit, modified to use a destructor directly on jcon,
  which is more robust than relying on it only being freed via conn --RR ]
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Rusty Russell
4275ca5da3 lightningd: don't free log on shutdown until after lightningd freed.
Based-on-patch-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:06:56 +09:30
Rusty Russell
2fe2a0bcf9 peer_control: don't double-free on permanent fail of non-persistent peer.
peer_fail_permanent() frees peer->owner, but for bad_peer() we're
being called by the sd->badpeercb(), which then goes on to
io_close(conn) which is a child of sd.

We need to detach the two for this case, so neither tries to free the
other.

This leads to a corner case when the subd exits after the peer is gone:
subd->peer is NULL, so we have to handle that too.

Fixes: #282
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 15:37:43 +02:00
Rusty Russell
b7bb0be944 subd: remove context arg, as we're always owned by lightningd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 15:37:43 +02:00
Rusty Russell
8741043a0e test_lightningd: rename has_failed to allow_failure.
Note that it should really be a flag to daemon on construction, too,
but that may interfere with another concurrent branch so I've deferred.

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
f3025ce147 onchaind: simplify reasonable-depth code a little.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
f082c7b80e lightningd: add FIXMEs for future work.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
96a85a786c opening: save the correct commitment tx when we are the fundee.
We were saving *our* commitment tx, not theirs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
d04b7af56e onchain: add initial onchain state.
We have a race where we start onchaind, but state is unchanged, so checks
like peer_control.c's:

	peer_ready = (peer->owner && peer->state == CHANNELD_AWAITING_LOCKIN);
	if (!peer_ready) {
		log_unusual(peer->log,
			    "Funding tx confirmed, but peer state %s %s",
			    peer_state_name(peer->state),
			    peer->owner ? peer->owner->name : "unowned");
	} else {
		subd_send_msg(peer->owner,
			      take(towire_channel_funding_locked(peer,
								 peer->scid)));
	}

Can send to the wrong daemon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
72b215f6fe Make all internal message numbers unique.
We were sending a channeld message to onchaind, which was v. confusing
due to overlap.  We make all the numbers distinct, which means we can
also add an assert() that it's valid for that daemon, which catches
such errors immediately.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
a8c60ed289 towire: remove useless double-invert in towire_bool.
GCC optimizes it out anyway: I sent an uninitialized var and it sent 8!
The receiver checks the value is 0 or 1 anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
aea77653d3 onchaind: update bolt #5, and implement failure of timed-out onchain HTLCs.
We re-use the value for reasonable_depth given by the master, and we
tell it when our timeout transactions reach that depth.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
5666ba551f onchaind: fail htlcs which are not included in commitment tx.
As per update 149cf020d6

Fixes: #249
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
4ce9f73e00 BOLT update to fae35903ae600d61124a3920363094b02d2ae8e5
This details handling of onchain HTLCs, and we document how we
implement it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
1ac9e0b532 onchain: implement penalty transaction.
Fixes: #242
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
d640dc3a6f test_lightningd.py: fix wait_for_logs with duplicate entries.
In the next test, we wait for multiple 'sendrawtx exit 0' which
doesn't work because we use a set not a list, and the current code
would match multiple against the same thing.  The result was we didn't
wait for the final sendrawtransaction, and occasionally had test
failures as a result.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
0e569209ec onchaind: immediately broadcast proposals which don't need to wait.
This removes an extra call and simplifies handling of "immediate" txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
ab8251c214 lightningd: dev-reenable-commit RPC command to re-enable commit timer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
ce160d9b17 lightnind: _ dev-disconnect argument to suppress commit timer.
Required for catching daemon in exact state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
5957dd9303 lightnind: dev-sign-last-tx RPC command for testing penalty txs.
We don't normally let daemons send invalid txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30