Commit Graph

2061 Commits

Author SHA1 Message Date
Christian Decker
cfac9a1f73 htlcs: Remove in-memory htlc_stubs
We pull them from the database on-demand, where we're storing them
anyway. No need to keep them in memory as well.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-10 23:59:34 +00:00
Christian Decker
896a67d1d6 peer: Pass in the htlc_stub directly to tell_if_missing
No idea why we were iterating over the list of stubs and then passing
in the index instead of a pointer to the stub directly.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-10 23:59:34 +00:00
Christian Decker
a668cb4f2b onchain: Load HTLC stubs upon starting onchaind
This wires in the loading of `struct htlc_stub`s on-demand when
starting `onchaind` so that we don't need to keep them in memory.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-10 23:59:34 +00:00
Christian Decker
49ed7c9ab0 wallet: Add primitives to retrieve htlc_stubs for channel
We'd like to not keep them in memory and retrieve them on-demand when
`onchaind` is launched. This uses the `channel_htlcs` table as backing
but only fetches the minimal necessary information.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-10 23:59:34 +00:00
Christian Decker
98f63856f7 wallet: Remove FIXME
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-10 23:59:34 +00:00
Rusty Russell
297e278132 gossip: send the *other* node's cltv_expiry_delta in channel_announce.
Include tests from example doc.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
Rusty Russell
cd8edd1716 test_lightningd: allow multiple extra options to lightningd
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
Rusty Russell
18e3f9820f Send/receive cltv_expiry_delta in open/accept channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
Rusty Russell
2a28173891 Typo fix: CTLV -> CLTV.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
Rusty Russell
e137e2527f Update BOLT references with typo fixes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
William Casarin
52f741de97 tools/generate-wire.py: generalize python3 path
Generalize generate-wire.py python3 path

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:12:37 +02:00
Christian Decker
9a106bf799 wallet: Add invoice removal support
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
f5a412d90d pytest: Change the channel persistence test to add inflight HTLCs
Now that we have HTLC persistence we'd also like to test it. This
kills the second node in the middle of an HTLC, it'll recover and
finish the flow.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
11903aed6c wallet: Wiring in invoice persistence into JSON-RPC and master
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
a005bce155 wallet: Adding primitives to save/load invoices to wallet
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
048680d0f2 db: Add db_prepare and db_exec_prepared to expose native binding
This makes executing a query/command a two step process, but allows us
to use the native binding and avoid having to build queries as SQL
strings. Two major advantages are that we are no longer vulnerable to
SQL injections and that we do not have to hex-encode binary fields
like private keys, hashes, and routing onions, halving the storage
requirements for those.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
067d2d7435 db: Added table for invoices
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
5a2242cde7 invoice: Refactor to make the invoice status explicit
So far we were tracking the status by including it either in the paid
or the unpaid list. This refactor makes the state explicit, which
matches the planned DB schema much better.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
c1493ae60c lightningd: Added loading of HTLCs upon startup
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
b4732f6091 wallet_tests: Testing the rewiring of loaded htlcs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
4114f6f79b db: Clear errors before executing new query
This was causing me some trouble by making it look like the last query
failed, when it really was an old one. No need to drag failures around
for longer than needed.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
e162a53e86 wallet_tests: Enable logging in the unit tests
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
df9b8e22b4 wallet: Added helper to rewire HTLCs after loading from DB
This is a necessary evil since at the time we load `struct htlc_out`
associated with a channel we might not have loaded the `struct
htlc_in` that it depends on, so we defer the rewiring until we have
loaded all HTLCs for all channels. At that point rewiring MUST work,
otherwise we report a failure.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
c6f210a46c htlcs: Needed to add origin_htlc_id to outgoing HTLCs
While loading HTLCs from the database we might not yet have all the
incoming HTLCs loaded when loading a dependent htlc_out. So we defer
the wiring of the HTLCs until we are sure we have them loaded.

This is also the first step towards keeping that association only in
the database, since otherwise we cannot selectively load channels from
DB.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
eb5ed961ed wallet: Added loading of HTLCs from the database
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
53763ba6a4 wallet: Store HTLCs to the wallet
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
b8fa765c3f wallet: Add primitives to store htlcs in the database
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
3dda72c885 db: Create table for HTLCs
Also added a small warning to one of the used enums not to reorder or
insert values. They'd break the update path.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
a4967d74f1 pytest: Added a simple breakpoint shorthand
I was typing this over and over again, adding this shorthand mainly to
remember it :-)

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
cf15670c3f pytest: Disabling test_penalty_outhtlc since it is too flaky
The test fails very often and causes the CI build to die. Needs to be
fixed.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Christian Decker
249083bcc1 options: Do not schedule a reconnect attempt with --no-reconnect
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:25:08 +10:30
Christian Decker
359c83b855 options: Move non-legacy options into options.c
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:25:08 +10:30
Christian Decker
48796f4f39 cli: Add --no-reconnect cli flag
Especially when testing we might want to disable the automatic
reconnection logic in order not to masquerade bugs that disappear when
reconnecting.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:25:08 +10:30
Rusty Russell
243fc2c05a utils: save log file on failure.
This broke somewhere in the recent changes, because we override
TailalbleProc stop().  Break out log extractor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-03 12:56:08 +02:00
Christian Decker
f35e29607d wire: Fix include order in wire_io.h
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 14:40:34 +02:00
Rusty Russell
29b83aed2a wire: restore BE endian to wire headers for internal messages.
We don't anticipate daemons across machines, but you never know.

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell
3d316518fd wire: use 26-bit lengths for inter-daemon messaging.
Fixes: #289
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell
32631b4278 generate-wire.py: add --bolt arg, use size->type hacks only when that's specified.
For our own internal comms CSVs, we should always name explicit types.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell
1b31d1c5a0 bitcoind: handle up to 60 seconds of outage.
Seems to go out to lunch on reorgs:

   +136792.168286138 lightningd(9465):BROKEN: bitcoin-cli getchaintips exited 28: 'error code: -28
   error message:
   Rewinding blocks...

Closes: #286
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:39:35 +02:00
Rusty Russell
a2d4e09b80 bitcoind: wait for startup.
We don't hit this in testing, since we wait for startup already.  Hacking
tests to avoid that, I tested this code by hand.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:39:35 +02:00
Rusty Russell
66c1da795f lightningd: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
8bb20d127d channeld: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
c850e3c0a4 gossipd: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
1398a208f8 hsmd: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
3404c393b7 common/io_debug: replacement for ccan/io's poll which does sanity checks.
For now we just check for outstanding take() or tal_tmpctx().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
e587ec3bd3 tal_tmpctx: keep information around so we can find leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
7200002773 tests: detect and close leaks in unit tests.
Fixes: #288
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Christian Decker
5029f95066 pytest: Use futures to wait for failing sendpays
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Christian Decker
0b350d2f5f pytest: Integrate with known/allowed failures
Moved the flagging for allowed failures into the factory getter, and
renamed into `may_fail`. Also stopped the teardown of a node from
throwing an exception if we are allowed to exit non-cleanly.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30
Christian Decker
c1f4c86589 pytest: Make sure to clean up all lightningds after failures
A failed returncode check could result in the cleanup for other
lightningds to be skipped. Now make sure to cleanup all and then
rethrow an exception that contains all returncodes.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 10:06:56 +09:30