Commit Graph

165 Commits

Author SHA1 Message Date
ZmnSCPxj
693c718160 wallet: Add DB fields for recording routing failures. 2018-03-14 05:33:09 +00:00
Christian Decker
387ba76920 wallet: Add method to retrieve outpoints from the utxoset
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00
Rusty Russell
6c1233de44 channel: reserve a bip32 index as soon as channel is opened.
This simplifies things, and means it's always in the database.  Our
previous approach to creating it on the fly had holes when it was
created for onchaind, causing us to use another every time we
restarted.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell
242a934b8c wallet: add check-source to makefile.
And fix up resulting breakage.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell
9d9c1c3db6 wallet: wrap missing last_processed_block handling in COMPAT_V052.
Our testing also reveals a bug: we start lightningd and shut it down
before fully processing the blockchain, so we don't set
last_processed_block.  Fix that by setting it immediately once we have
a block: worst case it goes backwards a little.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Christian Decker
aba3d5f34d wallet: Clean up spent UTXOs once they are buried deeply enough
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
8958a082ec wallet: Mark utxo outpoints as spent
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
27e0ec694c wallet: Add primitive to register new utxoset outpoint to the wallet
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
d8d11e5689 wallet: Add outpointfilter for the utxoset
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
5a1b4b91e2 wallet: Fix output extraction when we own multiple outputs
When we already know about an output we would stop scanning the remaining
outputs. Known outputs happen whenever we extracted from our own transactions
and then extracted again from blocks. We would not update if the first update
fails.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
ae30942e3f chaintopo: Record outpoint spends for owned outputs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
b3c94a5256 wallet: Update existing owned outputs with their confirmation height
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
7fc06d9b3e wallet: Add outpointfilter to wallet so we can pass it all outputs
Will be used later to filter out outputs we are interested in, and
trigger db updates with them.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
227dc36146 utxo: Add blockheight and spendheight to outputs to track state
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
9440b23852 wallet: Add primitive to roll back to a specific height
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
504202973f wallet: Add primitives to store blockchain internally
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Igor Cota
cb6820d445 Do preimage, hash and consequentially bolt11 generation before we store the invoice in db.
This way we store bolt11 to the table as well, in case a client needs it later
2018-03-01 03:55:48 +00:00
ZmnSCPxj
978e5c67d8 invoices: Remove persistent in-memory invoice structures. 2018-02-28 11:17:08 +01:00
ZmnSCPxj
f05c86618c invoices: Change iteration interface to be more abstract. 2018-02-28 11:17:08 +01:00
ZmnSCPxj
9b4c6699f9 invoices: Semantically separate invoice details from invoice.
In preparation for removing in-memory invoice structures.
Invoice details are requested rarely anyway.
2018-02-28 11:17:08 +01:00
ZmnSCPxj
f7f55edcdb wireaddr: Do hostname resolution in parse_wireaddr.
Fixes: #834
2018-02-26 03:13:55 +00:00
Rusty Russell
86a04c59d4 lightningd: maintain a status billboard for each channel.
Each state (effectively, each daemon) has two slots: a permanent slot
if something permanent happens (usually, a failure), and a transient
slot which summarizes what's happening right now.

Uncommitted channels only have a transient slot, by their very nature.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-23 18:02:00 +01:00
practicalswift
7be95cee94 Remove unused parameter ctx in wallet_peer_load(const tal_t *ctx, ...) 2018-02-22 10:46:30 +01:00
Rusty Russell
097b9345eb wallet: load route_nodes and route_channels when seeking a particular entry.
This got missed when 299b280f7 got merged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 01:38:32 +00:00
practicalswift
4fccbbae69 Assert that we get the assumed return value from sqlite3_step(...) 2018-02-19 12:18:59 +01:00
Rusty Russell
611ecc60ae lightningd: rename peer_state -> channel_state, remove OPENINGD.
And now we can finally do the db upgrade to remove any OPENINGD
channels once, since we never put them back.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell
cc460095ca lightningd: make new_channel a proper constructor.
It's giant, but it's encapsulating at least.  It is called from the wallet
code when loading channels, or from the opening code when converting
an uncommitted_channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell
e0603d7221 channel: now we're always complete, fields don't have to be optional.
Now any struct channel is a genuine channel, the following fields are
always valid:

1. funding_txid: doesn't need to be a pointer.
2. our_msatoshi: doesn't need to be a pointer.
3. last_sig: doesn't need to be a pointer.
4. channel_info: doesn't need to be a pointer.

In addition, 'last_tx' is always valid.

The main effect is to remove a whole heap of branches from the wallet code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell
3c0be71d37 wallet: make static, not dynamic decision to insert for everything.
Since we create new entries from wallet_channel_insert(), there's no
need for the branches.  And indeed, many wallet functions can be
static.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell
72108f0cb9 wallet: don't use rowid for the channel's DBID.
We derive the seed from this, so it needs to be unique, but using
rowid forced us to put the channel into the db early, before it
was ready.

Instead, use a counter to ensure uniqueness, initialized when we load
existing peers.  This doesn't need to touch the database at all.

As we now have only two places where the channel is committed (the
funder and fundee paths), so we create a new explicit
'wallet_channel_insert()' function: 'wallet_channel_save()' now just
updates.

Note that this also fixes some weirdness in
wallet_channels_load_active: we strangely avoided loading channels in
CLOSINGD_COMPLETE (which fortunately was a transient state, so
unlikely anyone hit this).  Note that since the lines above already
delete all the OPENINGD channels, we now simply load them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Christian Decker
b2819f9f97 wallet: Add check that wallet matches the network on startup
Adds a simple check that compares genesis-blockhashes from the
chainparams against the blockhash that the wallet was created
with. The wallet is network specific, so mixing is always a bad idea.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-17 11:36:17 +00:00
Rusty Russell
37373f2c16 wallet: provide better comments on wallet_first_blocknum.
ZmnSCPxj queried the unilateral close case, so make that clearer.
Christian raise concerns about existing channels, so make it clear
what we're doing there too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell
ef9c6cb8c3 wallet: don't scan from worst-case start on first use.
We only need to do that if it's possible there's something to find:
either we have an unspent output from a unilateral close, or we've
ever handed out an address.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell
6620305606 wallet: use last_processed_block to determine scan start.
With fallback depending on chainparams: this means the first upgrade
will be slow, but after that it'll be fast.

Fixes: #990
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell
55d962046b Rename (almost) all destructors to destroy_<type>.
We usually did this, but sometimes they were named after what they did,
rather than what they cleaned up.

There are still a few exceptions:
1. I didn't bother creating destroy_xxx wrappers for htable routines
   which already existed.
2. Sometimes destructors really are used for side-effects (eg. to simply
   mark that something was freed): these are clearer with boutique names.
3. Generally destructors are static, but they don't need to be: in some
   cases we attach a destructor then remove it later, or only attach
   to *some* cases.  These are best with qualifiers in the destroy_<type>
   name.

Suggested-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
6a3ccafaf9 wallet: don't implicitly remove peers, but do it explicitly.
This provides a sanity check that we are in sync, and also keeps the
logic in the program and out of the SQL.

Since the destructor now doesn't clean up the peer, there are some
wider changes to be made when cleaning up.  Most notably we create
lots of channels in run-wallet.c and they previously freed the peer:
now we need free the peer explicitly, so we need to free them first.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
cf7c399cc5 htlc: keep channel pointer, not peer pointer.
And move the no-remaining-htlcs check from the peer destructor to the
channel destructor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
32411de90e lightningd: split struct peer into struct peer and struct channel.
Much like the database; peer contains id, address, channel contains
per-channel information.  Where we create a channel, we always create
the peer too.

For the moment, peer->log and channel->log coexist side-by-side, to
reduce some of the churn.

Note that this changes the API to dev-forget-channel: if we have more
than one channel, we insist they specify the short-channel-id.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
65c09c895d wallet: properly handle case where peer has no address when saving channel.
In practice, it currently always does, so we've never hit an error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
81ca1db347 wallet: delete peers with no channels.
ON DELETE CASCADE goes the other way: we should clean up peers with no
channels from db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
38a313af0d wallet: delete channels in state OPENINGD.
Both when we forget about an opening peer, and at startup.  We're
going to be relying on this, and the next patch, as we refactor
peer/channel handling to mirror the db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
15eaf56d79 wallet: add ld pointer.
This will be required to give it direct access to the ld->peers list.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Christian Decker
37b96e9c95 wallet: Load any stored htlc_sigs when restoring channel
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-11 01:13:07 +01:00
Christian Decker
3bb619fee4 wallet: Add primitive to store htlc_sigs in the database
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-11 01:13:07 +01:00
practicalswift
4f4756bd20 Fix a-vs-an typos 2018-02-08 22:49:34 +01:00
Christian Decker
4f5c47b43b wallet: Do not consider OPENINGD channels as active
Still writing the channel since some of the channel setup parameters
depends on `chan->id` to be set. If we later set the `chan->id`
signatures fail. This prevents OPENINGD channels showing up after
restarting.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-07 00:47:13 +00:00
Christian Decker
3fdb055606 wallet: Make first_blocknum a field of wallet_channel
We were sideloading it, which is awkward, now it's a field that we can
actually use in the code.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-07 00:47:13 +00:00
ZmnSCPxj
1e6747c28e wallet: Pass in timers object during construction.
In preparation for expiration.
2018-02-05 08:52:42 +00:00
Rusty Russell
9b8fe618f6 pay: remove cmd pointer from htlc_out.
Maintaining it was always fraught, since the command could go away
if the JSON RPC died.  Most recently, it was broken again on shutdown
(see below).

In future we may allow pay commands to block on previous payments, so
it won't even be a 1:1 mapping.  Generalize it: keep commands in a
simple list and do a lookup when a payment fails/succeeds.

Valgrind error file: valgrind-errors.5732
==5732== Invalid read of size 8
==5732==    at 0x4149FD: remove_cmd_from_hout (pay.c:292)
==5732==    by 0x468BAB: notify (tal.c:237)
==5732==    by 0x469077: del_tree (tal.c:400)
==5732==    by 0x4690C7: del_tree (tal.c:410)
==5732==    by 0x46948A: tal_free (tal.c:509)
==5732==    by 0x40F1EA: main (lightningd.c:362)
==5732==  Address 0x69df148 is 1,512 bytes inside a block of size 1,544 free'd
==5732==    at 0x4C2EDEB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5732==    by 0x469150: del_tree (tal.c:421)
==5732==    by 0x46948A: tal_free (tal.c:509)
==5732==    by 0x4198F2: free_htlcs (peer_control.c:1281)
==5732==    by 0x40EBA9: shutdown_subdaemons (lightningd.c:209)
==5732==    by 0x40F1DE: main (lightningd.c:360)
==5732==  Block was alloc'd at
==5732==    at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5732==    by 0x468C30: allocate (tal.c:250)
==5732==    by 0x4691F7: tal_alloc_ (tal.c:448)
==5732==    by 0x40A279: new_htlc_out (htlc_end.c:143)
==5732==    by 0x41FD64: send_htlc_out (peer_htlcs.c:397)
==5732==    by 0x41511C: send_payment (pay.c:388)
==5732==    by 0x41589E: json_sendpay (pay.c:513)
==5732==    by 0x40D9B1: parse_request (jsonrpc.c:600)
==5732==    by 0x40DCAC: read_json (jsonrpc.c:667)
==5732==    by 0x45C706: next_plan (io.c:59)
==5732==    by 0x45D1DD: do_plan (io.c:387)
==5732==    by 0x45D21B: io_ready (io.c:397)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-02 20:33:49 +01:00
ZmnSCPxj
299b280f78 pay: Save nodes and channels used on route to payment. 2018-02-01 00:46:06 +00:00