Commit Graph

390 Commits

Author SHA1 Message Date
ZmnSCPxj
f83c4ff903 wallet: Add msatoshi_to_us_min and msatoshi_to_us_max statistics for channels.
So we know how much counterparty could theoretically steal from us
 (msatoshi_to_us - msatoshi_to_us_min) and how much we could
 theoretically steal from counterparty (msatoshi_to_us_max -
 msatoshi_to_us).
For more piloting goodness.
2018-04-05 19:01:53 +02:00
Rusty Russell
19d5305658 wallet/test: fix Makefile so test correctly depend on wallet files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-04 02:31:41 +00:00
Rusty Russell
6bb47276ce lightningd: put min/max feerates into db, struct channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-04 02:31:41 +00:00
Rusty Russell
8975fc2ceb libwally: use their secp context for all our daemons.
I didn't convert all tests: they can still use a standalone context.
It's just marginally more efficient to share the libwally one for all
our daemons which link against it anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-03 14:03:28 +02:00
Christian Decker
1133820ad6 db: Add a cleanup migration to clear any dangling utxoset entries
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-02 21:07:13 +02:00
Christian Decker
eb531e6da7 db: Make sure to enable foreign keys even after forking
This may be causing #1280, since with `--daemon` the DB is being reopened
without enabling the foreign key relations and hence the delete cascades.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-02 21:07:13 +02:00
Christian Decker
a8d587c418 wallet: Return any eventual outpoint scid when marking it spent
Just return the short_channel_id matching the outpoint that we just marked as
spent.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-30 16:35:00 +02:00
Christian Decker
ba7341ec87 cleanup: Make blockheights unsigned
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-27 23:17:17 +00:00
Christian Decker
286cfefc62 wallet: Remove pruned outpoints from utxoset outpoint_filter
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-27 17:14:58 +02:00
practicalswift
6269a4c55d Remove unused functions not covered by unit tests 2018-03-26 23:35:56 +00:00
ZmnSCPxj
b914062465 pay: Also record how much we actually ended up sending. 2018-03-26 05:01:49 +00:00
practicalswift
7e9750ffee Reduce variable scopes 2018-03-26 01:31:21 +00:00
ZmnSCPxj
0bb9bcc0f1 wallet: Track some channel usage statistics.
Fixes: #1049
2018-03-26 01:08:52 +00:00
Rusty Russell
76e8a11380 wallet: use json_escaped for invoice label.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
Rusty Russell
d92579f627 common/json_escaped: new type which explicitly notes a string is already JSON.
Trivial to use as a string, but it still means you should be careful
around it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
practicalswift
a4059ef83e Use expected LIGHTNING_DIR_FILE_H define 2018-03-25 23:54:21 +00:00
practicalswift
98f49c0837 Remove include in file foo.c that is already included in foo.h 2018-03-25 23:54:21 +00:00
practicalswift
0bf1b01425 Fix typos 2018-03-25 15:53:01 +02:00
ZmnSCPxj
d5d31864cf wallet: Remove onion-decoding information from db on payment success/fail.
Fixes: #1177
2018-03-21 13:56:18 +01:00
Mark Beckwith
489d8d04a6 Fund all (#1249)
* Funding channel with "all"
* Added test for fund all
2018-03-21 00:06:04 +01:00
ZmnSCPxj
159e736fc1 invoices: Implement autocleaninvoice. 2018-03-20 17:25:51 +01:00
ZmnSCPxj
3b0081aebb invoices: Add delexpiredinvoice command. 2018-03-20 17:25:51 +01:00
ZmnSCPxj, ZmnSCPxj jxPCSmnZ
044705a547 peer_control: Fix a use-after-free bug. (#1237)
This bug is a classic case of being lazy:
1. peer_accept_channel() allocated its return off the input message,
   rather than taking an explicit allocation context.  This concealed the
   lifetime nature of the return.
2. The context for sanitize_error was the error itself, rather than the
   more obvious tmpctx (connect_failed does not take).

The global tmpctx removes the "efficiency" excuse for grabbing a random
object to use as context, and is also nice and explicit.

All-the-hard-work-by: @ZmnSCPxj
2018-03-19 09:27:09 +00:00
ZmnSCPxj
9efe123a0d lightningd/json: Move json helpers specific for lightningd to new module. 2018-03-19 00:27:55 +00:00
Rusty Russell
1b9791f0f5 pay: delete HTLC when we delete payment.
This fixes the root cause of https://github.com/ElementsProject/lightning/issues/1212
where we deleted the payment because we wanted to retry, then retry failed
so we had an (old) HTLC without a matching payment.  We then fed that
HTLC to onchaind, which tells us it's missing, and we try to fail the
payment and deref a NULL pointer.

Fixes: #1212
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 04:01:06 +00:00
practicalswift
03e2e404f2 Mark unused parameters as such (using "UNUSED") 2018-03-16 01:17:40 +00:00
practicalswift
9970419b3a Remove redundant return from void function 2018-03-16 01:17:40 +00:00
Rusty Russell
0a6e3d1e13 utils: remove tal_tmpctx altogether, use global.
In particular, we now only free tmpctx at the end of main().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell
de5d84097e lightningd: remove ltmp context now we have tmpctx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
ZmnSCPxj
60f5abdc9b wallet: Make wallet_payment_store idempotent. 2018-03-14 05:33:09 +00:00
ZmnSCPxj
693c718160 wallet: Add DB fields for recording routing failures. 2018-03-14 05:33:09 +00:00
Rusty Russell
46cc7c281e features: more general accessor functions.
As we add more features, the current code is insufficient.

1. Keep an array of single feature bits, for easy switching on and off.
2. Create feature_offered() which checks for both compulsory and optional
   variants.
3. Invert requires_unsupported_features() and unsupported_features()
   which tend to be double-negative, all_supported_features() and
   features_supported().
4. Move single feature definition from wire/peer_wire.h to common/features.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Christian Decker
387ba76920 wallet: Add method to retrieve outpoints from the utxoset
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00
Christian Decker
d363a6870d wallet: Fix header ordering
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-07 18:55:51 +01:00
Rusty Russell
f8c8d55565 db: migrate old shutdown_keyidx_local -1 values onto key 0.
It would be better to give them unique values, but we don't fully support
db migrate anyway and this is simple (though they will end up using the
same key for multiple channel closes if created before this commit).

Note that even if bip32_max_index is currently unset, it defaults to 0
so it will be found.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell
6c1233de44 channel: reserve a bip32 index as soon as channel is opened.
This simplifies things, and means it's always in the database.  Our
previous approach to creating it on the fly had holes when it was
created for onchaind, causing us to use another every time we
restarted.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell
242a934b8c wallet: add check-source to makefile.
And fix up resulting breakage.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell
e76f863329 txfilter: clean up prototypes.
Use const, add TAKES to declaration.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Rusty Russell
9d9c1c3db6 wallet: wrap missing last_processed_block handling in COMPAT_V052.
Our testing also reveals a bug: we start lightningd and shut it down
before fully processing the blockchain, so we don't set
last_processed_block.  Fix that by setting it immediately once we have
a block: worst case it goes backwards a little.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
John Barboza
30b290cb8f Explicit error message when disconnect fails
Whether the peer is not connected or not gossiping.
2018-03-07 16:14:01 +01:00
John Barboza
0a1351fd51 gossipd: command to disconnect from a gossiping peer
lightning-cli disconnect <peer id>
2018-03-07 16:14:01 +01:00
Christian Decker
aba3d5f34d wallet: Clean up spent UTXOs once they are buried deeply enough
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
8958a082ec wallet: Mark utxo outpoints as spent
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
50a8e32e3f db: Add index on utxoset.spendheight to speed up blockchain reorgs
Otherwise we would be doing a table scan per block being reorged/rescanned.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
27e0ec694c wallet: Add primitive to register new utxoset outpoint to the wallet
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
d8d11e5689 wallet: Add outpointfilter for the utxoset
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
ec014be2a8 db: Add utxoset table
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
5a1b4b91e2 wallet: Fix output extraction when we own multiple outputs
When we already know about an output we would stop scanning the remaining
outputs. Known outputs happen whenever we extracted from our own transactions
and then extracted again from blocks. We would not update if the first update
fails.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
ae30942e3f chaintopo: Record outpoint spends for owned outputs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
2680e6d9ff wallet: Move txfilter into wallet
Transaction filters are strongly related to the wallet, this move just
makes it a bit more explicit.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
b3c94a5256 wallet: Update existing owned outputs with their confirmation height
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
7fc06d9b3e wallet: Add outpointfilter to wallet so we can pass it all outputs
Will be used later to filter out outputs we are interested in, and
trigger db updates with them.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
247aa13a58 db: Add index covering the output heights
Avoids performing a table scan, now deletes on blocks are a lot
faster.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
4e2c1dd038 jsonrpc: Add confirmation status to listfunds output
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
227dc36146 utxo: Add blockheight and spendheight to outputs to track state
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
7f00230657 db: Add confirmation_height and spend_height columns to outputs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
9440b23852 wallet: Add primitive to roll back to a specific height
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
504202973f wallet: Add primitives to store blockchain internally
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
fea03dbca9 db: Add a table to track blockheaders
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
John Barboza
b50912d8b1 Print node alias when listing peers 2018-03-02 02:25:30 +00:00
Rusty Russell
c5d41a23d7 short_channel_id: just use structeq.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-01 23:33:56 +01:00
Igor Cota
cb6820d445 Do preimage, hash and consequentially bolt11 generation before we store the invoice in db.
This way we store bolt11 to the table as well, in case a client needs it later
2018-03-01 03:55:48 +00:00
Igor Cota
3371168d54 Add bolt11 to the invoice struct.
We store this in case an RPC client asks for it
2018-03-01 03:55:48 +00:00
Igor Cota
2bb9797ddb Add bolt11 to the invoices db table as TEXT.
Saving in case an RPC client needs it later
2018-03-01 03:55:48 +00:00
ZmnSCPxj
efc5da9ad4 invoices: Use common code for loading r and rhash. 2018-02-28 11:17:08 +01:00
ZmnSCPxj
978e5c67d8 invoices: Remove persistent in-memory invoice structures. 2018-02-28 11:17:08 +01:00
ZmnSCPxj
f05c86618c invoices: Change iteration interface to be more abstract. 2018-02-28 11:17:08 +01:00
ZmnSCPxj
e9112af0bc invoices: Move expiration timer to invoices object.
In preparation for removing in-memory invoices objects.
2018-02-28 11:17:08 +01:00
ZmnSCPxj
7a6fe0704e invoices: Unify waitone/any waiters to invoices object.
In preparation for removing in-memory invoice objects.
Since they will be removed, there will no longer be any
individual invoice objects to attach to.
2018-02-28 11:17:08 +01:00
ZmnSCPxj
9b4c6699f9 invoices: Semantically separate invoice details from invoice.
In preparation for removing in-memory invoice structures.
Invoice details are requested rarely anyway.
2018-02-28 11:17:08 +01:00
Rusty Russell
b7ed5670d5 lightningd: close and reopen db across fork for daemonize
Fixes: #1092
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-27 00:09:34 +01:00
ZmnSCPxj
f7f55edcdb wireaddr: Do hostname resolution in parse_wireaddr.
Fixes: #834
2018-02-26 03:13:55 +00:00
ZmnSCPxj
bd07a91782 jsonrpc: Let Base58 '2..' address work on both regtest and testnet. 2018-02-26 03:09:15 +00:00
John Barboza
b028a0a439 invoice: add fallback address to invoice command
* Modifies invoice command to have the following format
  invoice <msatoshi> <label> <desc> <?expiry> <?fallbackaddr>
* Adds support for Segwit bcrt1 addresses for withdraw
* Add test case for fallback address in invoice creation
* Create a common json_tok_address_scriptpubkey to be used
  by invoice and withdraw commands.
2018-02-26 03:09:15 +00:00
Christian Decker
7c8883ae7d wallet: Fix an off by one error in dev-listaddrs
Reported-By: @Xian001
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-23 21:53:40 +00:00
Rusty Russell
cccdb53bec channel_states: fold all the ONCHAIND states into one.
The billboard is now far more useful to tell what's going on, and this
gets us closer to a state == owner mapping.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-23 18:02:00 +01:00
Rusty Russell
86a04c59d4 lightningd: maintain a status billboard for each channel.
Each state (effectively, each daemon) has two slots: a permanent slot
if something permanent happens (usually, a failure), and a transient
slot which summarizes what's happening right now.

Uncommitted channels only have a transient slot, by their very nature.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-23 18:02:00 +01:00
Rusty Russell
655ae90e5e wallet: remove obsolete peers.
Fixes: #1029
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-22 12:06:39 +01:00
practicalswift
d873bf60bf Remove unused parameter fieldname in json_add_log(..., const char *fieldname, ...) 2018-02-22 10:46:30 +01:00
practicalswift
7be95cee94 Remove unused parameter ctx in wallet_peer_load(const tal_t *ctx, ...) 2018-02-22 10:46:30 +01:00
practicalswift
1f46262278 Remove unused parameter testname in create_test_db(const char *testname) 2018-02-22 10:46:30 +01:00
practicalswift
91a9c2923f Mark intentionally unused parameters as such (with "UNUSED") 2018-02-22 01:09:12 +00:00
Rusty Russell
e92b710406 tools/generate-wire.py: remove length argument from fromwire_ routines.
We always hand in "NULL" (which means use tal_len on the msg), except
for two places which do that manually for no good reason.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell
719290a4c4 txwatch: remove unused callback arg, hide struct definitions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell
ae8fb96d06 lightningd/channel_control: routines to control channeld (move from peer_control.c)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell
372040bbd5 lightningd/onchain_control: routines to control onchaind (move from peer_control.c)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell
9176ee628c lightningd/closing_control: routines to control closingd (move from peer_control.c)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell
a83ff83328 lightningd/connect_control: routines to control connecting (move from peer_control.c)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell
4cf274b1c0 lightningd/opening_control: routines to control openingd (move from peer_control.c)
We also fold opening_got_hsm_funding_sig() into the caller; it was
previously a callback before we decided to always use the HSM
synchronously.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell
d47d85fa30 wallet: add list of upgrades.
Useful for debugging a db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 01:38:32 +00:00
Rusty Russell
097b9345eb wallet: load route_nodes and route_channels when seeking a particular entry.
This got missed when 299b280f7 got merged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 01:38:32 +00:00
practicalswift
4fccbbae69 Assert that we get the assumed return value from sqlite3_step(...) 2018-02-19 12:18:59 +01:00
Rusty Russell
cfa50d393a openingd: use peer_failed like normal instead of boutique negotiation_failed.
Because peer_failed would previously drop the connection, we had a
special 'negotiation_failed' message which made the master hand it
back to gossipd.  We don't need that any more.

This also meant we no longer need a special hook in read_peer_msg
for openingd to send this message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell
02d469b3d4 peer_failed: hand fds back to master when we fail.
master now hands it back to gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell
611ecc60ae lightningd: rename peer_state -> channel_state, remove OPENINGD.
And now we can finally do the db upgrade to remove any OPENINGD
channels once, since we never put them back.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell
cc460095ca lightningd: make new_channel a proper constructor.
It's giant, but it's encapsulating at least.  It is called from the wallet
code when loading channels, or from the opening code when converting
an uncommitted_channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell
4ab7589427 sqlite3_column_arr: helper to allocate and read an array from a blob.
We use it on the secrets array for the moment, but it's also useful
for remote_shutdown_scriptpubkey, as used in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell
e0603d7221 channel: now we're always complete, fields don't have to be optional.
Now any struct channel is a genuine channel, the following fields are
always valid:

1. funding_txid: doesn't need to be a pointer.
2. our_msatoshi: doesn't need to be a pointer.
3. last_sig: doesn't need to be a pointer.
4. channel_info: doesn't need to be a pointer.

In addition, 'last_tx' is always valid.

The main effect is to remove a whole heap of branches from the wallet code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell
d2f691b288 subd: make functions more generic, don't assume 'struct channel'.
This means the caller needs to supply an explicit log to base the
subd log on, and also a callback for error handling.

The callback is kind of ugly, but it gets reworked towards the end
of this series.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell
a2c6ec6c9b lightningd: use tal_link for log_book.
BackgroundL Each log has a log_book: many logs can share the same one,
as each one can have a separate prefix.

Testing tickled a bug at the end of this series, where subd was
logging to the peer's log_book on shutdown, but the peer was already
freed.  We've already had issues with logging while lightningd is
shutting down.

There are times when reference counting really is the right answer,
this seems to be one of them: the 'struct log' share the 'struct
log_book' and the last 'struct log' cleans it up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00