Commit Graph

1042 Commits

Author SHA1 Message Date
practicalswift
e91a8dff12 Change log level for some common debug messages from "info" to "debug" 2018-01-16 03:20:27 +00:00
Christian Decker
5319ff1bc0 bitcoind: Do not crash when getblock fails
This is a common occurence on pruned nodes. By calling the callback
upon failures, we communicate that we couldn't verify the txoutput. We
fail safe rejecting any channel we can't verify.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-15 19:29:01 +00:00
Rusty Russell
4b663e596a lightningd: don't crash when txout lookup fails.
Gossipd already correctly handles the "empty output means lookup failed" case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-15 19:29:01 +00:00
Rusty Russell
c66df31674 subd: pass absolute path as argv[0].
This means we print out the correct path with --debugger, which
can be vital if there are multiple binaries (eg. compiled vs installed).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-15 19:26:08 +00:00
Rusty Russell
904a3e4ae3 MAX_FUNDING_SATOSHI: clean up def to be max, fix name, share with openingd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-15 12:45:12 +01:00
Christian Decker
351718e90f jsonrpc: Return an error if funding amount is too large
Fixes #516

Reported-by: @instagibbs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-15 12:45:12 +01:00
Rusty Russell
dbef4a1827 pay: remove gratuitous check for msatoshitok being 'null'.
json_get_params does this for us.

Fixes: 78adf0b (pay: allow 'null' msatoshi field.)
Reported-by: ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-15 12:44:02 +01:00
Christian Decker
dd599706cd peer: Save whenever we change last_was_revoke as well
Pulling up the save call from `peer_save_commitsig_received` into its
caller `peer_got_commitsig` and adding a call to
`peer_sending_commitsig`

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-15 12:43:22 +01:00
practicalswift
4214c5ad57 Use consistent formatting for CLI command descriptions 2018-01-15 04:20:30 +00:00
practicalswift
caf57b24ea Use correct description for the "help" CLI command 2018-01-15 04:20:30 +00:00
practicalswift
447afa923d Avoid segfault on CLI command "dev-memleak" when $LIGHTNINGD_DEV_MEMLEAK is not set 2018-01-15 02:42:59 +00:00
Björge Dijkstra
44836ef509 Fix double free of message buffer.
Message buffer `why` is allocated in the `peer` context and also freed when peer is freed.
Only explicitly free the buffer when peer itself is not freed yet.
2018-01-15 02:12:10 +00:00
William Casarin
3c0d2813a0 bitcoind: properly handle spent outputs in gettxout
exit status is not enough to detect spent outputs. gettxout will return a
success exit code and 0 bytes.

Signed-off-by: William Casarin <jb55@jb55.com>
2018-01-14 23:49:59 +00:00
Christian Decker
7fc8e3a4e2 gossip: Pass --channel-update-interval to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
Christian Decker
0419688b0c gossip: Added --channel-update-interval argument
We'll pass this down to gossip and make sure to re-announce/update
channels every so often. This is also used as a pruning timer, i.e.,
channels that have not been updated in 2 x channel-update-interval
will be pruned from the local view.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
Rusty Russell
6d7c8c8dd0 doc: manpages for pay, listpayments and decodepay.
Also contains some help message clarifications.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-14 23:10:10 +00:00
Rusty Russell
a1a0b33577 pay: allow 'null' msatoshi field.
Since most callers use positional arguments, we should allow a 'null'
literal where we require no value at all.

Also adds some more value tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-14 23:10:10 +00:00
Rusty Russell
2f2fb0c2a1 invoice: add msatoshi_received field.
Paid invoices need to know how much was actually paid: both for the case
where no 'msatoshi' amount was specified, and for the normal case, where
clients are permitted to overpay in order to help them disguise their
payments.

While we migrate the db, we leave this field as 0 for old paid
invoices.  This is unhelpful for accounting, but at least clearly
indicates what happened if we find this in the wild.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-14 23:10:10 +00:00
Rusty Russell
66885163c9 JSON: Rename rhash to payment_hash in delinvoice, invoice, listinvoice, waitinvoice, waitanyinvoice.
'rhash' is the old terminology, but 'payment_preimage' and
'payment_hash' were decided on for the BOLTs, so we should fix that here.

We still use rhash internally, but that's much easier to fix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-14 23:10:10 +00:00
Rusty Russell
cc4caf9d69 invoice: common routine for invoice to json.
Different commands (listinvoice, delinvoice, waitinvoice,
waitanyinvoice) returned different fields, as not all were updated.
This makes them uniform.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-14 23:10:10 +00:00
Rusty Russell
cce432b77f wallet_invoice_nextpaid: return a struct invoice.
This reuses the same code internally, and also now means that we deal
correctly with "any" msatoshi invoices: the old code would a return
'msatoshi' of 0 in that case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-14 23:10:10 +00:00
ZmnSCPxj
83e76e3ac3 features: Move feature-handling code to a common/features.c source. 2018-01-13 11:29:42 +01:00
ZmnSCPxj
2d8304f176 pay: Change online help of 'listinvoices' to use 'msatoshi' as per actual response returned. 2018-01-13 03:06:05 +00:00
ZmnSCPxj
c402d834e7 invoice: Rename 'amount' argument of 'invoice' RPC to 'msatoshi'.
The manfile and the online help use 'msatoshi', the returned
response uses 'msatoshi', nearly every invoice-related
monetary amount is labelled 'msatoshi' and not 'amount'.
2018-01-13 03:06:05 +00:00
William Casarin
2e796ffa67 peer_control: send addrhints in activate_peer
Since we now have addresses in the database, we can resend them as hints to
gossipd on startup.

Signed-off-by: William Casarin <jb55@jb55.com>
2018-01-12 19:47:36 +01:00
Rusty Russell
2bc2502b23 wire: move extract_channel_id to here.
It can be useful for other daemons.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-12 09:43:01 +01:00
Igor Cota
e97857847a peer_control: make hardcoded feerates consistent between json_fund_channel and peer_offer_channel
Stops some transactions from failing on the other side due to insufficient funds.
2018-01-12 03:40:31 +00:00
ZmnSCPxj
816298123f invoice: Allow "any" msatoshi, for amountless invoices.
Fixes: #534
2018-01-12 01:43:55 +00:00
ZmnSCPxj
caab95b922 wallet_payment: Make msatoshi field nullable. 2018-01-12 01:43:55 +00:00
ZmnSCPxj
50471bf4fe invoice: Make msatoshi field nullable.
Note that the database itself has the field nullable (sqlite3
does not even have non-nullable columns) but our in-memory
structures did not.
2018-01-12 01:43:55 +00:00
ZmnSCPxj
7221476553 invoice: Clarify comments about the two loops in resolve_invoice. 2018-01-12 01:28:32 +00:00
practicalswift
a900551815 Use tal_hex(...) instead of tal_hexstr(...) 2018-01-12 00:55:46 +00:00
Rusty Russell
d4c8210a9e gossipd: don't hang if we try to connect to already-connected peer.
Closes: #287
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:47:47 +01:00
Rusty Russell
e22dd5ad71 lightningd: implement gossip_get_txout/gossip_get_txout_reply.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell
2b1eb6a677 bitcoind: getoutput so we can check short_channel_ids.
It would be nice if bitcoind had an RPC to do this in one, but that's
a bit much to ask for.  We could also hand around proofs, for lite nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell
9a8eca4fee Remove obsolete FIXME messages.
1. htlc->fail has been changed to a u8 *.
2. wallet_get_newindex saves to the db.
3. peer->next_htlc_id is saved to the db in peer_save_commitsig_sent() below.
4. We do store commit in peer_save_commitsig_received(peer, commitnum),
   and the fixme below talks about HTLC sigs.
5. We do commit shachain and next_per_commit_point in wallet_shachain_add_hash
   and update_per_commit_point respectively.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell
55909a7a39 lightningd: call peer_last_tx() to update peer->last_tx before saving to db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell
8cf97e904d lightningd: remove unused "forever_confirms" parameter.
It's 100, but we never wired this up, so remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell
7e0bc88e2a gossip: fix default broadcast interval, move option.
This now makes a few more gossip tests time out without --dev-broadcast-interval

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell
171ce689dc lightningd/chaintopology: remove unneeded chain_topology pointer from block.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell
3332cd6395 chain_topology: remove unneeded struct outgoing_tx topo pointer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
practicalswift
ab0c26be91 Check return value of json_get_params(...) call in json_getlog(...) and json_getpeers(...)
All other users of json_get_params(...) check the return value:

```
lightningd/chaintopology.c:     if (!json_get_params(buffer, params,
lightningd/chaintopology.c:     if (!json_get_params(buffer, params,
lightningd/dev_ping.c:  if (!json_get_params(buffer, params,
lightningd/gossip_control.c:    if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/invoice.c:   if (!json_get_params(buffer, params, "label", &labeltok, NULL)) {
lightningd/invoice.c:   if (!json_get_params(buffer, params,
lightningd/jsonrpc.c:   if (!json_get_params(buffer, params,
lightningd/pay.c:       if (!json_get_params(buffer, params,
lightningd/pay.c:       if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
lightningd/peer_control.c:      if (!json_get_params(buffer, params,
wallet/walletrpc.c:     if (!json_get_params(buffer, params,
wallet/walletrpc.c:     if (!json_get_params(buffer, params, "tx", &txtok, NULL)) {
```
2018-01-10 23:43:50 +00:00
Rusty Russell
467ca0a996 lightningd: suppress feerate debug messages when nothing changes.
Every 30 seconds, these flood the logs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-10 16:55:36 +01:00
ZmnSCPxj
e717963210 lightningd: Do not search for ourself in the daemon dir.
In preparation for putting our daemons into pkglibexecdir; lightningd
itself will be in bindir, separate from our subdaemons.
2018-01-09 14:52:48 +01:00
ZmnSCPxj
7f88585d84 lightningd: When searching subd fall back to pkglibexecdir. 2018-01-09 14:52:48 +01:00
ZmnSCPxj
c572c3bbee lightningd: Add --test-daemons-only option. 2018-01-09 14:52:48 +01:00
practicalswift
4bdd2452f2 Make sure fsync, connect and close are never accidentally passed negative arguments 2018-01-09 14:50:50 +01:00
Rusty Russell
795e42d757 chaintopology: fix use-after-free which sometimes causes fatal() in clear_otx_peer
I've only seen this under travis, so I can't verify that this fixes it,
but it's certainly a bug which could cause that issue.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-09 14:49:52 +01:00
practicalswift
dcb4039a96 Check lseek(...) return value 2018-01-09 13:52:12 +01:00
Rusty Russell
a8de8a3140 json_getchannels: add public flag.
Fixes: #509
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:09:31 +01:00
Rusty Russell
7975a1cbfc json_getchannels: short_id -> short_channel_id, use standard format.
We include flags separately, so name this field as per spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:09:31 +01:00
Christian Decker
3db4474465 hsm: Generate fully signed funding transaction in the HSM
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker
37c2873c88 wallet: Store outputs from onchaind in the DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker
0bb264e1a2 wallet: Added unilateral close info to utxo
This is necessary to grad the their_unilateral/to-us outputs since
they aren't being harvested by `onchaind`

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker
8b6ab6a901 wallet: Remember scriptpubkey that we pass to onchaind on init
This is the scriptpubkey that onchaind spends all funds to, except for
the their_unilateral/to-us case, so we better recognize that address.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker
5f29a218b7 onchain: Add message to notify about their_unilater/to-us outputs
This is the only case in which we don't respend to a simple keyindex'd
pubkey, so we need to handle this for future spends.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker
85eb743e02 wallet: Detect close transactions and track their outputs in DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker
2a3c9e96f0 txfilter: Expose the txfilter_add_scriptpubkey function
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker
e4e9e22fe8 wallet: Use newindex primitive to generate change key index
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
Christian Decker
b13a972929 gossip: Do not return empty address-objects for getnodes
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 09:48:15 +00:00
practicalswift
d02b890b55 assert(tal_count(log) > 0) in log_to_json(...) 2018-01-08 09:39:45 +00:00
Rusty Russell
05a3db886b peer_control: fix memleak in dev-disconnect.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 16:08:50 +01:00
Rusty Russell
ee939c8dcb lightingd: fix memleak when we reload from db.
We always arm the funding_lockin_cb, even if we don't need to.  If we
have an short_channel_id already from the db, this was replacing it
and leaking the old one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell
fb81e6c14b chaintopology: always start 100 blocks back.
Since we panic when we see our root reorg out, even if we're not doing
anything yet, restoring the 100 block margin is the simplest fix.

Unfortunately this means adding a 100-block spacer in the tests, so things
don't get confused.

Fixes: #511
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell
36316957e3 lightningd: set parent correctly for loaded peers.
The current code makes the channel the parent, which is a cycle.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell
a5c65e2c9b lightningd: reactivate peers.
This is surprisingly simple.  We set up the watches for funding tx
depth and the funding output, then if it's not onchain we ask gossipd
to reconnect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell
1b41335121 chain_topology: two-stage startup.
Load the first block we're possibly interested in, then load the peers so
we can restore the tx watches, then finally replay to the current tip.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell
7b735e5de8 lightningd: scan blockchain from first possible block.
Eventually we want to save blockchain in db to avoid this scan, but
for the moment, we need to reload as far back as we may be interested in.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell
93c589efe8 wallet: save block height when we first create channel.
This gives us a lower bound on where funding tx could be.

In theory, it could be lower than this if we get a reorganization, but
in practice this is already a 1-block buffer (since we can't get into
current block, only the next one).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell
4dea3c4747 lightningd: remove unused db stubs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-05 15:05:21 +01:00
Rusty Russell
ba22484901 lightningd: simplify permanent failure.
Turns out everyone wanted a formatted string anyway.

Inspired-by: practicalswift
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-03 19:56:28 +00:00
ZmnSCPxj
5eceaa7be9 invoice: Modify waitanyinvoice interface to use pay_index. 2018-01-03 01:12:49 +00:00
ZmnSCPxj
3003b7346c invoice: Add pay_index member to struct invoice.
In preparation for change in interface of waitanyinvoice.
2018-01-03 01:12:49 +00:00
practicalswift
66e7c54810 Avoid out-of-bounds read in increase(u32 feerate_per_kw) 2018-01-02 18:37:22 +01:00
practicalswift
3d39312212 Fix typos 2018-01-02 15:09:36 +01:00
Rusty Russell
1e51275792 lightningd: don't save wallet_channel twice on creation.
peer_channel_new() does exactly that already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-02 13:21:25 +01:00
Rusty Russell
7e062fd637 onchaind: don't assert() when htlc fulfilled twice.
In the normal (peer-to-peer) path, the HTLC state prevents us fulfilling
twice, but this goes out the window with onchain HTLCs.

The actual assert which caught it was lightningd/pay.c:70 (payment_succeeded)
in the test_htlc_in_timeout test, after the next commit.

So add an assert earlier (in fulfill_our_htlc_out) and check in the
one caller where it can be true.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-02 13:21:25 +01:00
ZmnSCPxj
f2f116d613 lightningd: Add missing final cleanups. 2018-01-02 01:31:04 +00:00
practicalswift
c6b5e72e3b Pass bool true instead of "true" to json_add_bool(..., bool value) 2018-01-02 01:18:42 +00:00
ZmnSCPxj
8e3c97762e invoice: Properly implement waitinvoice.
Fixes: #444
2018-01-02 00:36:16 +00:00
ZmnSCPxj
773f7524dd invoice: Factor out invoice deletion code.
In preparation for adding a deletion trigger for
invoice waiters on individual invoices.
2018-01-02 00:36:16 +00:00
ZmnSCPxj
1de339eff6 invoice: Use wallet_invoice_nextpaid in waitanyinvoice implementation.
Fixes: #442
2018-01-01 23:13:31 +00:00
practicalswift
84dd65485c Remove duplicate includes 2017-12-28 16:09:24 +01:00
Rusty Russell
6debaccfd5 chaintopology: only do callbacks once chain has settled.
This is only important for testing, where we add 100 blocks at once
and time out under valgrind.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell
887e9dcc44 travis: reenable check-source (without BOLT text).
We've been slipping, so fix up minor issues too so it compiles.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell
0650653658 bitcoind: delete chaintips code.
We don't need it any more, with the simpler topology approach.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell
1d9a8e5484 chaintopology: load forwards, not backwards.
We used to load the new tip and work backwards until we joined up with
the previous tip.  That consumed quite a lot of memory if there were
many blocks.

Instead, just poll on blocknum+1, and grab it once that succeeds.  If
prev is different from what we expect (reorg), we free the current tip
and try again.

We could theoretically miss a reorg which is the same length (2 block
reorg with more work due to difficulty adjustment), but even if that
happened we'd catch up on the next block.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell
6f6d7a5e44 chaintopology: get fees using a timer, not on each block.
It definitely changes when we get a block, but it also changes between
blocks as mempool fills.  So put it on its own timer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell
985a0b431f getblockhash: don't get upset if we fail.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell
810abb6b21 bitcoin: create new wrapper type bitcoin_blkid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_blkid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Rusty Russell
0237e0b28c bitcoin: create new wrapper type bitcoin_txid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_txid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Rusty Russell
553ebc936b json: add json_add_txid.
I prefer the typesafety of specific functions, rather than having the
caller know that txids are traditionally reversed in bitcoin.

And we already have a bitcoin_txid_to_hex() function for this.

Closes: #411
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
William Casarin
7ecccd50b9 wireaddr: add ip[:port] parsing
* Add port parsing support to parse_wireaddr. This is in preparation for storing
addresses in the peers table. This also makes parse_wireaddr a proper inverse of
fmt_wireaddr.

* Move parse_wireaddr to common/wireaddr.c this seems like a better place for
it. I bring along parse_ip_port with it for convenience. This also fixes some
issues with the upcoming ip/port parsing tests.

Signed-off-by: William Casarin <jb55@jb55.com>
2017-12-21 09:56:20 +00:00
Rusty Russell
90a5ba043c htlc_out: initialize hout->key.id, in case we destroy it before setting.
We set hout->key.id when channeld tells us what it is, but if channeld
dies before that we free the hout, and our destructor logs it:

Valgrind error file: valgrind-errors.20312
==20312== Use of uninitialised value of size 8
==20312==    at 0x53ABC9B: _itoa_word (_itoa.c:179)
==20312==    by 0x53B041F: vfprintf (vfprintf.c:1642)
==20312==    by 0x53B17D5: buffered_vfprintf (vfprintf.c:2330)
==20312==    by 0x53AEAA5: vfprintf (vfprintf.c:1301)
==20312==    by 0x53B7D63: fprintf (fprintf.c:32)
==20312==    by 0x128BAC: hout_subd_died (peer_htlcs.c:316)
==20312==    by 0x16D8E0: notify (tal.c:240)
==20312==    by 0x16DD95: del_tree (tal.c:400)
==20312==    by 0x16DDE7: del_tree (tal.c:410)
==20312==    by 0x16DDE7: del_tree (tal.c:410)
==20312==    by 0x16E1B4: tal_free (tal.c:509)
==20312==    by 0x162B5C: io_close (io.c:443)
==20312==    by 0x12D563: sd_msg_read (subd.c:508)
==20312==    by 0x161EA5: next_plan (io.c:59)
==20312==    by 0x1629A2: do_plan (io.c:387)
==20312==    by 0x1629E0: io_ready (io.c:397)
==20312==    by 0x164319: io_loop (poll.c:305)
==20312==    by 0x118E21: main (lightningd.c:334)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 09:13:28 +01:00
Rusty Russell
3543530172 build_utxos: fix weight calculation, and make more accurate.
Accuracy improvements:

1. We assumed the output was a p2wpkh, but it can be user-supplied now.
2. We assumed we always had change; remove this for wallet_select_all.

Calculation out-by-one fixes:

1. We need to add 1 byte (4 sipa) for the input count.
2. We need to add 1 byte (4 sipa) for the output count.
3. We need to add 1 byte (4 sipa) for the output script length for each output.
4. We need to add 1 byte (4 sipa) for the input script length for each input.
5. We need to add 1 byte (4 sipa) for the PUSH optcode for each P2SH input.

The results are now a slight overestimate (due to guessing 73 bytes
for signature, whereas they're 71 or 72 in practice).

Fixes: #458
Reported-by: Jonas Nick @jonasnick
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:56:15 +01:00
Rusty Russell
abafcae5a6 lightningd/build_utxos: don't tell them they can pass NULL for change.
They can't: wallet_select_coins() doesn't allow it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:56:15 +01:00
Rusty Russell
e695e5db09 lightningd: require explicit LIGHTNINGD_DEV_MEMLEAK env var to do memleak.
Otherwise every allocation and free is slowed down.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
6b9c525f35 lightningd: use env var not cmdline to suppress backtrace.
We now set it up *before* parsing cmdline, so this is more convenient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Christian Decker
ee25547576 Re-enable builds with DEVELOPER=0
Two changes:
 - Fixed the function signature of noleak_ to match in both
   configurations
 - Added memleak.o to linker for tests

Generating the stubs for the unit tests doesn't really work since the
stubs are checked in an differ between the two configurations, so
adding memleak to the linker fixes that, by not requiring stubs to be
generated in the first place.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-20 12:43:10 +01:00
Rusty Russell
3ed496e45b lightningd: free failed output htlcs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
ba6904f833 pay: close leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
85f065a6e7 peer_control: close leak in sign_last_tx.
We can call this multiple times.  The best solution is to add and remove
the signature so it's always unsigned as we expect it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
ed8baf5f88 funding_spent: fix leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
8c4eecc089 sphinx: fix leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
f19e02eeaa pay: close leak in payment_failed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
38a0ee8638 jsonrpc: current pending commands are not a leak.
The pay command in particular, attaches a reasonable number of
temporaries to cmd, knowing they'll be freed once cmd is done.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
1356913143 invoice: remove strange code.
This is called when we load from database: clearly our tests aren't thorough
enough because we were allocating and initializing `r` in an unused structure.

invs is also the owner already; functions which steal are a bit surprising
to callers, so we either document them, or just don't do it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
adf8740b96 peer_control: fix opening fundee leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
4957921e02 log: mark ltmp as notleak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
e64320c76c options: suppress "leak" from opt_table allocation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
ccb7047291 lightningd: add notleak annotations.
We have things which we don't keep a pointer to, but aren't leaks.
Some are simply eternal (eg. listening sockets), others cases are
io_conn tied to the lifetime of an fd, and timers which expire.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
6c951bf7e0 invoice: move link to the top of the structure.
memleak doesn't detect pointers to within an object, only pointers to their
exact address (it's simpler this way).  Moving the linked list to the
top of the structure means it can follow the chain.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
cec87ba08b peer_control: move link to the top of the structure.
memleak doesn't detect pointers to within an object, only pointers to their
exact address (it's simpler this way).  Moving the linked list to the
top of the structure means it can follow the chain.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
70d01b22e1 memleak: add second exclude arg to exclude current commands' jcon.
This is not a child of cmd, since they have independent lifetimes, but
we don't want to noleak them all, since it's only the one currently in
progress (and its children) that we want to exclude.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
dfc132b2fe memleak: add backtrace to allocations.
We use the tal notifiers to attach a `backtrace` object on every
allocation.

This also means moving backtrace_state from log.c into lightningd.c, so
we can hand it to memleak_init().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
c956d9f5eb lightningd: tal memleak detection, dev-memleak command.
This is a primitive mark-and-sweep-style garbage detector.  The core is
in common/ for later use by subdaemons, but for now it's just lightningd.
We initialize it before most other allocations.

We walk the tal tree to get all the pointers, then search the `ld`
object for those pointers, recursing down.  Some specific helpers are
required for hashtables (which stash bits in the unused pointer bits,
so won't be found).

There's `notleak()` for annotating things that aren't leaks: things
like globals and timers, and other semi-transients.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
95df553813 lightningd: add dev-memdump command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
8c665835fa jsonrpc: make explicit call to indicate cmd is still pending.
jsonrpc handlers usually directly call command_success or
command_fail; not doing that implies they're waiting for something
async.

Put an explicit call (currently a noop) there, and add debugging
checks to make sure it's used.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Christian Decker
2988e290cf gossip: Added internal gossip message to add a local channel
Couldn't find a good place to put these messages, we probably want to
do the same capability based request routing that we did for the HSM,
but for now this just defines the message in the master messages file.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-20 06:14:53 +00:00
Rusty Russell
477a529856 pay: make sure we don't think payment in progress if it immediately fails.
If send_htlc_out() fails, it doesn't initialize pc->out; that can
make us think it's still in progress.

Reported-by: Jonas Nick
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-19 14:40:26 +01:00
Christian Decker
a8a6d1d669 channel: Directly send announcements and updates to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Rusty Russell
94711969f9 gossipd: hand out gossip_index to other daemons.
When gossipd sends a message, have a gossip_index.  When it gets back a
peer, the current gossip_index is included, so it can know exactly where
it's up to.

Most of this is mechanical plumbing through openingd, channeld and closingd,
even though openingd and closingd don't (currently) read gossip, so their
gossip_index will be unchanged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
67aa95c194 gossipd: hand back peer, don't hand a new peer.
All peers come from gossipd, and maintain an fd to talk to it.  Sometimes
we hand the peer back, but to avoid a race, we always recreated it.

The race was that a daemon closed the gossip_fd, which made gossipd
forget the peer, then master handed the peer back to gossipd.  We stop
the race by never closing the gossipfd, but hand it back to gossipd
for closing.

Now gossipd has to accept two fds, but the handling of peers is far
clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
b83ac58a98 subd: if a required daemon exits, wait instead of killing it.
Otherwise we always say it died because we killed it, so we don't get
the exit status.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
6b232de7b1 openingd: return to master for more gossip when negotiation fails.
We can open other channels, if we want.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
899bf3fde9 subd: add transaction to subd exit corner case.
As demonstrated in the test at the end of this series, openingd dying
spontaneously causes the conn to be freed which causes the subd to be
destroyed, which fails the peer, which hits the db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
20d7ddcb6e json_fund_channel: give more details than "peer died".
Rather than using the destructor, hook up the cmd so we can close it.
peers are allocated off ld, so they are only destroyed explicitly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Christian Decker
b5f737e563 gossip: Passing alias, color and wireaddrs through to gossipd
First step towards `gossipd` managing the `node_announcement`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker
67c6d4d1f1 opts: Change alias to be u8*, better matches the unicode nature
We are still generating only char* style aliases, but the field is
defined to be unicode, which doesn't mix too well with char.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Rusty Russell
65fd7ce132 bitcoind: don't leak memory on every call to bitcoin-cli.
Fixes: #412
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-13 22:46:10 +01:00
Rusty Russell
b00525b8c8 options: don't leak memory when people override options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-13 22:46:10 +01:00
Rusty Russell
0610f66c34 bolt11: handle r value fee spec change.
We don't use it yet, but now we'll decode correctly.

See: https://github.com/lightningnetwork/lightning-rfc/pull/317
lightning-rfc commit: ef053c09431442697ab46e83f9d3f86e3510a18e

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-12 11:45:44 +01:00
practicalswift
4452e3f3f1 Remove redundant code 2017-12-11 03:38:37 +00:00
ZmnSCPxj
7c09b6f478 Load defaults for purpose of help. 2017-12-11 03:21:07 +00:00
practicalswift
2ed53025f9 Remove redundant remote_shutdown_scriptpubkey assignment 2017-12-09 16:17:58 +01:00
practicalswift
61c47c09d0 Fix typos 2017-12-08 13:07:20 +01:00
Rusty Russell
5444084625 onchaind: fix reversed CSV delays.
Our config -> their CSV delay, their config -> our CSV delay.

Fixes: #395
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-04 13:00:57 +01:00
Rusty Russell
af5dbbc9f8 json_connect: separate port arg so we can parse IPv6 addresses.
Fixes: #391
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-04 11:15:59 +01:00
Rusty Russell
0069787eb6 lightningd: fix crash on bad connect host arg.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-04 11:15:59 +01:00
Christian Decker
9ba99d2b2d hsm: Cleanup after merging control and client libraries
Change all calls to use the correct serialization and deserialization
functions, include the correct headers and remove the control
messages.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-03 17:06:13 +01:00
Christian Decker
8ff69e0307 hsm: channeld now also just uses the handle_client entry point
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-03 17:06:13 +01:00
Christian Decker
6ce264d3a2 hsm: Remove special case of hsmfd_ecdh, replaced with client and cap
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-03 17:06:13 +01:00
Rusty Russell
933c1794a1 gossipd: gossip_get_update / gossip_get_update_reply from channeld.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:54:12 +01:00
Rusty Russell
2bed246e10 channeld: generate error messages instead of having master do it.
The master now hands channeld either an error code, and channeld
generates the error message, or an error message relayed from another
node to pass through.

This doesn't fill in the channel_update yet: we need to wire up gossipd
to give us that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:54:12 +01:00
Rusty Russell
ee8274b7a1 channeld: handle wrapping error messages messages.
Currently lightningd does this, but channeld is perfectly capable of doing it.
channeld is also in a far better position to add channel_updates to it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:54:12 +01:00
Rusty Russell
be35895847 lightningd: instead of malformed code, keep generic error code.
This is in preparation for generating the actual error messages inside
channeld.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:54:12 +01:00
Rusty Russell
06d7c40c63 lightningd: make minimum feerate significantly lower.
estimatesmartfee 4 ECONOMICAL was too high for lnd, so drop it, with some
increased security risk.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-01 18:30:57 +01:00
Christian Decker
00cad3a2ff fixup! txfilter: Add a simple transaction filter 2017-11-29 14:39:12 +01:00
Christian Decker
c29923a623 topology: Add transaction filtering to connect_block
The filter is being populated while initializing the daemon and by
adding new keys as they are being generated. The filter is then used
in connect_block to identify transactions of interest.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-29 14:39:12 +01:00
Christian Decker
1eb3c9d2b3 txfilter: Add a simple transaction filter
This is mainly used to filter for transactions that may be of interest
to us, i.e., whether one of our keys is the recipient. It currently
does onyl simple scriptpubkey checks, but will eventually be extended
to use bloomfilters and add more sophisticated checks.

For now the goal is to speed up the processing of blocks during startup.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-29 14:39:12 +01:00
Christian Decker
26aa0e2aa4 topogoly: Extract owned funds from transactions in blocks
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-29 14:39:12 +01:00
Christian Decker
9c4f075c6e topogoly: Reduce log noise
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-29 14:39:12 +01:00
Rusty Russell
af7e6158af Makefile: clean needs to do more, distclean should remove everything.
I checked this with git status --ignored after a full build and 'make distclean'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:29:03 +01:00
Rusty Russell
18db1105c7 json_invoice: return absolute expiry_time
Requested-by: Nadav Ivgi <nadav@shesek.info>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
Rusty Russell
45f5bb7fac bolt11: move to common/ and sign via callback.
JSON stuff is moved to lightningd/invoice.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
Rusty Russell
78d0a5e840 bech32: move to common/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
Christian Decker
79443d6f94 htlc: Defer saving the outgoing payment until we store the HTLC
This addresses a performance regression introduced by
6ceb375650. We were storing it in an
otherwise empty DB transaction, which means that DB transaction was no
longer a no-op. Now we defer storing until we need to store the
corresponding HTLC anyway, so we can just piggyback on top of that
transaction.

This is also more consistent since we'd be forgetting the payment
anyway if we restart between adding the HTLC and committing to it.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-23 23:45:28 +00:00
Rusty Russell
1a38c98a7e chaintopology: implement dev-setfees to force feerate changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
552e56d002 channeld: send update_fee messages.
We only send them when we're not awaiting revoke_and_ack: our
simplified handling can't deal with multiple in flights.

Closes: #244
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
062f18127d channeld: handle incoming feerate changes.
We allow from the 100-confirm economical rate to 5x the immediate rate.

Closes: #243
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
b836b452dc feerate: keep feerates separately for each side.
When we support changing them, they can be different during the transition.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
24b4326b30 onchaind: use normal feerate, not last feerate from transaction.
This is used by onchaind when creating its own transactions.  Note we
don't send updates yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
ccf86a138a chaintopology: add notify_feerate_change() callback.
We'll use this to tell peers to change feerate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
d44088db83 fund-channel: use real feerate.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
e2287ce545 openingd: use real feerates.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
f1e4cad9d4 feerate: use u32 everywhere.
The wire protocol uses this, in the assumption that we'll never see feerates
in excess of 4294967 satoshi per kiloweight.

So let's use that consistently internally as well.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
88af0f5bf8 tests: change --override-fee-rate to --override-fee-rates and use in tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
7151c65535 chaintopology: track three different feerates.
Depending on what we're doing, we can want different ones.  So use
IMMEDIATE (estimatesmartfee 2 CONSERVATIVE), NORMAL (estimatesmartfee
4 ECONOMICAL) and SLOW (estimatesmartfee 100 ECONOMICAL).

If one isn't available, we try making each one half the previous.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
ef4d54df94 chaintopology: use satoshi-per-kw everywhere.
This means we convert it when retrieving from bitcoind; internally it's
always satoshi-per-1000-weight aka millisatoshi-per-weight.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
b954662f06 bitcoind: use estimatesmartfee.
estimatefee is deprecated.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
35a6ab8151 invoice: don't allow payments in expired invoices.
Fixes: #363
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 00:11:04 +01:00
Rusty Russell
1e6d5d82aa invoice: record expiry in data structure.
We don't enforce it yet, but you can query it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 00:11:04 +01:00
Rusty Russell
2d05644dfd json_invoice: add optional expiry argument.
This just includes it in the bolt11 return for the moment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 00:11:04 +01:00
Rusty Russell
fbc5ba12e4 invoice: simplify find_invoice_by_label.
Passing a list_head is a remnant of when we had two lists.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 00:11:04 +01:00
Rusty Russell
70f15fb068 json_invoice: remove 'r' argument.
It was only for debugging early on.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 00:11:04 +01:00
Rusty Russell
40315bfb91 test: fix dependencies.
Test objects must be added to $(ALL_OBJS) so they correctly depend on
CCAN headers etc.

Also, each test in a subdir must depend on headers and src in the parent
directory, as it will often #include them directly.

Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell
37b4ab306e run-channel.c: move under channeld/
This also fixes dependencies, since it actually depends on channeld objects.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell
593d0fa625 lightningd: initialize dev_subdaemon_fail to false.
Our testsuite uses --dev-fail-on-subdaemon-fail, so I didn't notice this
until I turned that off to chase a bug.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell
5fb4577890 channeld/full_channel: fix incorrect reutrn from channel_rcvd_revoke_and_ack.
It was always returning false; it was supposed to return true if
we had added pending changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Christian Decker
77497b526f jsonrpc: Add listpayments method
Returns a list of transfers and their state.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-20 06:28:03 +00:00
Christian Decker
e0d86376e2 wallet: Track outgoing payments in the database
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-20 06:28:03 +00:00
Christian Decker
6ceb375650 wallet: Record issued invoices in the payments table
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-20 06:28:03 +00:00
Rusty Russell
5281541ec6 closingd: start with proper maximum fee, not our guesstimate.
Fixes: #348
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:41:45 +01:00
Rusty Russell
46f2e17905 openingd: update to BOLT with htlckey.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell
fc05779f78 subdaemons: pass back and forth the htlc points.
Openingd sets it to the same as the payment point for the remote side.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell
654c2f4eb8 onchaind: use HTLC key for htlc signatures.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell
4db460903a htlc_tx: wire up the htlc points.
All the callers need to pass it in: currently channeld and openingd just
fake it by copying the payment point.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell
fe5614a489 basepoints/secrets: add htlc entry
Currently derive_basepoints just sets it to match the payment point/secret.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Christian Decker
adfe494b08 invoice: Fix waitanyinvoice
There were two bugs: we weren't returning the next from the given
label but the one matching the label, and we were appending new
invoices to the head instead of the tail, which meant we'd be
traversing in the wrong order.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-15 00:07:44 +00:00
Rusty Russell
f95afc55d8 rpc: report netaddr as array.
Thought we don't handle it at the moment, nodes can certainly have multiple
addresses, and we should display them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-12 12:16:22 +01:00
Christian Decker
89f016f524 jsonrpc: Only print netaddr in getpeers when we know it
Fixes #285
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-12 12:16:22 +01:00
Rusty Russell
956350e62e lightningd: check peers don't leave dangling HTLCs when they die.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-11 01:29:51 +00:00
Rusty Russell
e9337820a0 onchaind: remove htlcs when peer is irrevocably committed.
We don't track them accurately when in onchaind, but we don't want to:
onchaind can be restarted at any time.

Once it's all settled, we're clear to clean them up.

Before this, valgrind could complain about deferncing hout->key.peer:

Valgrind error file: valgrind-errors.10876
==10876== Invalid read of size 4
==10876==    at 0x41F8AF: peer_on_chain (peer_control.h:127)
==10876==    by 0x42340D: notify_new_block (peer_htlcs.c:1461)
==10876==    by 0x40A08D: connect_block (chaintopology.c:96)
==10876==    by 0x40A96B: topology_changed (chaintopology.c:313)
==10876==    by 0x40AC85: add_block (chaintopology.c:384)
==10876==    by 0x40ABF0: gather_previous_blocks (chaintopology.c:363)
==10876==    by 0x4051B3: process_rawblock (bitcoind.c:410)
==10876==    by 0x4044DD: bcli_finished (bitcoind.c:155)
==10876==    by 0x454665: destroy_conn (poll.c:183)
==10876==    by 0x454685: destroy_conn_close_fd (poll.c:189)
==10876==    by 0x45DF89: notify (tal.c:240)
==10876==    by 0x45E43A: del_tree (tal.c:400)
==10876==  Address 0x6929208 is 2,120 bytes inside a block of size 2,416 free'd
==10876==    at 0x4C2EDEB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==10876==    by 0x45E513: del_tree (tal.c:421)
==10876==    by 0x45E849: tal_free (tal.c:509)
==10876==    by 0x41A8E9: handle_irrevocably_resolved (peer_control.c:1172)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-11 01:29:51 +00:00
Christian Decker
07e5a9ef9f htlc: Allow for exactly min_final_cltv_expiry cltv delta
We are announcing that we are willing to accept incoming payments with
current_height + min_final_cltv_expiry + slack, assuming that the
sender adds some slack. In particular we'd reject the payment if
slack=0 which is allowed by the spec.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-11-11 01:28:03 +00:00
Rusty Russell
4fb472b7a7 db: we must always be in a transaction, remove nested, call fatal()
We save location where transaction was started, in case we try to nest.
There's now no error case; db_exec_mayfail() is the only one.

This means the tests need to override fatal() if they want to intercept
these errors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-06 10:24:34 +01:00
Rusty Russell
7133a2f9b3 wallet: assume db errors will be fatal, don't check.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-06 10:24:34 +01:00
Rusty Russell
bccd2f8cf4 bitcoind: wrap callbacks in transaction.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-06 10:24:34 +01:00
Rusty Russell
8229769438 lightningd: do initial database load within a transaction.
Safest, and we can then assert that all db calls are in transactions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-06 10:24:34 +01:00
Rusty Russell
7586f3ed54 timers: wrap all calls in transactions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-06 10:24:34 +01:00
Rusty Russell
01361ab21b jsonrpc: wrap all calls in transaction. 2017-11-06 10:24:34 +01:00
Rusty Russell
3a596d6dda subd: wrap all message callbacks in a transaction.
Including destructors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-06 10:24:34 +01:00
Rusty Russell
8cef36cbd7 lightningd: fail htlcs we fulfill if peer unresponsive after deadline.
Closes: #241
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-03 05:18:49 +00:00
Rusty Russell
1142c44c29 lightningd: fail htlcs we offer if peer unresponsive after deadline.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-03 05:18:49 +00:00
Rusty Russell
9662589ed8 lightningd: move notify_new_block() callback to peer_htlcs.
And change prototype to take the lightningd structure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-03 05:18:49 +00:00
Rusty Russell
bdabb24734 lightningd: note our complience on not forwarding an HTLC post deadline.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-03 05:18:49 +00:00
Rusty Russell
c14b159166 lightningd: remove --deadline-blocks option.
We will derive it from other factors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-03 05:18:49 +00:00
Rusty Russell
a55ce607a1 bitcoind: contain ld pointer.
This is a subset of a "bitcoind: wrap callbacks in transaction." from
the everything-in-transaction branch, but we need the ld pointer now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-03 05:18:49 +00:00
Rusty Russell
860a76b1c9 lightningd: update to add wire_expiry_too_far.
From recently-merged BOLT update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 09:46:05 +01:00
Rusty Russell
112ae0d0f5 common/test/run-json: test JSON escaping.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 09:46:05 +01:00
Rusty Russell
b7774fcb0a bolt11: comment on weird assignment-then-check test.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
a9002eac52 Update to latest BOLT.
And nail "make check-source" to that specific version (which is a commit id,
not a branch name, so needs a different syntax for git).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
6c21da69e6 bolt11: 'c' support for min_final_cltv_expiry.
Based on latest draft spec, using variable length encoding.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
e1dc75e954 pay: new all-in-one RPC command.
Closes: #240
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
d577769350 invoice: provide bolt11 invoice.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
8d09734536 bolt11: add decodepay RPC helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
15e1e4b099 bolt11: support for encoding/decoding and checking.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
9ec5cb7ba2 script: enhance is_p2sh/is_p2pkh/is_p2wsh/is_p2wpkh to extract addr.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
7ed81cdf7b pay: factor out actual payment sending from json part of sendpay.
We're going to reuse this for the new 'pay' all-in-one command.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
79df10f782 bech32: pull in bech32 sample code, almost untouched.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
c2a0c51c30 lightningd: check for excessive HTLC locktimes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell
0000dc5908 lightningd: Allow in excess of ctlv_expiry.
We add a test which both overpays and over-sets the delta on the
intermediate and final nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell
71e794a046 lightningd: split ctlv_expiry and final_cltv.
These need to be different for testing the example in BOLT 11.

We also use the cltv_final instead of deadline_blocks in the final hop:
various tests assumed 5 was OK, so we tweak utils.py.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell
68dae5648d gossipd: route correctly using final CLTV value.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell
bc9918ad46 dev: option not to do backtracing.
It crashes under valgrind, causing a valgrind error: valgrind gives us a
backtrace anyway, so we don't need it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 04:14:33 +00:00
Rusty Russell
21305c0d28 fatal: cause a backtrace.
Much nicer for debugging.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 04:14:33 +00:00
Rusty Russell
82f252c79a test_permfail_new_commit: fix intermittant failure.
Normally, we get an error as soon as we send WIRE_REVOKE_AND_ACK.  But if the
commit timer goes off, we get some extra cycles, during which the other side
can reconnect.  In this case, we simply kill the channeld before it fails,
and never check for the permfail string.

    b'lightning_channeld(18613): TRACE: dev_disconnect: -WIRE_REVOKE_AND_ACK'
    b'lightning_channeld(18613): TRACE: Trying commit'
    b'lightning_channeld(18613): TRACE: htlc 0: SENT_ADD_REVOCATION->SENT_ADD_ACK_COMMIT'
    b'lightning_channeld(18613): TRACE: htlc added REMOTE: local +0 remote -200000000'
    b'lightning_channeld(18613): TRACE: sending_commit: HTLC REMOTE 0 = SENT_ADD_ACK_COMMIT/RCVD_ADD_ACK_COMMIT'
    b'lightning_gossipd(18590): TRACE: Responder: Act 1'
    b'lightning_channeld(18613): TRACE: Derived key 034aab0b5cb755de836cffb34c053ba115fba6fe75414e8f56261e23c80eabb1fe from basepoint 03e0a7bb422b254f54bc954be05bd6823a7b7a4b996ff8d3079ca211590fb5df39, point 02f3bf525b6ca595bf85d63e89c95fc59c0fde3ae434b55c8093bbb5c64849da37'
    b'lightningd(18465): Connected json input'
    b'lightningd(18465):jcon fd 16: Success'
    b'lightningd(18465):jcon fd 16: Closing (Bad file descriptor)'
    b'lightning_gossipd(18590): TRACE: Responder: Act 2'
    b'lightning_gossipd(18590): TRACE: Responder: Act 3'
    b'lightning_gossipd(18590): UPDATE WIRE_GOSSIP_PEER_CONNECTED'
    b'lightning_gossipd(18590): UPDATE WIRE_GOSSIP_PEER_CONNECTED'
    b'lightningd(18465): peer 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518: Peer has reconnected, state CHANNELD_NORMAL'
    b'lightning_channeld(18613): Status closed, but not exited. Killing'

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-28 13:33:00 +02:00
Rusty Russell
0c7ca9ab7c gossipd: call to return all connected peers.
And we report these through the getpeers JSON RPC again (carefully: in
our reconnect tests we can get duplicates which this patch now filters
out).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
a7d6326bef type_to_string: format wireaddr.
Good for printing, and removes some code from peer_control.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
78cd25d620 ipaddr: rename to wireaddr.
In future it will have TOR support, so the name will be awkward.

We collect the to/fromwire functions in common/wireaddr.c, and the
parsing functions in lightningd/netaddress.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
4bd0352951 lightningd: try to figure out our own IP automatically.
Most of the code is from bitcoind, to handle the weird different non-public
IP ranges.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
329269d9d0 lightningd: support multiple addresses.
Currently only ipv4 and ipv6.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
bd1cac34ce netaddr: remove.
We use ipaddr everywhere now, so we can remove this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
dfd60a2047 gossipd: tell the master the peer's address.
This will let us remove peer->netaddr.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
ac92138603 common: remove unused assert() headers.
Auditing for assert/abort in common/ code used by lightningd, this is all
that showed up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell
81db5896e1 common/json: remove asserts() which may trigger from user input.
They don't currently, since callers check, but be safe.  In addition,
handle NULL returns from these in the bitcoind code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell
11b43a422b lightningd: close one possibly-reachable abort.
There are others, but they really are casued by bad failure.  We need a
parachute system for these.

Closes: #176
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell
3c6eec87e3 Add DEVELOPER flag, set by default.
This is a bit messier than I'd like, but we want to clearly remove all
dev code (not just have it uncalled), so we remove fields and functions
altogether rather than stub them out.  This means we put #ifdefs in callers
in some places, but at least it's explicit.

We still run tests, but only a subset, and we run with NO_VALGRIND under
Travis to avoid increasing test times too much.

See-also: #176
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell
a2dc71b0a1 lightningd: close a take() leak.
test_routing_gossip (__main__.LightningDTests) ... lightningd: Outstanding taken pointers: lightningd/peer_control.c:2352:towire_errorfmt(ld, ((void *)0), "Can't resolve your address")

This caused by the other end closing due to the next bug.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-25 18:34:35 +02:00
Rusty Russell
7d62de8632 lightningd: fix typo in fatal error.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-25 18:34:35 +02:00
Rusty Russell
b6a2b8c58b Add --rgb and --alias options.
And derive random ones from nodeid if they don't choose.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-25 09:16:14 +00:00
Rusty Russell
ebdecebb1a channeld: send channel_announce and initial update to master, not gossipd.
There is a race we see sometimes under valgrind on Travis which shows
gossipd receiving the node_announce from master before it reads the
channel_announce from channeld, and thus fails.  The simplest solution
is to send the channel_announce and channel_update to master as well,
so it can ensure it sends them to gossipd in order

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell
7f38943956 options: show the default network setting in --help.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell
7e022b522c gossipd: don't try to handle padding inside fromwire_ipaddr.
It makes it impossible to embed an ipaddr in another structure, since we
always try to skip over any zeroes, which may swallow a following field.

Do the skip specially for the case where we're parsing routing messages:
we never use padding for our own internal messages anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell
79962b3588 lightningd: return transaction from fundchannel RPC.
Lets tests figure out the short channel name, for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell
48cedef756 peer_control: remove unique_id field.
It's now completely useless.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell
ffaa15c7da hsm: remove unique_id.
It was only for error messages, so replace it with pubkey.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell
c3bed51b2d test_lightningd.py: make HSM seeds constant for tests.
Makes it easier to compare before/after failures.  Ideally, we should
run under Travis both with this option and with the seed based on the
entire tmp path (which is still reproducible with determination, but
not fixed every run like this is).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell
4c9f7542b2 subd: Clarify description of subd_release_peer.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-22 16:24:10 +02:00
Rusty Russell
1954844fbf lightningd: make peer_fail_permanent() only save the first error for peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
0b953b86fe subd: automatically detect if callback frees subd.
This involves a tricky callback internally, but far less error-prone.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00