Commit Graph

143 Commits

Author SHA1 Message Date
Rusty Russell
96f05549b2 common/utils.h: add tal_arr_expand helper.
We do this a lot, and had boutique helpers in various places.  So add
a more generic one; for convenience it returns a pointer to the new
end element.

I prefer the name tal_arr_expand to tal_arr_append, since it's up to
the caller to populate the new array entry.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Rusty Russell
168bec0974 lightningd: move channel/peer/htlc load into own function.
Also, wallet has no business wiring up HTLCs; move that code to
peer_htlcs.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-03 05:01:40 +00:00
Rusty Russell
807e62b05d moveonly: move feerate routines from peer_htlcs.c to channel_control.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 02:17:51 +00:00
Rusty Russell
6338ae8a44 channeld: update fees if we're restarting.
This is a noop if we're opening a new channel (channel_fees_can_change(channel)
is false until funding locked in), but important if we're restarting.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 02:17:51 +00:00
Rusty Russell
b6a63024c1 lightningd: fix double-free on multiple HTLC timeouts.
We can close a connection with a peer to timeout an HTLC, but
we need to clear the pointer otherwise next time we try, we'll
free an expired pointer:

```
lightningd: Fatal signal 6 (version v0.6-336-gfcd1eb5-modded)
0x13ce86 crashdump
	common/daemon.c:37
0x5739f1f ???
	???:0
0x5739e97 ???
	???:0
0x573b800 ???
	???:0
0x1850c3 call_error
	ccan/ccan/tal/tal.c:93
0x18528b check_bounds
	ccan/ccan/tal/tal.c:165
0x1852ca to_tal_hdr
	ccan/ccan/tal/tal.c:174
0x185bfb tal_free
	ccan/ccan/tal/tal.c:472
0x1343a8 peer_sending_commitsig
	lightningd/peer_htlcs.c:1035
0x114f25 channel_msg
	lightningd/channel_control.c:159
0x13756b sd_msg_read
	lightningd/subd.c:474
0x177c1f next_plan
	ccan/ccan/io/io.c:59
0x178717 do_plan
	ccan/ccan/io/io.c:387
0x178755 io_ready
	ccan/ccan/io/io.c:397
0x17a336 io_loop
	ccan/ccan/io/poll.c:310
0x120589 main
	lightningd/lightningd.c:455
0x571cb96 ???
	???:0
0x10e6d9 ???
	???:0
0xffffffffffffffff ???
	???:0
2018-08-16T06:41:21.249Z lightningd(869): FATAL SIGNAL 6 (version v0.6-336-gfcd1eb5-modded)
2018-08-16T06:41:21.250Z lightningd(869): backtrace: common/daemon.c:42 (crashdump) 0x13ceda
2018-08-16T06:41:21.250Z lightningd(869): backtrace: (null):0 ((null)) 0x5739f1f
2018-08-16T06:41:21.250Z lightningd(869): backtrace: (null):0 ((null)) 0x5739e97
2018-08-16T06:41:21.251Z lightningd(869): backtrace: (null):0 ((null)) 0x573b800
2018-08-16T06:41:21.251Z lightningd(869): backtrace: ccan/ccan/tal/tal.c:93 (call_error) 0x1850c3
2018-08-16T06:41:21.251Z lightningd(869): backtrace: ccan/ccan/tal/tal.c:165 (check_bounds) 0x18528b
2018-08-16T06:41:21.252Z lightningd(869): backtrace: ccan/ccan/tal/tal.c:174 (to_tal_hdr) 0x1852ca
2018-08-16T06:41:21.252Z lightningd(869): backtrace: ccan/ccan/tal/tal.c:472 (tal_free) 0x185bfb
2018-08-16T06:41:21.252Z lightningd(869): backtrace: lightningd/peer_htlcs.c:1035 (peer_sending_commitsig) 0x1343a8
2018-08-16T06:41:21.252Z lightningd(869): backtrace: lightningd/channel_control.c:159 (channel_msg) 0x114f25
2018-08-16T06:41:21.253Z lightningd(869): backtrace: lightningd/subd.c:474 (sd_msg_read) 0x13756b
2018-08-16T06:41:21.253Z lightningd(869): backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x177c1f
2018-08-16T06:41:21.253Z lightningd(869): backtrace: ccan/ccan/io/io.c:387 (do_plan) 0x178717
2018-08-16T06:41:21.253Z lightningd(869): backtrace: ccan/ccan/io/io.c:397 (io_ready) 0x178755
2018-08-16T06:41:21.253Z lightningd(869): backtrace: ccan/ccan/io/poll.c:310 (io_loop) 0x17a336
2018-08-16T06:41:21.253Z lightningd(869): backtrace: lightningd/lightningd.c:455 (main) 0x120589
2018-08-16T06:41:21.254Z lightningd(869): backtrace: (null):0 ((null)) 0x571cb96
2018-08-16T06:41:21.254Z lightningd(869): backtrace: (null):0 ((null)) 0x10e6d9
2018-08-16T06:41:21.254Z lightningd(869): backtrace: (null):0 ((null)) 0xffffffffffffffff
Log dumped in crash.log
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 16:55:15 +02:00
Rusty Russell
9f175deecd lightningd: update feerate upon receiving revoke_and_ack from fundee.
1. l1     update_fee ->    l2
2. l1 commitment_signed -> l2 (using new feerate)
3. l1  <- revoke_and_ack   l2
4. l1 <- commitment_signed l2 (using new feerate)
5. l1  -> revoke_and_ack   l2

When we break the connection after #3, the reconnection causes #4 to
be retransmitted, but it turns out l1 wasn't telling the master to set
the local feerate until it received the commitment_signed, so on
reconnect it uses the old feerate, with predictable results (bad
signature).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-22 18:54:53 +02:00
Mark Beckwith
a3178b8177 param: remove old callback code
Cleaned up remaining code. Reduced comment noise. Reverted
macro names back to the original.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith
9b28ecf8fc param: upgraded json_tok_pubkey
Also add json_to_pubkey as utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith
8ebc95b7b0 param: upgraded json_tok_bool
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Rusty Russell
65c882ca3a Minor cleanups.
1. connect convenience variable for improved readabilty.
2. a comment explaining that timer is on channel, not HTLC.
3. use modern python style in test_htlc_send_timeout

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell
223cd97c94 lightningd: kill channeld if we added an HTLC and it didn't commit in 30 seconds.
This effectively constrains how long we'll delay an outgoing HTLC.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell
5cf34d6618 Remove tal_len, use tal_count() or tal_bytelen().
tal_count() is used where there's a type, even if it's char or u8, and
tal_bytelen() is going to replace tal_len() for clarity: it's only needed
where a pointer is void.

We shim tal_bytelen() for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-30 11:31:17 +02:00
Christian Decker
fbbc5899e4 invoices: Make the invoice_details more idiomatic
This seems like a premature optimization: it tried to cut down the number of
allocations by reusing the same `struct invoice_details` while iterating through
a number of results. But this sidesteps the checks by `valgrind` and we'd miss a
missing field that was set by the previous iteration.

Reported-by: @rustyrussell
Signed-off-by: Christian Decker <@cdecker>
2018-07-30 03:04:45 +00:00
Rusty Russell
162879d6a2 channeld: use fulfilled_htlc and failed_htlc msgs in single htlc case.
We use these for receiving arrays at init time, we should also use them
for fulfull/fail of HTLCs in normal operation.  That we we benefit from all
those assertions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell
1119dd5577 channeld: always receive and maintain short_channel_id of failing channel.
The master tells us the short_channel_id of the outgoing channel when
failing an HTLC, but channeld didn't store it anywhere.  It also
didn't tell channeld the short_channel_id in the case where we're
reconnecting and it's feeding us an array of failed htlcs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Mark Beckwith
f850849486 Modern param style for all remaining files
Removed `json_get_params`.

Also added json_tok_percent and json_tok_newaddr. Probably should
have been a separate PR but it was so easy.

[ Squashed comment update for gcc workaround --RR ]
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-20 01:14:02 +00:00
Rusty Russell
e217bc1220 per-commit-secret is a struct secret, not a sha256.
Well, it's generated by shachain, so technically it is a sha256, but
that's an internal detail.  It's a secret.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell
68a8eeea21 htlc_wire: rename malformed to failcode in struct failed_htlc.
I'm not completely convinced that it's only ever set to a failcode
with the BADONION bit set, especially after the previous patches in
this series.  Now that channeld can handle arbitrary failcodes passed
this way, simply rename it.

We add marshalling assertions that only one of failcode and failreason
is set, and we unmarshal an empty 'fail' to NULL (just the the
generated unmarshalling code does).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell
6e9ae98e1e lightningd: don't send uninialized malformed fields to channeld.
==1224== Uninitialised byte(s) found during client check request
==1224==    at 0x152CAD: memcheck_ (mem.h:247)
==1224==    by 0x152D18: towire (towire.c:17)
==1224==    by 0x152DA1: towire_u16 (towire.c:28)
==1224==    by 0x142189: towire_failed_htlc (htlc_wire.c:29)
==1224==    by 0x16343F: towire_channel_init (gen_channel_wire.c:596)
==1224==    by 0x115C2C: peer_start_channeld (channel_control.c:249)
==1224==    by 0x131701: peer_connected (peer_control.c:503)
==1224==    by 0x117820: gossip_msg (gossip_control.c:182)
==1224==    by 0x139D97: sd_msg_read (subd.c:500)
==1224==    by 0x139676: read_fds (subd.c:327)
==1224==    by 0x179D52: next_plan (io.c:59)
==1224==    by 0x17A84F: do_plan (io.c:387)
==1224==  Address 0x1ffefffabe is on thread 1's stack
==1224==  in frame #2, created by towire_u16 (towire.c:26)

Followed by:

2018-06-18T21:53:04.129Z lightningd(1224): 03933884aaf1d6b108397e5efe5c86bcf2d8ca8d2f700eda99db9214fc2712b134 chan #1: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel d0101486543e1a8b6871556a4fe1fba4ad4d83ce7f6f92919fd17bd1545d2fd5: UpdateFailMalformedHtlc message doesn't have BADONION bit set

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell
fed5a117e7 Update ccan/structeq.
structeq() is too dangerous: if a structure has padding, it can fail
silently.

The new ccan/structeq instead provides a macro to define foo_eq(),
which does the right thing in case of padding (which none of our
structures currently have anyway).

Upgrade ccan, and use it everywhere.  Except run-peer-wire.c, which
is only testing code and can use raw memcmp(): valgrind will tell us
if padding exists.

Interestingly, we still declared short_channel_id_eq, even though
we didn't define it any more!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-04 23:57:00 +02:00
Rusty Russell
e549bc6ecf lightningd: fix up BOLT references.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Mark Beckwith
7f437715d5 Added error code parameter to command_fail
Until now, `command_fail()` reported an error code of -1 for all uses.
This PR adds an `int code` parameter to `command_fail()`, requiring the
caller to explicitly include the error code.

This is part of #1464.

The majority of the calls are used during parameter validation and
their error code is now JSONRPC2_INVALID_PARAMS.

The rest of the calls report an error code of LIGHTNINGD, which I defined to
-1 in `jsonrpc_errors.h`.  The intention here is that as we improve our error
reporting, all occurenaces of LIGHTNINGD will go away and we can eventually
remove it.

I also converted calls to `command_fail_detailed()` that took a `NULL` `data`
parameter to use the new `command_fail()`.

The only difference from an end user perspecive is that bad input errors that
used to be -1 will now be -32602 (JSONRPC2_INVALID_PARAMS).
2018-05-26 12:17:36 +02:00
ZmnSCPxj
097a8e72d1 channel_control: Forget if unconfirmed for a long time and we are fundee.
We should forget this as it is a potential DoS if we remember every
funding txid that an attacker gave in a `funding_created` but never
broadcasted.
2018-05-23 14:37:32 -07:00
Rusty Russell
edf1b3cec9 More option cleanups.
Because we have too many which are never used and I don't want to document
them.

1. Remove unused anchor_onchain_wait.  When implemented, it should be
   hardcoded to 100 or more.
2. Remove anchor_confirms_max.  10 always reasonable, and we can readd
   an override option should someone need it.
3. max_htlc_expiry should be the same as locktime_max (which increases
   from 3 to 5 days by default): they're both a limit on how long
   funds can be locked up.
4. channel_update_interval should always be a dev option.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
ZmnSCPxj
f83c4ff903 wallet: Add msatoshi_to_us_min and msatoshi_to_us_max statistics for channels.
So we know how much counterparty could theoretically steal from us
 (msatoshi_to_us - msatoshi_to_us_min) and how much we could
 theoretically steal from counterparty (msatoshi_to_us_max -
 msatoshi_to_us).
For more piloting goodness.
2018-04-05 19:01:53 +02:00
Rusty Russell
5f1c77d249 test_lightning.py: add test for onchain with different feerates.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-04 02:31:41 +00:00
Rusty Russell
6bb47276ce lightningd: put min/max feerates into db, struct channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-04 02:31:41 +00:00
ZmnSCPxj
0bb9bcc0f1 wallet: Track some channel usage statistics.
Fixes: #1049
2018-03-26 01:08:52 +00:00
Rusty Russell
76e8a11380 wallet: use json_escaped for invoice label.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
practicalswift
98f49c0837 Remove include in file foo.c that is already included in foo.h 2018-03-25 23:54:21 +00:00
Rusty Russell
0a6e3d1e13 utils: remove tal_tmpctx altogether, use global.
In particular, we now only free tmpctx at the end of main().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
ZmnSCPxj
61882ed9df pay: Add hook for triggering storage of payments. 2018-03-14 05:33:09 +00:00
Corné Plooy
b857b2e843 Add assertions in various places to ensure tal_fmt doesn't receive NULL as argument for strings. 2018-03-06 19:26:21 +01:00
ZmnSCPxj
978e5c67d8 invoices: Remove persistent in-memory invoice structures. 2018-02-28 11:17:08 +01:00
ZmnSCPxj
9b4c6699f9 invoices: Semantically separate invoice details from invoice.
In preparation for removing in-memory invoice structures.
Invoice details are requested rarely anyway.
2018-02-28 11:17:08 +01:00
practicalswift
74841ef567 Remove unused parameter payment_hash in forward_htlc(..., const struct sha256 *payment_hash, ...) 2018-02-22 10:46:30 +01:00
practicalswift
91a9c2923f Mark intentionally unused parameters as such (with "UNUSED") 2018-02-22 01:09:12 +00:00
Rusty Russell
1192f16733 lightningd: fix crash when htlc failed and source is onchain.
If the source channel is onchain, we try to send a message to onchaind
which (1) doesn't care, (2) doesn't take a channel_fail_htlc msg, and
(3) causes us to crash in subd.c:

	assert(!strstarts(sd->msgname(fromwire_peektype(msg_out)), "INVALID"));

Fixes: #821
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:58:48 +01:00
Rusty Russell
e92b710406 tools/generate-wire.py: remove length argument from fromwire_ routines.
We always hand in "NULL" (which means use tal_len on the msg), except
for two places which do that manually for no good reason.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell
36652a3df3 lightningd/peer_control: reorder, move more functions out.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell
e0603d7221 channel: now we're always complete, fields don't have to be optional.
Now any struct channel is a genuine channel, the following fields are
always valid:

1. funding_txid: doesn't need to be a pointer.
2. our_msatoshi: doesn't need to be a pointer.
3. last_sig: doesn't need to be a pointer.
4. channel_info: doesn't need to be a pointer.

In addition, 'last_tx' is always valid.

The main effect is to remove a whole heap of branches from the wallet code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell
8db8c51201 lightningd: struct uncommitted_channel for opening channels.
Each peer can have one 'uncommitted' channel, which is in the process
of opening.  This is used for openingd, and then on return we convert
it into a full-fledged struct channel and commit it into the database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-19 02:56:51 +00:00
Rusty Russell
8f7a19d1a3 onchain: handle case where multiple HTLCs exist for same payment_hash.
We will have probably failed the others, but either way, don't try to
fulfill an HTLC we've already failed.

Fixes: #394
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:57:42 +01:00
Rusty Russell
55d962046b Rename (almost) all destructors to destroy_<type>.
We usually did this, but sometimes they were named after what they did,
rather than what they cleaned up.

There are still a few exceptions:
1. I didn't bother creating destroy_xxx wrappers for htable routines
   which already existed.
2. Sometimes destructors really are used for side-effects (eg. to simply
   mark that something was freed): these are clearer with boutique names.
3. Generally destructors are static, but they don't need to be: in some
   cases we attach a destructor then remove it later, or only attach
   to *some* cases.  These are best with qualifiers in the destroy_<type>
   name.

Suggested-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
329e31bbe7 lightningd/peer_htlcs: remove remaining peer_ shims.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
cf7c399cc5 htlc: keep channel pointer, not peer pointer.
And move the no-remaining-htlcs check from the peer destructor to the
channel destructor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
409fef582d subd: keep pointer to channel, not peer.
This rolls through many other functions, making them take channel not peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
b7680412e3 lightningd: rename peer_fail functions to channel_fail.
And move them into channel.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
32411de90e lightningd: split struct peer into struct peer and struct channel.
Much like the database; peer contains id, address, channel contains
per-channel information.  Where we create a channel, we always create
the peer too.

For the moment, peer->log and channel->log coexist side-by-side, to
reduce some of the churn.

Note that this changes the API to dev-forget-channel: if we have more
than one channel, we insist they specify the short-channel-id.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Christian Decker
9f8f0ccec9 peer: Store htlc_sigs when they are updated
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-11 01:13:07 +01:00