Commit Graph

372 Commits

Author SHA1 Message Date
Rusty Russell
bbc7a79542 lightningd: always use an onionreply for locally generated incoming HTLC errors (unless BADONION).
This cleans up the "local failure" callers for incoming HTLCs to hand
an onionreply instead of making us generate it from the code inside
make_failmsg.

(The db path still needs make_failmsg, so that's next).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-deprecated: Plugins: htlc_accepted_hook "failure_code" only handles simple cases now, use "failure_message".
2020-02-25 11:12:12 +10:30
Rusty Russell
590b2db88e lightningd: make local htlc failures pass a wiremsg for errors, not a failcode.
Unfortunately the invoice_payment_hook can give us a failcode, so I simply
restrict it to the two sensible ones.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-deprecated: plugins: invoice_payment_hook "failure_code" only handles simple cases now, use "failure_message".
2020-02-25 11:12:12 +10:30
Rusty Russell
5af3a135be lightningd: separate path for failed_htlc when an onion is bad.
We tell channeld that an htlc is bad by sending it a 'struct
failed_htlc'.  This usually contains an onionreply to forward, but for
the case where the onion itself was bad, it contains a failure code
instead.

This makes the "send a failed_htlc for a bad onion" a completely
separate code path, then we can work on removing failcodes from the
other path.

In several places 'failcode' is now changed to 'badonion' to reflect
that it can only be a BADONION failcode.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-25 11:12:12 +10:30
Rusty Russell
72d55d3e3b lightningd: store raw msg rather than code for locally-failed outgoing HTLCs
At the moment, we store e.g. WIRE_TEMPORARY_CHANNEL_FAILURE, and then
lightningd has a large demux function which turns that into the correct
error message.

Such an enum demuxer is an anti-pattern.

Instead, store the message directly for output HTLCs; channeld now
sends us an error message rather than an error code.

For input HTLCs we will still need the failure code if the onion was
bad (since we need to prompt channeld to send a completely different
message than normal), though we can (and will!) eliminate its use in
non-BADONION failure cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-25 11:12:12 +10:30
Rusty Russell
a150b09665 wallet: Add new htlc column "localfailmsg" for outgoing htlcs.
We're going to change our internal structure next, so this is preparation.
We populate existing errors with temporary node failures, for simplicity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-25 11:12:12 +10:30
Rusty Russell
cec18dfd6e lightningd: remove always-NULL argument to add_fail.
It's only called from the db code, and failing_channel is always NULL.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-25 11:12:12 +10:30
Rusty Russell
ed839bfda0 channeld: get the onionreply back from lightningd for failed htlcs.
Instead of making it ourselves, lightningd does it.  Now we only have
two cases of failed htlcs: completely malformed (BADONION), and with
an already-wrapped onion reply to send.

This makes channeld's job much simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-25 11:12:12 +10:30
Rusty Russell
7ab5c424b6 gossipd: provide (stripped) channel_update when resolving a channel.
I hadn't realized that lightningd asks gossipd every time we forward
a payment.  But I'm going to abuse it here to get the latest channel_update,
otherwise (as lightningd takes over error message generation) lightningd
needs to do an async request at various painful points.

So have gossipd tell us the lastest update (stripped so compatible with
the strange in-onion-error format).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-25 11:12:12 +10:30
Rusty Russell
c7bbdd76d3 lightningd: handle fail_htlc_in with no known outgoing channel.
Turn it into temporary node failure: this only happens if we restart
with a failed htlc in, but it's clearer and more robust to handle it
generically.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-25 11:12:12 +10:30
Rusty Russell
bb9b761dda channeld: don't get details of our own failed htlcs at init.
For incoming htlcs, we need failure details in case we need to
re-xmit them.  But for outgoing htlcs, lightningd is telling us it
already knows they've failed, so we just need to flag them failed
and don't need the details.

Internally, we set the ->fail to a dummy non-NULL value; this is
cleaned up next.

This matters for the next patch, which moves onion handling into
lightningd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-25 11:12:12 +10:30
Rusty Russell
ad3c7f3a1f lightningd: clean up weird call to send_htlc_out.
1. forward_htlc sets hout to NULL.
2. forward_htlc passes &hout to send_htlc_out.
3. forward_htlc checks the failcode and frees(NULL) and sets hout to NULL
   (again).  This in fact covers every failcode which send_htlc_out returns.

We should ensure send_htlc_out sets *houtp to NULL on failure; in fact,
both callers pass houtp, so we can make it unconditional.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-25 11:12:12 +10:30
Rusty Russell
b84b4b4695 lightningd: rename htlc_in and htlc_out failuremsg fields to failonion.
This is clearer, especially when we also deal with raw not-yet-onion-wrapped
failure messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-25 11:12:12 +10:30
Rusty Russell
fc4d01cd9d db: put scids in forwards even if we didn't actually send.
If the peer is not connected, or other error which means we don't
actually create an outgoing HTLC, we don't record the
short_channel_id.  This is unhelpful!

Pass the scid down to the wallet code, and explicitly hand the
scid and amount down to the notification code rather than handing it
the htlc_out (which it doesn't need).

Changelog-Changed: JSON API: `listforwards` now shows `out_channel` even if we couldn't forward.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-12 22:27:57 -06:00
Christian Decker
490550d508 plugin: Allow multiple plugins to register the htlc_accepted hook
Make the `htlc_accepted` hook the first chained hook in our repertoire. The
plugins are called one after the other in order until we have no more plugins
or the HTLC was handled by one of the plugins. If no plugins handles the HTLC
we continue to handle it internally like always.

Handling in this case means the plugin returns either `{"result": "resolve",
...}` or `{"result": "fail", ...}`.

Changelog-Changed: plugin: Multiple plugins can now register for the htlc_accepted hook.
2020-02-11 15:57:22 +10:30
Christian Decker
9a2a09efd6 plugin: Introduce plugin type to allow singleton and chaining
The newly introduced type is used to determine what the call semantics of the
hook are. We have `single` corresponding to the old behavior, as well as
`chain` which allows multiple plugins to register for the hook, and they are
then called sequentially (if all plugins return `{"result": "continue"}`) or
exit the chain if the hook event was handled.
2020-02-11 15:57:22 +10:30
Rusty Russell
6450a7e315 lightningd: update spec and remove allowance for final node to receive a fee.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Now correctly reject "fees" paid when we're the final hop (lightning-rfc#711)
2020-01-31 06:07:56 +00:00
Rusty Russell
295ca2a436 lightningd: remove things we deprecated 6 months ago.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Relative plugin paths are not relative to startup (deprecated v0.7.2.1)
Changelog-Removed: Dummy fields in listforwards (deprecated v0.7.2.1)
2020-01-31 03:48:01 +00:00
Rusty Russell
8eda489ae2 lightningd: don't stop processing revoke_and_ack if an HTLC is invalid.
This shouldn't happen if channeld is working properly, but I'm going to
change that, and this current code means we stop responding at that point
(not every failpath in peer_accepted_htlc() called channel_internal_error).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-29 21:15:25 +01:00
Rusty Russell
262e4c840f sphinx: use struct secret for shared secret.
Generally I prefer structures over u8, since the size is enforced at
runtime; and in several places we were doing conversions as the code
using Sphinx does treat struct secret as type of the secret.

Note that passing an array is the same as passing the address, so
changing from 'u8 secret[32]' to 'struct secret secret' means various
'secret' parameters change to '&secret'.  Technically, '&secret' also
would have worked before, since '&' is a noop on array, but that's
always seemed a bit weird.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-24 10:01:44 +10:30
Rusty Russell
1099f6a5e1 common: use struct onionreply.
This makes it clear we're dealing with a message which is a wrapped error
reply (needing unwrap_onionreply), not an already-wrapped one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-23 16:17:42 +10:30
Rusty Russell
ddce5573c7 channeld: use wirestring for failure strings.
I think this code predated wirestring.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-23 16:17:42 +10:30
Rusty Russell
9a72016640 lightningd: remove unused allocation.
Not a leak, since it's off tmpctx, but send_htlc_out allocates this itself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-23 16:17:42 +10:30
Christian Decker
4be1868b8a pay: Invert ownership of wallet_payment
`wallet_payment_store` would free the `wallet_payment` instance which would
then cause us to reload it from the DB. Instead of doing the store->free->load
dance we now tell `wallet_payment_store` whether it should take ownership and
leave it alone if not.

Passing the payment around instead of referencing it through payment_hash and
partid is a nice side-effect.
2020-01-13 23:34:46 +01:00
Vasil Dimov
2ea91f834c Add the missing space between "if" and "("
Changelog-None
2020-01-06 12:57:59 +01:00
Rusty Russell
72aa315b5e lightningd: save the fee_states into the database.
This is the final step: we pass the complete fee_states to and from
channeld.

Changelog-Fixed: "Bad commitment signature" closing channels when we sent back-to-back update_fee messages across multiple reconnects.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:15:48 +01:00
Rusty Russell
8e3234e67a lightningd: sew in htlc set.
The invoice_try_pay code now takes a set, rather than a single htlc, but
it's basically the same thing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell
8b129b439b lightningd: cleanup redundant args from handle_localpay
The cltv_expiry and payment_hash are in hin, so no need to hand them
in here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell
0e4a30c635 doc: update experimental bolt version quotes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell
12985331f7 htlcs: remove origin_htlc_id from htlc_out.
This is a transient field, so rework things so we don't leave it in
struct htlc_out.  Instead, load htlc_in first and connect htlc_out to
them as we go.

This also changes one place where we use it instead of the am_origin
flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell
345ca9b122 db: add partid field to htlc_out.
This is in preparation for partial payments.  For existing payments,
partid is 0 (to match the corresponding payment).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell
2d18c3a209 db: add partid, total_msat fields to payment entries.
This is in preparation for partial payments.  For existing payments,
partid is 0 (arbitrarity) and total_msat is msatoshi.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Rusty Russell
d56513362a lightningd: don't always defer commitment_signed if we're not synced.
Because my node runs under valgrind, it can take quite a while to
sync; nodes tend to disconnect and reconnect if you block too long.

This is particularly problematic since we often update fees: when the
other side sends its commitment_signed we block.

In particular, this triggers the corner case we have where we
update_fee twice, disconnecting each time, and our state machine gets
confused (which is why we never saw this exact corner case before this
change in 0.7.3!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-11 16:20:50 +01:00
Christian Decker
ff5f7b194f sphinx: Return the error in parse_onionpacket
As suggested by @niftynei here: https://github.com/ElementsProject/lightning/pull/3260#discussion_r347543999

Suggested-by: Lisa Neigut <@niftynei>
Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <@cdecker>
2019-12-11 16:18:34 +01:00
Rusty Russell
f7ebbb2ec5 common: make sphinx code ignorant of payload format.
Now "raw_payload" is always the complete string (including realm or length
bytes at the front).

This has several effects:
1. We can receive an decrypt an onion which is grossly malformed.
2. We can still hand this to the htlc_accepted hook.
3. We then fail it unless the htlc_accepted accepts it manually.
4. The createonion API now takes the raw payload, and does not know
   anything about "style".

The only caveat is that the sphinx code needs to know the payload
length: we have a call for that, which simply tells it to copy the
entire onion (and treat us as the final node) if it's invalid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-09 14:33:31 +01:00
Rusty Russell
bb538a1862 common: don't crash on bad sphinx payload.
It's cleanest to eliminate the SPHINX_INVALID_PAYLOAD altogether.

lightning_channeld: FATAL SIGNAL (version v0.7.3-242-gb1583bb-modded)
0x55a8169eed08 send_backtrace
	common/daemon.c:41
0x55a8169fc3eb status_failed
	common/status.c:206
0x55a8169fc657 status_backtrace_exit
	common/subdaemon.c:25
0x55a8169eedbb crashdump
	common/daemon.c:57
0x7f0eaff8446f ???
	???:0
0x7f0eaff843eb ???
	???:0
0x7f0eaff63898 ???
	???:0
0x55a8169fb29f route_step_decode
	common/sphinx.c:759
0x55a8169fb60a process_onionpacket
	common/sphinx.c:834
0x55a8169d9b34 get_shared_secret
	channeld/channeld.c:605
0x55a8169d9d35 handle_peer_add_htlc
	channeld/channeld.c:649
0x55a8169dd88d peer_in
	channeld/channeld.c:1838
0x55a8169e11a8 main
	channeld/channeld.c:3233
0x7f0eaff651e2 ???

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-09 14:33:31 +01:00
Christian Decker
69c17d2d31 wire: Let the TLV _is_valid function actually return validity
I got this one wrong myself, since the function name implied a boolean
result. So I changed it to take the optional err_index as argument.
2019-12-03 00:37:15 +00:00
Rusty Russell
e5247a68b6 lightningd: check payment secret on htlc receipt.
We don't set the secret to compulsory (yet!) but put code in for the
future.  Meanwhile, if there is a secret, check it is correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell
ebac3d2a85 spec: update to experimental BOLTs with secret/total_amount.
Also pulls in a new onion error (mpp_timeout).  We change our
route_step_decode_end() to always return the total_msat and optional
secret.

We check total_amount (to prohibit mpp), but we do nothing with
secret for now other than hand it to the htlc_accepted hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell
50d6941e89 lightningd: remove redundant htlc_accepted_hook_payload fields
Now we cache them in the route_step, don't need to copy them here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Christian Decker
d1df4d6959 htlc: Add a checker function tellung us whether we can continue
This function ensures we have all the infos we need to continue if the
htlc_accepted hook tells us to. It also enforces well-formedness of the TLV
payload if we have a TLV payload.

Suggested-by: List Neigut <@niftynei>
Signed-off-by: Christian Decker <@cdecker>
2019-11-22 04:40:25 +00:00
Christian Decker
d7b28ac480 htlc: Consolidate validation after the htlc_accepted hook returns
This now enforces all rules for validity, both for the TLV format and checking
that the required fields have been provided.
2019-11-22 04:40:25 +00:00
Christian Decker
fc14e5eab0 htlcs: Make necessary payload fields optional and derfer validation
We make the fields in `htlc_accepted_payload` optional (NULL if not present in
the payload) and defer validation till after the hook call.
2019-11-22 04:40:25 +00:00
Christian Decker
d69a43780c sphinx: Use the new fromwire_tlv_payload function
We wire in the code-generated function, which removes the upfront validation
and add the validation back after the `htlc_accepted` hook returns. If a
plugin wanted to handle the onion in a special way it'll not have told us to
just continue.
2019-11-22 04:40:25 +00:00
darosior
f075b87137 bitcoind: remove the chainparams member
We now have a global constant, prefer to use it instead of having
two variables with the same utility.
2019-11-15 13:14:08 +01:00
Rusty Russell
323e4f6288 dev: add option to prevent HTLC timeouts.
This is required for the protocol tests, which can be slow.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:19:01 +01:00
Rusty Russell
2a2259083a lightningd: handle tlv-style payloads.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: JSON API: `htlc_accepted` hook has `type` (currently `legacy` or `tlv`) and other fields directly inside `onion`.
Changelog-deprecated: JSON API: `htlc_accepted` hook `per_hop_v0` object deprecated, as is `short_channel_id` for the final hop.
2019-11-14 10:15:33 +01:00
Rusty Russell
a76518a029 common/sphinx: rename hop_data to hop_data_legacy.
This highlights the various places we need to change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Rusty Russell
21d2cc663b lightningd: apply feerate changes correctly.
Feerate changes are asymmetric, as they can only be sent by the funder.

For FUNDER, the remote feerate is set when upon send of
commitment_signed, and the local feerate is set on receipt of
revoke_and_ack.

For non-funder, the local feerate is set on receipt of
commitment_signed, and the remote feerate set on send of
revoke_and_ack.  In our code, these two happen together.

channeld gets this right, but lightningd ignored the funder/fundee
distinction, and as a result, receipt of a commitment_signed by the
funder altered fees in the database.  If there was a reconnection
event or restart, then these (incorrect) values would be used, causing
us to complain about a 'Bad commit_sig signature' and close the
channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-28 13:07:41 -05:00
Rusty Russell
f019dc3d71 lightningd: fix sizeof() argument correctly.
c25ce826ab claimed to fix this, but didn't;
this is the correct fix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-08 09:51:22 -05:00
Yash Bhutwala
c25ce826ab take the size of 'shared_secret' itself rather than its address 2019-10-07 11:32:33 -05:00
Rusty Russell
fa686c5ca7 channeld: reject wumbo payments with more style.
WIRE_REQUIRED_CHANNEL_FEATURE_MISSING anticipates a glorious Wumbo future,
and is closer to correct (it's a PERM failure).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-03 23:27:23 +00:00
Rusty Russell
049529542a lightningd: delay reprocessing of incoming htlcs at startup until plugins ready.
Fixes: #2923
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-01 16:55:32 +02:00
Rusty Russell
189b2f1313 BOLT: update CSV to latest bolt version.
This removes the WIRE_FINAL_EXPIRY_TOO_SOON which leaked too much info,
and adds the blockheight to WIRE_INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
2600a6ed2e channeld: get current block height when an HTLC fails.
We need it to put in the error code for
WIRE_INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
6349222ea2 Spec: Update to latest BOLT, include our first global feature definition.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
trueptolemy
4929034a40 json: Make payment_hash use json_add_sha256 2019-08-21 09:32:21 +08:00
Rusty Russell
f18b911032 lightningd: listforwards shouldn't put in zero fields for fields we don't know.
Technically, this is an API change :(  So I made it conditional.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-15 03:12:56 +00:00
lisa neigut
802ebe768c rpc: fix crash 'listforwards' when payment_hash is empty
```
2019-08-14T17:50:39.100Z **BROKEN** lightningd(11355): FATAL SIGNAL 11 (version v0.7.2rc1)
2019-08-14T17:50:39.100Z **BROKEN** lightningd(11355): backtrace: common/daemon.c:45 (send_backtrace) 0x563349d07879
2019-08-14T17:50:39.100Z **BROKEN** lightningd(11355): backtrace: common/daemon.c:53 (crashdump) 0x563349d078c9
2019-08-14T17:50:39.100Z **BROKEN** lightningd(11355): backtrace: (null):0 ((null)) 0x7efd7b996f1f
2019-08-14T17:50:39.100Z **BROKEN** lightningd(11355): backtrace: ccan/ccan/str/hex/hex.c:59 (hex_encode) 0x563349d57fec
2019-08-14T17:50:39.100Z **BROKEN** lightningd(11355): backtrace: lightningd/json.c:380 (json_add_hex) 0x563349cd9dd3
2019-08-14T17:50:39.100Z **BROKEN** lightningd(11355): backtrace: lightningd/peer_htlcs.c:2151 (json_format_forwarding_object) 0x563349cfa7ac
2019-08-14T17:50:39.100Z **BROKEN** lightningd(11355): backtrace: lightningd/peer_htlcs.c:2198 (listforwardings_add_forwardings) 0x563349cfa99d
2019-08-14T17:50:39.100Z **BROKEN** lightningd(11355): backtrace: lightningd/peer_htlcs.c:2216 (json_listforwards) 0x563349cfaa55
2019-08-14T17:50:39.100Z **BROKEN** lightningd(11355): backtrace: lightningd/jsonrpc.c:650 (parse_request) 0x563349cdc184
2019-08-14T17:50:39.100Z **BROKEN** lightningd(11355): backtrace: lightningd/jsonrpc.c:748 (read_json) 0x563349cdc5ae
2019-08-14T17:50:39.100Z **BROKEN** lightningd(11355): backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x563349d4bbe5
2019-08-14T17:50:39.100Z **BROKEN** lightningd(11355): backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x563349d4c762
2019-08-14T17:50:39.101Z **BROKEN** lightningd(11355): backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x563349d4c7a0
2019-08-14T17:50:39.101Z **BROKEN** lightningd(11355): backtrace: ccan/ccan/io/poll.c:445 (io_loop) 0x563349d4e7f5
2019-08-14T17:50:39.101Z **BROKEN** lightningd(11355): backtrace: lightningd/io_loop_with_timers.c:24 (io_loop_with_timers) 0x563349cd8afe
2019-08-14T17:50:39.101Z **BROKEN** lightningd(11355): backtrace: lightningd/lightningd.c:834 (main) 0x563349cded3a
2019-08-14T17:50:39.101Z **BROKEN** lightningd(11355): backtrace: (null):0 ((null)) 0x7efd7b979b96
2019-08-14T17:50:39.101Z **BROKEN** lightningd(11355): backtrace: (null):0 ((null)) 0x563349cc5909
2019-08-14T17:50:39.101Z **BROKEN** lightningd(11355): backtrace: (null):0 ((null)) 0xffffffffffffffff
```

[ Modified to simply omit field --RR ]
2019-08-15 03:12:56 +00:00
lisa neigut
58fb1528dd add_htlc hook: fix crash when failing UPDATE failcode
Passing in an UPDATE failcode crashes, since the next hop's channel id
was passed in as NULL. Fixed by passing in id.

```
2019-08-15T00:19:49.639Z **BROKEN** lightningd(17070): FATAL SIGNAL 11 (version v0.7.2rc1-8-gbf3b77a-modded)
2019-08-15T00:19:49.639Z **BROKEN** lightningd(17070): backtrace: common/daemon.c:45 (send_backtrace) 0x55fef4ef036f
2019-08-15T00:19:49.639Z **BROKEN** lightningd(17070): backtrace: common/daemon.c:53 (crashdump) 0x55fef4ef03bf
2019-08-15T00:19:49.639Z **BROKEN** lightningd(17070): backtrace: (null):0 ((null)) 0x7f7762401f1f
2019-08-15T00:19:49.639Z **BROKEN** lightningd(17070): backtrace: lightningd/peer_htlcs.c:104 (fail_in_htlc) 0x55fef4edd9d7
2019-08-15T00:19:49.639Z **BROKEN** lightningd(17070): backtrace: lightningd/peer_htlcs.c:785 (htlc_accepted_hook_callback) 0x55fef4edf2c7
2019-08-15T00:19:49.640Z **BROKEN** lightningd(17070): backtrace: lightningd/plugin_hook.c:86 (plugin_hook_callback) 0x55fef4ee765f
2019-08-15T00:19:49.640Z **BROKEN** lightningd(17070): backtrace: lightningd/plugin.c:251 (plugin_response_handle) 0x55fef4ee44b2
2019-08-15T00:19:49.640Z **BROKEN** lightningd(17070): backtrace: lightningd/plugin.c:341 (plugin_read_json_one) 0x55fef4ee4637
2019-08-15T00:19:49.640Z **BROKEN** lightningd(17070): backtrace: lightningd/plugin.c:366 (plugin_read_json) 0x55fef4ee4764
2019-08-15T00:19:49.640Z **BROKEN** lightningd(17070): backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x55fef4f38c7a
2019-08-15T00:19:49.640Z **BROKEN** lightningd(17070): backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x55fef4f397f7
2019-08-15T00:19:49.640Z **BROKEN** lightningd(17070): backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x55fef4f39835
2019-08-15T00:19:49.640Z **BROKEN** lightningd(17070): backtrace: ccan/ccan/io/poll.c:445 (io_loop) 0x55fef4f3b88a
2019-08-15T00:19:49.640Z **BROKEN** lightningd(17070): backtrace: lightningd/io_loop_with_timers.c:24 (io_loop_with_timers) 0x55fef4ec0afe
2019-08-15T00:19:49.640Z **BROKEN** lightningd(17070): backtrace: lightningd/lightningd.c:834 (main) 0x55fef4ec6f5a
2019-08-15T00:19:49.640Z **BROKEN** lightningd(17070): backtrace: (null):0 ((null)) 0x7f77623e4b96
2019-08-15T00:19:49.640Z **BROKEN** lightningd(17070): backtrace: (null):0 ((null)) 0x55fef4ead909
2019-08-15T00:19:49.640Z **BROKEN** lightningd(17070): backtrace: (null):0 ((null)) 0xffffffffffffffff
```
2019-08-15 02:24:18 +00:00
Rusty Russell
c3a35416da lightningd: don't allow channeld to accept HTLCs if we're not synced.
We want to still allow incoming connections, and reestablishment of
channels, but if one tries to give us an HTLC, stall until we're
synced.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell
6195a878f7 lightningd: don't allow sending of HTLCs while still syncing.
If we don't know block height, we shouldn't be sending HTLCs.  This
stops us forwarding HTLCs as well as new payments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 22:09:09 +02:00
Rusty Russell
b73a85a75e lightningd: don't say 'killing channel' when HTLC times out.
We're actually only killing the connection.  I saw this in my logs,
but it was all OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-07 21:12:52 +08:00
Rusty Russell
02609773c0 lightningd: suppress gcc-7.4.0 error
In file included from wallet/test/run-wallet.c:15:0:
./lightningd/peer_htlcs.c: In function ‘htlcs_reconnect’:
./lightningd/peer_htlcs.c:2060:15: error: ‘failcode’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
   }  else if (failcode) {
               ^~~~~~~~
./lightningd/peer_htlcs.c:2056:19: error: ‘failcode’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
          failcode != 0
          ~~~~~~~~~^~~~

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
trueptolemy
a449a91ae2 JSON: Warp the process of forward payment json object
Warp this process as a new function: 'void json_format_forwarding_object()'. This function will be used in 'forward_event' next, and can ensure the consistent json object structure for forward_payment between 'listforwards' API and 'forward_event' notification.
2019-08-01 18:49:25 +08:00
trueptolemy
bcec6bb6cc API: 'listforwards' now include 'payment_hash' field
'payment_hash' can help users learn more about the forward payment.
2019-08-01 18:49:25 +08:00
Christian Decker
5dff67900e tx: Add chainparams when deserializing transactions from wire msgs
This is the other origin, besides `bitcoin_tx`, where we create `bitcoin_tx`
instances, so add the context as soon as possible. Sadly I can't weave the
chainparams into the deserialization code since that'd need to change all the
generated wire code as well.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
Christian Decker
b83d15ea4a sphinx: Remove standalone v0 payload in favor of the unionized one
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
lisa neigut
5c07afac7d bolt: update to BOLT spec changes (extract format + type specifications)
updates the bolt version to 6639cef095a2ecc7b8f0c48c6e7f2f906fbfbc58.

this requires us to use the new bolt parser at generate-bolt.py
and updates to all of the type specifications (ie. from u8 -> byte)
2019-07-16 06:10:58 +00:00
lisa neigut
254ee8a5d3 bolt-spec: bring up to date at 309e86d471faf90c1f6c531701c16887684badb9 2019-07-16 06:10:58 +00:00
Rusty Russell
d5bd1682f5 lightningd: free timers on shutdown.
Direct leak of 1024 byte(s) in 2 object(s) allocated from:
    #0 0x7f4c84ce4448 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10c448)
    #1 0x55d11b782c96 in timer_default_alloc ccan/ccan/timer/timer.c:16
    #2 0x55d11b7832b7 in add_level ccan/ccan/timer/timer.c:166
    #3 0x55d11b783864 in timer_fast_forward ccan/ccan/timer/timer.c:334
    #4 0x55d11b78396a in timers_expire ccan/ccan/timer/timer.c:359
    #5 0x55d11b774993 in io_loop ccan/ccan/io/poll.c:395
    #6 0x55d11b72322f in plugins_init lightningd/plugin.c:1013
    #7 0x55d11b7060ea in main lightningd/lightningd.c:664
    #8 0x7f4c84696b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)

To fix this, we actually make 'ld->timers' a pointer, so we can clean
it up last of all.  We can't free it before ld, because that causes
timers to be destroyed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-30 16:41:30 +09:30
Rusty Russell
a450962b49 channeld/lightningd/hsmd: strengthen our checks against 0-output txs.
If we ever do this, we'd end up with an unspendable commitment tx anyway.
It might be able to happen if we have htlcs added from the non-fee-paying
party while the fees are increased, though.  But better to close the
channel and get a report about it if that happens.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-27 05:55:26 +00:00
Rusty Russell
bb7bbd03c5 lightningd: have json_stream_success start the "result" object.
"result" should always be an object (so that we can add new fields),
so make that implicit in json_stream_success.

This makes our primitives well-formed: we previously used NULL as our
fieldname when calling the first json_object_start, which is a hack
since we're actually in an object and the fieldname is 'result' (which
was already written by json_object_start).

There were only two cases which didn't do this:
1. dev-memdump returned an array.  No API guarantees on this.
2. shutdown returned a string.

I temporarily made shutdown return an empty object, which shouldn't
break anything, but I want to fix that later anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Rusty Russell
220449e1cd ccan: import ccan/json_out and ccan/json_escape.
These are generalized from our internal implementations.

The main difference is that 'struct json_escaped' is now 'struct
json_escape', so we replace that immediately.

The difference between lightningd's json-writing ringbuffer and the
more generic ccan/json_out is that the latter has a better API and
handles escaping transparently if something slips through (though
it does offer direct accessors so you can mess things up yourself!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-12 02:00:15 +00:00
Christian Decker
9e511cbf3e plugin: Use the json_add_secret wrapper to add the shared_secret
This was incorrectly handled before, hence the wrapper which checks
correctness of the arguments.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-09 02:40:34 +00:00
Christian Decker
4c57d44252 channel: Along with the last_tx also remember its type
This takes the guesswork out of `drop_to_chain` and allows us to annotate the
last_tx consistently.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-08 02:58:27 +00:00
Christian Decker
2db4f190b3 plugin: Cleanup the htlc_accepted_hook_deserialize interface
Since we have more or less given up on the separation between response
callback and deserialization we can also just have the individual parts
returned.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Suggested-by: Rusty Russell <@rustyrussell>
2019-06-04 00:27:15 +00:00
Christian Decker
6db1e76156 startup: Tell peer_accepted_htlc whether we are replaying
It disables the error when attempting to do a state transition from
`RCVD_ADD_ACK_REVOCATION` to `RCVD_ADD_ACK_REVOCATION` which was done before
getting to this point.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-04 00:27:15 +00:00
Christian Decker
909913c265 htlc: Replay unprocessed HTLCs loaded from the DB 2019-06-04 00:27:15 +00:00
Christian Decker
adb984dd45 hooks: Add the raw payload to the htlc_accepted call
Since we might soon be changing the payload it is a good idea to not just
expose the v0 payload, but also the raw payload for the plugin to
interpret. This might also include payloads that `lightningd` itself cannot
understand, but the plugin might.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Suggested-by: Corné Plooy <@bitonic-cjp>
2019-06-04 00:27:15 +00:00
Christian Decker
1561ffaea0 hooks: Add cltv_expiry_delta to the htlc_accepted hook
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Suggested-by: Corné Plooy <@bitonic-cjp>
2019-06-04 00:27:15 +00:00
Christian Decker
2b81e02a2e plugin: Parse response for htlc_accepted hook
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-04 00:27:15 +00:00
Christian Decker
bf53821f1a plugin: Populate the request for the htlc_accepted hook
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-04 00:27:15 +00:00
Christian Decker
7499f7ddd4 plugin: Add the htlc_accepted hook
This is a rather simple hook that allows a plugin to take control over
HTLCs that were accepted, but weren't resolved as part of an invoice
or forwarded to the next hop yet.

The goal is to allow plugins to terminate a route early, perform
intermediate checks before the payment is accepted (check inventory or
service delivery before accepting in order to avoid a refund for
example) or handle an onion differently if it has a different
realm (cross-chain atomic swaps).

This doesn't implement serializing the payload or deserializing it,
instead just passes the full context along. The details for
serializing and deserializing will be implemented in a future commit.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-04 00:27:15 +00:00
darosior
323adb467a jsonrpc: Add a category field to commands.
A new string field is added to the command structure and is specified at the creation of each native command, and in the JSON created by 'json_add_help_command()'.
2019-06-03 00:02:25 +00:00
Rusty Russell
401bd9f8ef json: rename json_add_amount_sat to json_add_amount_sat_compat.
New fields don't have to be spelled out twice.

The raw version are called _only, so we don't miss a call
accidentally.  We can rename them when we finally deprecated old
fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-20 20:31:07 -04:00
trueptolemy
3445abb553 json-rpc: Add failcode info in listforwards 2019-05-03 11:45:57 +02:00
trueptolemy
11e9ad0ee0 lightningd: store failcode in FORWARD_LOCAL_FAILED case 2019-05-03 11:45:57 +02:00
Rusty Russell
6630b99cf7 lightningd: move local invoice resolution into invoice.c function.
We're going to make it async, so start by moving the core code into
invoice.c and having that directly call fail/success functions for the
htlc.

We add an extra check in fulfill_htlc() that the HTLC state is correct:
that can't happen now, but may once we're async.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-12 03:32:48 +00:00
Christian Decker
27afc804d5 json-rpc: Include received and resolved time to listforward result
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-04-10 23:48:39 +00:00
Rusty Russell
a2fa699e0e Use node_id everywhere for nodes.
I tried to just do gossipd, but it was uncontainable, so this ended up being
a complete sweep.

We didn't get much space saving in gossipd, even though we should save
24 bytes per node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Rusty Russell
6765423393 Documentation: Update to BOLT v1.0.
Mainly typo fixes, but we removed the INCORRECT_PAYMENT_AMOUNT error
altogether.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-01 13:22:05 +02:00
Michael Schmoock
8594f6e369 htlc: forward channel specific fee settings 2019-03-15 02:48:18 +00:00
Rusty Russell
bac9a594b8 wallet: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
3ac0e814d0 daemons: use amount_msat/amount_sat in all internal wire transfers.
As a side-effect of using amount_msat in gossipd/routing.c, we explicitly
handle overflows and don't need to pre-prune ridiculous-fee channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
b8e484b508 struct channel_config: use amount_sat / amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
7e3928359a listpeers: add all the alternate "msat" and "sat" fields for channels.
These are undocumented, unfortunately, but at least that means I don't
have to update the docs!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell
ea7e13b5a7 lightningd: fix leak when next peer is unknown.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
26dda57cc0 utils: make tal_arr_expand safer.
Christian and I both unwittingly used it in form:

	*tal_arr_expand(&x) = tal(x, ...)

Since '=' isn't a sequence point, the compiler can (and does!) cache
the value of x, handing it to tal *after* tal_arr_expand() moves it
due to tal_resize().

The new version is somewhat less convenient to use, but doesn't have
this problem, since the assignment is always evaluated after the
resize.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
c3e96e058e bolt: Updated the BOLT specification to unify UNKNOWN_PAYMENT_HASH & INCORRECT_PAYMENT_AMOUNT
This is based on Christian's change, but removes all trace of the old codes.

I've proposed another spec change which removes this code altogether:
	https://github.com/lightningnetwork/lightning-rfc/pull/544

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Rusty Russell <@rustyrussell>
2019-01-15 02:19:56 +00:00
Rusty Russell
554c3ec7e5 channeld: process onion packet ourselves.
This covers all the cases where an onion can be malformed; this means
we know in advance that it's bad.  That allows us to distinguish two
cases: where lightningd rejects the onion as bad, and where the next
peer rejects the next onion as bad.  Both of those (will) set failcode
to one of the BADONION values.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-08 19:20:28 +01:00
Rusty Russell
59febcb968 sphinx: explain why parse_onionpacket fails.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-08 19:20:28 +01:00
Rusty Russell
7e01efbd1f lightningd: clean up htlc_in->shared_secret to be optional.
We currently use 'all-zeroes' as 'unknown', but NULL is more natural
even if we have to send it as all-zeroes over the wire due to
expressiveness limitations in our generation code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-08 19:20:28 +01:00
Rusty Russell
3006844f92 lightningd: don't allow zero cltv HTLCs. (#2214)
Fixes: #2077
Fixes: #2213
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-04 01:08:35 +01:00
Rusty Russell
68bb36b210 json-rpc: make commands return 'struct command_result *'.
Usually, this means they return 'command_param_failed()' if param()
fails, and changing 'command_success(); return;' to 'return
command_success()'.

Occasionally, it's more complex: there's a command_its_complicated()
for the case where we can't exactly determine what the status is,
but it should be considered a last resort.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-20 03:22:32 +00:00
Rusty Russell
bc41ab2cb9 param: make json_tok_ handlers all return command_result, rename to param_
Handers of a specific form are both designed to be used as callbacks
for param(), and also dispose of the command if something goes wrong.

Make them return the 'struct command_result *' from command_failed(),
or NULL.  

Renaming them just makes sense: json_tok_XXX is used for non-command-freeing
parsers too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-20 03:22:32 +00:00
Rusty Russell
d7e233e47d Move json and param core functionality into common, for plugins.
json_escaped.[ch], param.[ch] and jsonrpc_errors.h move from lightningd/
to common/.  Tests moved too.

We add a new 'common/json_tok.[ch]' for the common parameter parsing
routines which a plugin might want, taking them out of
lightningd/json.c (which now only contains the lightningd-specific
ones).

The rest is mainly fixing up includes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell
8015e7dcfb jsonrpc: add the obj token to the callback.
This (will) avoid the plugin having to walk back from the params object
as it currently does.

No code changes; I removed UNUSED and UNNEEDED labels from the other
parameters though (as *every* json_rpc callback needs to call param()
these days, they're *always* used).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell
dffe2f516a signature: wrap almost all signatures in struct bitcoin_signature.
This is prep work for when we sign htlc txs with
SIGHASH_SINGLE|SIGHASH_ANYONECANPAY.

We still deal with raw signatures for the htlc txs at the moment, since
we send them like that across the wire, and changing that was simply too
painful (for the moment?).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-06 23:11:51 +01:00
Rusty Russell
40943c9867 Update for latest bolt version: bca814e270dcbee2fea51c0a26ca99efef261f2b
The only change is that the final_incorrect_htlc_amount field is now 64
bit.  Since no implementation yet parses that field, we just updated it
quietly in the spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-28 23:51:05 +00:00
Rusty Russell
c236361efd wireaddr: update bolt version, remove 'padding' from addresses.
Nobody used this, so it was removed from the spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-28 23:51:05 +00:00
Rusty Russell
c919551109 onchaind: include htlc id in htlc_stub so we agree on what HTLC we're closing.
If there are two HTLCs with the same preimage, lightningd would always
find the first one.  By including the id in the `struct htlc_stub`
it's both faster (normal HTLC lookup) and allows lightningd to detect
that onchaind wants to fail both of them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-23 16:55:35 +02:00
Rusty Russell
7eec2253e9 lightningd: fix up compile errors from bad merge.
The new listforwards was old-style, and the new API was merged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-20 03:34:58 +00:00
Rusty Russell
305795b01e common/json: move JSON creation routines into lightningd/
It's the only user of them, and it's going to get optimized.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

gossip.pydiff --git a/common/test/run-json.c b/common/test/run-json.c
index 956fdda35..db52d6b01 100644
2018-10-19 22:02:11 +00:00
Christian Decker
6b7546b94d json-rpc: Rename getroutestats and move stats to getinfo
Signed-off-by: Christian Decker <@cdecker>
2018-10-19 21:58:27 +00:00
Christian Decker
6d333f16cc wallet: Correctly handle forwards when channels or htlcs are deleted
The left join should make sure we still get the results but
referencing the fields and/or attempting to write them to the JSON-RPC
result will cause unforeseen problems. So just omit if we forgot
something.
2018-10-19 21:58:27 +00:00
Christian Decker
facd7d16aa json-rpc: Add listforwardings command 2018-10-19 21:58:27 +00:00
Christian Decker
4b4c549c9d htlcs: Wire up the forward statistics on HTLC updates
Signed-off-by: Christian Decker <@cdecker>
2018-10-19 21:58:27 +00:00
Rusty Russell
3991425111 gossipd: don't accept forwarding short_channel_ids we don't own.
Gossipd provided a generic "get endpoints of this scid" and we only
use it in one place: to look up htlc forwards.  But lightningd just
assumed that one would be us.

Instead, provide a simpler API which only returns the peer node
if any, and now we handle it much more gracefully.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-15 23:04:17 +00:00
Rusty Russell
1e467bb986 lightningd: fail HTLCs which are in-transit as we shut down.
This is the source of failure in the test_restart_many_payments stress
test: we don't commit the outgoing HTLC immediately, instead waiting for
gossip to tell us the peer for the outgoing channel, then waiting for
that channeld to tell is it's committed.  The result was incoming HTLCs
with no outgoing.

I initially pushed the HTLCs through that same path, but of course
(since peers are not connected yet!) the only result was that we failed
these HTLCs immediately.  So I chose the far simpler course of just
failing them directly.

To reproduce this, I had to increase the test_restart_many_payments
num to 10, and run it with nice -20 taskset -c 0.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-10 18:14:13 +02:00
Rusty Russell
0226ef0572 htlc: rename local flag to am_origin, add FIXME.
Noted by @cdecker, the term 'local' is grossly overused, and the hout
preimage is basically only used as a sanity check (though I've just put
a FIXME there for now).

Also eliminated spurious blank line which crept into wallet.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell
6c96bcacd7 lightningd: fix inconsistency without COMPAT enabled.
We don't expect payment or payment->route_channels to be NULL without an
old db, but putting an assert there reveals that we try to fail an HTLC
which has already succeeded in 'test_onchain_unwatch'.

Obviously we only want to fail an HTLC which goes onchain if we don't
already have the preimage!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell
4146950496 lightningd: don't access htlc_in's failoutchannel on db restore.
failoutchannel tells us which channel to send an update for (specifically
for temporary_channel_failure); but we don't save it into the db.  It's
not even clear we should, since it's a corner case and the channel might
not even exist when we come back.

So on db restore, change such errors to WIRE_TEMPORARY_NODE_FAILURE
which doesn't need an update.

We also don't memset it to 0 in the normal case (we only access if it
failcode has the UPDATE bit set) so valgrind will trigger if we're
wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell
79ebb8a92e db: save the failcode / failuremsg into db.
Now we can finally move the fixup code under COMPAT_V061, so it's only
for old nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell
d85251ac6c db: fix up HTLCs which are missing failure information.
We don't save them to the database, so fix things up as we load them.

Next patch will actually save them into the db, and this will become
COMPAT code.

Also: call htlc_in_check() with NULL on db load, as otherwise it aborts
internally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell
ec74aadce2 lightningd: save outgoing HTLC's preimage to db.
We can now wrap the 'missing preimage' hack in COMPAT_V061.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell
65f6813706 lightningd: handle the case where the db contains a resolved HTLC without a preimage.
We need to handle this case (old db) before the next commit, which actually
fixes it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell
9ef67e50ff lightningd: don't leave htlc_out's in pointer dangling when htlc_in freed.
Now we know this can happen (see previous patch), we need to handle it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell
4040c53258 lightningd: handle case where incoming HTLC vanished before fulfilled outgoing.
We now need an explicit 'local' flag, rather than relying on the existence
of the 'in' pointer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell
b77906634e lightningd: even more HTLC consistency checking: check states.
This means we need to check when we've altered the state, so the checks
are moved to the callers of htlc_in_update_state and htlc_out_update_state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 23:17:54 +00:00
Rusty Russell
96f05549b2 common/utils.h: add tal_arr_expand helper.
We do this a lot, and had boutique helpers in various places.  So add
a more generic one; for convenience it returns a pointer to the new
end element.

I prefer the name tal_arr_expand to tal_arr_append, since it's up to
the caller to populate the new array entry.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Rusty Russell
168bec0974 lightningd: move channel/peer/htlc load into own function.
Also, wallet has no business wiring up HTLCs; move that code to
peer_htlcs.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-03 05:01:40 +00:00
Rusty Russell
807e62b05d moveonly: move feerate routines from peer_htlcs.c to channel_control.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 02:17:51 +00:00
Rusty Russell
6338ae8a44 channeld: update fees if we're restarting.
This is a noop if we're opening a new channel (channel_fees_can_change(channel)
is false until funding locked in), but important if we're restarting.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 02:17:51 +00:00
Rusty Russell
b6a63024c1 lightningd: fix double-free on multiple HTLC timeouts.
We can close a connection with a peer to timeout an HTLC, but
we need to clear the pointer otherwise next time we try, we'll
free an expired pointer:

```
lightningd: Fatal signal 6 (version v0.6-336-gfcd1eb5-modded)
0x13ce86 crashdump
	common/daemon.c:37
0x5739f1f ???
	???:0
0x5739e97 ???
	???:0
0x573b800 ???
	???:0
0x1850c3 call_error
	ccan/ccan/tal/tal.c:93
0x18528b check_bounds
	ccan/ccan/tal/tal.c:165
0x1852ca to_tal_hdr
	ccan/ccan/tal/tal.c:174
0x185bfb tal_free
	ccan/ccan/tal/tal.c:472
0x1343a8 peer_sending_commitsig
	lightningd/peer_htlcs.c:1035
0x114f25 channel_msg
	lightningd/channel_control.c:159
0x13756b sd_msg_read
	lightningd/subd.c:474
0x177c1f next_plan
	ccan/ccan/io/io.c:59
0x178717 do_plan
	ccan/ccan/io/io.c:387
0x178755 io_ready
	ccan/ccan/io/io.c:397
0x17a336 io_loop
	ccan/ccan/io/poll.c:310
0x120589 main
	lightningd/lightningd.c:455
0x571cb96 ???
	???:0
0x10e6d9 ???
	???:0
0xffffffffffffffff ???
	???:0
2018-08-16T06:41:21.249Z lightningd(869): FATAL SIGNAL 6 (version v0.6-336-gfcd1eb5-modded)
2018-08-16T06:41:21.250Z lightningd(869): backtrace: common/daemon.c:42 (crashdump) 0x13ceda
2018-08-16T06:41:21.250Z lightningd(869): backtrace: (null):0 ((null)) 0x5739f1f
2018-08-16T06:41:21.250Z lightningd(869): backtrace: (null):0 ((null)) 0x5739e97
2018-08-16T06:41:21.251Z lightningd(869): backtrace: (null):0 ((null)) 0x573b800
2018-08-16T06:41:21.251Z lightningd(869): backtrace: ccan/ccan/tal/tal.c:93 (call_error) 0x1850c3
2018-08-16T06:41:21.251Z lightningd(869): backtrace: ccan/ccan/tal/tal.c:165 (check_bounds) 0x18528b
2018-08-16T06:41:21.252Z lightningd(869): backtrace: ccan/ccan/tal/tal.c:174 (to_tal_hdr) 0x1852ca
2018-08-16T06:41:21.252Z lightningd(869): backtrace: ccan/ccan/tal/tal.c:472 (tal_free) 0x185bfb
2018-08-16T06:41:21.252Z lightningd(869): backtrace: lightningd/peer_htlcs.c:1035 (peer_sending_commitsig) 0x1343a8
2018-08-16T06:41:21.252Z lightningd(869): backtrace: lightningd/channel_control.c:159 (channel_msg) 0x114f25
2018-08-16T06:41:21.253Z lightningd(869): backtrace: lightningd/subd.c:474 (sd_msg_read) 0x13756b
2018-08-16T06:41:21.253Z lightningd(869): backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x177c1f
2018-08-16T06:41:21.253Z lightningd(869): backtrace: ccan/ccan/io/io.c:387 (do_plan) 0x178717
2018-08-16T06:41:21.253Z lightningd(869): backtrace: ccan/ccan/io/io.c:397 (io_ready) 0x178755
2018-08-16T06:41:21.253Z lightningd(869): backtrace: ccan/ccan/io/poll.c:310 (io_loop) 0x17a336
2018-08-16T06:41:21.253Z lightningd(869): backtrace: lightningd/lightningd.c:455 (main) 0x120589
2018-08-16T06:41:21.254Z lightningd(869): backtrace: (null):0 ((null)) 0x571cb96
2018-08-16T06:41:21.254Z lightningd(869): backtrace: (null):0 ((null)) 0x10e6d9
2018-08-16T06:41:21.254Z lightningd(869): backtrace: (null):0 ((null)) 0xffffffffffffffff
Log dumped in crash.log
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 16:55:15 +02:00
Rusty Russell
9f175deecd lightningd: update feerate upon receiving revoke_and_ack from fundee.
1. l1     update_fee ->    l2
2. l1 commitment_signed -> l2 (using new feerate)
3. l1  <- revoke_and_ack   l2
4. l1 <- commitment_signed l2 (using new feerate)
5. l1  -> revoke_and_ack   l2

When we break the connection after #3, the reconnection causes #4 to
be retransmitted, but it turns out l1 wasn't telling the master to set
the local feerate until it received the commitment_signed, so on
reconnect it uses the old feerate, with predictable results (bad
signature).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-22 18:54:53 +02:00
Mark Beckwith
a3178b8177 param: remove old callback code
Cleaned up remaining code. Reduced comment noise. Reverted
macro names back to the original.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith
9b28ecf8fc param: upgraded json_tok_pubkey
Also add json_to_pubkey as utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith
8ebc95b7b0 param: upgraded json_tok_bool
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Rusty Russell
65c882ca3a Minor cleanups.
1. connect convenience variable for improved readabilty.
2. a comment explaining that timer is on channel, not HTLC.
3. use modern python style in test_htlc_send_timeout

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell
223cd97c94 lightningd: kill channeld if we added an HTLC and it didn't commit in 30 seconds.
This effectively constrains how long we'll delay an outgoing HTLC.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell
5cf34d6618 Remove tal_len, use tal_count() or tal_bytelen().
tal_count() is used where there's a type, even if it's char or u8, and
tal_bytelen() is going to replace tal_len() for clarity: it's only needed
where a pointer is void.

We shim tal_bytelen() for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-30 11:31:17 +02:00
Christian Decker
fbbc5899e4 invoices: Make the invoice_details more idiomatic
This seems like a premature optimization: it tried to cut down the number of
allocations by reusing the same `struct invoice_details` while iterating through
a number of results. But this sidesteps the checks by `valgrind` and we'd miss a
missing field that was set by the previous iteration.

Reported-by: @rustyrussell
Signed-off-by: Christian Decker <@cdecker>
2018-07-30 03:04:45 +00:00
Rusty Russell
162879d6a2 channeld: use fulfilled_htlc and failed_htlc msgs in single htlc case.
We use these for receiving arrays at init time, we should also use them
for fulfull/fail of HTLCs in normal operation.  That we we benefit from all
those assertions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Rusty Russell
1119dd5577 channeld: always receive and maintain short_channel_id of failing channel.
The master tells us the short_channel_id of the outgoing channel when
failing an HTLC, but channeld didn't store it anywhere.  It also
didn't tell channeld the short_channel_id in the case where we're
reconnecting and it's feeding us an array of failed htlcs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-27 14:12:00 +02:00
Mark Beckwith
f850849486 Modern param style for all remaining files
Removed `json_get_params`.

Also added json_tok_percent and json_tok_newaddr. Probably should
have been a separate PR but it was so easy.

[ Squashed comment update for gcc workaround --RR ]
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-20 01:14:02 +00:00
Rusty Russell
e217bc1220 per-commit-secret is a struct secret, not a sha256.
Well, it's generated by shachain, so technically it is a sha256, but
that's an internal detail.  It's a secret.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell
68a8eeea21 htlc_wire: rename malformed to failcode in struct failed_htlc.
I'm not completely convinced that it's only ever set to a failcode
with the BADONION bit set, especially after the previous patches in
this series.  Now that channeld can handle arbitrary failcodes passed
this way, simply rename it.

We add marshalling assertions that only one of failcode and failreason
is set, and we unmarshal an empty 'fail' to NULL (just the the
generated unmarshalling code does).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell
6e9ae98e1e lightningd: don't send uninialized malformed fields to channeld.
==1224== Uninitialised byte(s) found during client check request
==1224==    at 0x152CAD: memcheck_ (mem.h:247)
==1224==    by 0x152D18: towire (towire.c:17)
==1224==    by 0x152DA1: towire_u16 (towire.c:28)
==1224==    by 0x142189: towire_failed_htlc (htlc_wire.c:29)
==1224==    by 0x16343F: towire_channel_init (gen_channel_wire.c:596)
==1224==    by 0x115C2C: peer_start_channeld (channel_control.c:249)
==1224==    by 0x131701: peer_connected (peer_control.c:503)
==1224==    by 0x117820: gossip_msg (gossip_control.c:182)
==1224==    by 0x139D97: sd_msg_read (subd.c:500)
==1224==    by 0x139676: read_fds (subd.c:327)
==1224==    by 0x179D52: next_plan (io.c:59)
==1224==    by 0x17A84F: do_plan (io.c:387)
==1224==  Address 0x1ffefffabe is on thread 1's stack
==1224==  in frame #2, created by towire_u16 (towire.c:26)

Followed by:

2018-06-18T21:53:04.129Z lightningd(1224): 03933884aaf1d6b108397e5efe5c86bcf2d8ca8d2f700eda99db9214fc2712b134 chan #1: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel d0101486543e1a8b6871556a4fe1fba4ad4d83ce7f6f92919fd17bd1545d2fd5: UpdateFailMalformedHtlc message doesn't have BADONION bit set

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell
fed5a117e7 Update ccan/structeq.
structeq() is too dangerous: if a structure has padding, it can fail
silently.

The new ccan/structeq instead provides a macro to define foo_eq(),
which does the right thing in case of padding (which none of our
structures currently have anyway).

Upgrade ccan, and use it everywhere.  Except run-peer-wire.c, which
is only testing code and can use raw memcmp(): valgrind will tell us
if padding exists.

Interestingly, we still declared short_channel_id_eq, even though
we didn't define it any more!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-04 23:57:00 +02:00
Rusty Russell
e549bc6ecf lightningd: fix up BOLT references.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00