Changelog-Added: JSON-RPC: `listpeers` now includes 'last_feerate', 'next_feerate', 'initial_feerate' and 'next_fee_step' for channels in state DUALOPEND_AWAITING_LOCKIN
fixup! listpeers: include feerate info for RBF-candidate channels
We move over to the new "warning" paradigm, instead of using
an "rbf_fail" message.
Every failure is either a warning or an error; on warnings we
hang up and reconnect later, effectively resetting the state.
We're *mostly* set up for both sides doing RBF, except that it reverses
the callback flow (using the plugin vs RPC calls) and we're not
currently smart enough to flip between them gracefully
Users have no idea what they would pay for unilateral closes.
At least this gives them a clue!
Reported-by: @az0re on IRC.
Changelog-Added: JSON-RPC: `listpeers` now shows latest feerate and unilaral close fee.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
only needed for moving the subd->channel from an uncommitted_channel to
a channel; we removed uncommitted_channel from dual_open so it's no
longer necessary
We're going to move over to "unsaved channels" instead of using a
separate "uncommitted_channel" struct. This means a peer might have a
channel that's not saved to the database yet in its channel list
We need to make sure that there's at least one input that's represented
in every single RBF-attempt for this channel, to prevent "parallel"
subsequent RBFs from succeeding/opening (the multi-channel backdoor?!)
When the funding tx reaches depth, update the channel's data to the
"correct" funding transaction info from inflights (if necessary).
This will be necessary if:
- the transaction has been successfully RBF'd and
- the lesser fee transaction is the one successfully mined, OR
- the channel is in the process of being RBF'd
We were always prefixing the `message` field with the internal type
prefix 0x0407, followed by the length prefix. Neither is needed since
the type being constant is of no interest to the plugin and the length
being implicit due to the JSON-encoding.
Reported-by: Ilya Evdokimov
Changelog-Fixed: plugin: The `custommsg` hook no longer includes the internal type prefix and length prefix in its `payload`
Changelog-Deprecated: plugin: The `message` field on the `custommsg` hook is deprecated in favor of the `payload` field, which skips the internal prefix.
This avoids spamming the logs. We also remove the duplicate debug
logs on self-disable (plugin_kill logs it for us).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We currently log every kill at INFO level, even if it's during shutdown.
Change those to debug, but lift those where we got a malformed response.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We use waitpid() manually for subdaemons, so we need to step
around that (otherwise we could simply ignore them).
We could destroy subdaemons only once they've exited, but
that works badly with the sd->conn, which will be freed
when error (i.e. close) is detected, so the current code
is probably the best compromise.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If a plugin died due to connection close, we'd always say
"Plugin exited before completing handshake.", which was often
wrong.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We used to only set it for single-use offers (where it's required),
but it's still interesting for multi-use offers, so let's keep it
there.
We also put this field in the documentation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since we turned many errors into warnings, we want our tests to fail
when they happen unexpectedly. We make WARNING clear in the strings
we print, too, to help out.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
No more sending "all-channel" errors; in particular, gossipd now only
sends warnings (which make us hang up), not errors, and peer_connected
rejections are warnings (and disconnect), not errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `peer_connected` rejections now send a warning, not an error, to the peer.
This is in line with the warnings draft, where all-zeroes in a
channel_id is no longer special (i.e. it will be ignored).
But gossipd would send these if it got upset with us, so it's best
practice to ignore them for now anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we treat error messages from peer which refer to "all channels" as warnings, not errors.
We fix up the test by using pay, instead of sendpay (and making pay log
the expected message).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: sendpay no longer extracts updates from errors, the caller should do it from the `raw_message`.
From #clightning:
(11:24:10) andytoshi: hiya, i'm trying to set up a new lightningd node, and when i run lightningd --network=bitcoin --log-level=debug --daemon
(11:24:17) andytoshi: i get errors of the form fetchinvoice: Malformed JSON reply '2021-01-25T00:51:16.655Z DEBUG plugin-offers: disabled itself at init: offers not enabled in config
(11:24:43) andytoshi: there are a couple variants of this, but always some form of "something: failed to parse <a log line> as json"
Indeed, we close stdout, and it ends up being reused for some plugin.
But the real problem is that we log to stdout by default, which doesn't
make sense. If they really want to discard logs, they can use
--log-file=/dev/null.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON failures when --daemon is used without --log-file.
rearranges the`peer_connected_hook_payload` definition to the location
where this is used in the file.
Fixes certain blanklines and linebreaks to make the code look nicer.
Nit: The underscore in "openchannel_hook" is wrong, bcause the name of
the hook is just "openchannel". The "_hook" implied this to be part of
the name.
Changelog-None
It doesn't make sense currently without EXPERIMENTAL_FEATURES, and
check-manpages complains about it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
payload is owned by the peer, which is freed in this case, then we
free payload (again).
==1404== Invalid read of size 8
==1404== at 0x1F39E8: to_tal_hdr (tal.c:174)
==1404== by 0x1F43A4: tal_free (tal.c:479)
==1404== by 0x14B3D1: peer_connected_hook_cb (peer_control.c:1087)
==1404== by 0x15D6E9: plugin_hook_call_ (plugin_hook.c:288)
==1404== by 0x14B40E: plugin_hook_call_peer_connected (peer_control.c:1090)
==1404== by 0x14B5B8: peer_connected (peer_control.c:1135)
==1404== by 0x122FCF: connectd_msg (connect_control.c:310)
==1404== by 0x160291: sd_msg_read (subd.c:480)
==1404== by 0x15FBE7: read_fds (subd.c:308)
==1404== by 0x1E37D1: next_plan (io.c:59)
==1404== by 0x1E434E: do_plan (io.c:407)
==1404== by 0x1E438C: io_ready (io.c:417)
==1404== Address 0x2fcd2268 is 24 bytes inside a block of size 336 free'd
==1404== at 0x4C32D3B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1404== by 0x1F416E: del_tree (tal.c:421)
==1404== by 0x1F40F2: del_tree (tal.c:412)
==1404== by 0x1F442C: tal_free (tal.c:486)
==1404== by 0x148816: delete_peer (peer_control.c:120)
==1404== by 0x148899: maybe_delete_peer (peer_control.c:136)
==1404== by 0x13A970: destroy_uncommitted_channel (opening_common.c:29)
==1404== by 0x1F3BB1: notify (tal.c:240)
==1404== by 0x1F40A0: del_tree (tal.c:402)
==1404== by 0x1F442C: tal_free (tal.c:486)
==1404== by 0x13D3E9: peer_start_openingd (opening_control.c:911)
==1404== by 0x14B3C2: peer_connected_hook_cb (peer_control.c:1086)
==1404== Block was alloc'd at
==1404== at 0x4C31B0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1404== by 0x1F3C1B: allocate (tal.c:250)
==1404== by 0x1F41B4: tal_alloc_ (tal.c:428)
==1404== by 0x14B454: peer_connected (peer_control.c:1105)
==1404== by 0x122FCF: connectd_msg (connect_control.c:310)
==1404== by 0x160291: sd_msg_read (subd.c:480)
==1404== by 0x15FBE7: read_fds (subd.c:308)
==1404== by 0x1E37D1: next_plan (io.c:59)
==1404== by 0x1E434E: do_plan (io.c:407)
==1404== by 0x1E438C: io_ready (io.c:417)
==1404== by 0x1E6552: io_loop (poll.c:445)
==1404== by 0x12E2AD: io_loop_with_timers (io_loop_with_timers.c:24)
Fixes: #4329
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The fetchinvoice and offers plugins disable themselves if the option
isn't enabled (it's enabled by default on EXPERIMENTAL_FEATURES).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-offers` enables fetch, payment and creation of (early draft) offers.
By returning 'disable: <reason>' inside getmanifest or init result.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: plugins: plugins can now disable themselves by returning `disable`, even if marked important.
Note that this also changes so the feature is not represented in channels,
reflecting the recent drafts.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-onion-messages` enables send, receive and relay of onion messages.
We weren't sending a channel_open notification for dual-funded channels.
This is only sent for the 'accepter' side. We send it as soon as both
funding_tx sigs have been exchanged, even though it's possible the
funding transaction might be published without this having been the case.
Since we fail the channel if this happens, only notifying for good/valid
channels reaching the broadcast state is the right way to handle this.
Allow a user to switch on dual-funding without needing to compile
as EXPERIMENTAL_FEATURES.
Doesn't work yet, since everything is still behind
'EXPERIMENTAL_FEATURES' compile time flags... but useful for testing
Let plugins know when a channel open has failed.
We need to notify accepters now too, so we remove the check on who's
funding the channel before sending the 'failed' message from
dualopend->master.
We already know what the channel id is, we should go ahead and pass it
on to any listening plugins -- this makes clean up easier/possible
if a open fails early on and we've got reserved utxos.
When we're the opener, we get the upfront shutdown scriptpubkey (if
there is one) from the `close_to` param of `openchannel_init`.
We were passing it through dualopend, but we need to break the update
chain so that our test_option_upfront_shutdown_script test works (same
as on the openingd flow.)
Prior to this, all reconnect logic lived in channeld. If you
disconnected before we finished building a funding transaction, that was
no big deal. Now, however, we're waiting for the funding to lock in in
dualopend, instead of handing straight to channeld to wait.
So we need a way to restart dualopend.
We delegate the decision about what to do about 'out of bound' feerates
to the plugin (if one exists), however in the case that the plugin
doesnt exist or doesnt want to figure it out, we default to using the
'best' as their intended feerate, and rejecting if it's out of bounds.
We need to know if they've sent us their sigs message yet. Ideally, we'd
be able to check the 'finalness' of the PSBT, however if the peer
doesn't have any inputs to the channel this doesn't work.
Back in the days before dual-funding, the `channel` struct on subd was
only every one type per daemon (either struct channel or struct
uncommitted_channel)
The RBF requirement on dualopend means that dualopend's channel,
however, can now be two different things -- either channel or
uncommitted_channel.
To track the difference/disambiguate, we now track the channel type on a
flag on the subd. It gets updated when we swap out the channel.
This will make it possible to do RBF, since we can re-start the opening
process in dualopend while waiting for lock-in.
Note the new channel states are being used, DUALOPEND_INIT and
DUALOPEND_AWAITING_LOCKIN, to differentiate from openingd/channeld opens
v2 channel opens are going to happen over in dualopend. In order
to make sure that these don't end up in the wrong place/to keep track of
the difference between "waiting for sigs" and "have merely initiatlized
a channel", we add two new states to the channel state machine.
A channel that 'originates' in dualopend will only ever arrive at
channeld in the state CHANNELD_NORMAL.
Since this all stays in dualopend/dual_open_control, we can hold
onto the openchannel_signed command to wait for a response here locally.
Previously we were splitting across the channeld/openingd boundary.
We used this for dual funded opens, to track the receipt of signatures.
We're moving all of this over to dualopend now, however, so we no longer
need the PSBT in channeld.
This is fixed when payload is freed, but I noted a leak in the case of
an invalid payload:
```
E ValueError:
E Node errors:
E Global errors:
E - Node /tmp/ltests-ipt6ab_y/test_sendinvoice_1/lightning-2/ has memory leaks: [
E {
E "backtrace": [
E "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
E "ccan/ccan/tal/tal.c:471 (tal_alloc_arr_)",
E "gossipd/gossipd_wiregen.c:925 (fromwire_gossipd_got_onionmsg_to_us)",
E "lightningd/onion_message.c:99 (handle_onionmsg_to_us)",
E "lightningd/gossip_control.c:166 (gossip_msg)",
E "lightningd/subd.c:480 (sd_msg_read)",
E "ccan/ccan/io/io.c:59 (next_plan)",
E "ccan/ccan/io/io.c:407 (do_plan)",
E "ccan/ccan/io/io.c:417 (io_ready)",
E "ccan/ccan/io/poll.c:445 (io_loop)",
E "lightningd/io_loop_with_timers.c:24 (io_loop_with_timers)",
E "lightningd/lightningd.c:1016 (main)"
E ],
E "label": "gossipd/gossipd_wiregen.c:925:u8[]",
E "parents": [
E "lightningd/onion_message.c:96:struct onion_message_hook_payload",
E "lightningd/plugin_hook.c:87:struct hook_instance *[]"
E ],
E "value": "0x560779438db8"
E }
E ]
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Our new "decode" command will also handle bolt11. We make a few cleanups:
1. Avoid type_to_string() in JSON, instead use format functions directly.
2. Don't need to escape description now that JSON core does that for us.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The previous onion_message code required a confirmed, not-shutting-down
channel, not just a connection. That's overkill; plus before widespread
adoption we will want to connect directly as a last resort.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is required if we want to create a "bouncer" plugin (in my copious free time!)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `invoice` now takes an optional `cltv` parameter.
This makes for more useful errors. It prints where it was up to in
the guide, but doesn't print the entire JSON it's scanning.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Interestingly, we required that "sendrawtx" include "errmsg" field even
on success, otherwise we crashed in broadcast_remainder.
We only actually insist on an "errmsg" if success is false. And this
logic here is weird (the !success) was added by darosior in
947f5ddde1, which makes the msg checks redundant.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
A user reported that it is sometimes cumbersome to search an invoice
based on the payment hash or the bolt11 string in the full list, which
may be required when we don't have the label available.
This adds support for querying / filtering based on the `payment_hash`
or `bolt11` string.
Changelog-Added: JSON-RPC: `listinvoices` can now query for an invoice matching a `payment_hash` or a `bolt11` string, in addition to `label`
This makes use of the constant defined in the previous commits to more
accurately detect plaintext, encrypted, and invalid seeds. We now error
on invalid seeds.
Changelog-changed: hsmd: we now error at startup on invalid hsm_secret
Changelog-changed: hsmtool: all commands now error on invalid hsm_secret
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
This avoids duplication of both logic and error-prone values, such as
the salt. Grouping all hsm encryption logic into a public API will also
allow us to fuzz it.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
Changelog-changed: lightningd: the `--encrypted-hsm` now asks you to confirm your password when first set
Changelog-changed: hsmtool: the `encrypt` now asks you to confirm your password
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
This is similar to the createinvoice API, except we don't need to save
invoice requests in the database. We may, however, have to look up
payment_key for recurring invoice requests, and sign the message with
the payment_key.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Invoices are signed with our own key, but we use a transient payer_key with a
tweak for invoice_requests (and refunds).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. Hoist 7200 constant into the bolt12 heade2.
2. Make preimage the last createinvoice arg, so we could make it optional.
3. Check the validity of the preimage in createinvoice.
4. Always output used flag in listoffers.
5. Rename wallet offer iterators to offer_id iterators.
6. Fix paramter typos.
7. Rename `local_offer_id` parameter to `localofferid`.
8. Add reference constraints on local_offer_id db fields.
9. Remove cut/paste comment.
10. Clarify source of fatal() messages in wallet.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This takes an unsigned bolt11 (or bolt12 if EXPERIMENTAL_FEATURES) string
and signs it and puts it in the database.
The invoice command could now be moved out to a plugin, in fact.
Changelog-Added: JSON-RPC: `createinvoice` new low-level invoice creation API.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
