By default, we won't close a channel that we leased to a peer.
You can override this with the `force_lease_closed` flag.
Changelog-Added: JSON-RPC: close now has parameter to force close a leased channel (option_will_fund)
Typically we forget a channel if 2016 blocks have passed and
the funding transaction hasn't been mined yet, however we
SHOULD NOT forget these channels if we've got funds in them!
Useful for parsing a passed in feerate before calling lightningd with
it, e.g. when you need to know what the feerate is for a fundpsbt before
calling fundpsbt
Changelog-Added: JSON-RPC: new command `parsefeerate` which takes a feerate string and returns the calculated perkw/perkb
When a request comes through, we forward it over to the funder who
uses the currently set policy to figure out how to handle it.
Includes small update to the policy engine which decides whether or not
to fund a request.
Changelog-Experimental: Plugins: `openchannel2` hook now includes optional fields for a channel lease request
Using a 'feestep' is more restrictive than you'd want, instead we
enforce that the next feerate must be at least 1/64th more than the
last, but put no upper limit on it
Includes update to lnprototest changes
Contributed-By: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-EXPERIMENTAL: Protocol: Replaces init_rbf's `fee_step` for RBF of v2 opens with `funding_feerate_perkw`, breaking change
This adds a `u32 checksum` field to the plugin struct that is used
to identify if a plugin is outdated and needs to be restarted on `rescan`.
Note: Only affects non-important plugins.
Changelog-Added: Plugin: Restart plugin on `rescan` when binary was changed.
Incoming HTLCs are rejected by the HTLC logic if the payload contains
an even type that `lightningd` doesn't recognize. This is to prevent
us from accidentally accepting a payment that has extra semantics
attached (for example if we get a keysend payment and don't know what
to do with the TLV field containing the message we should reject it,
otherwise the overall semantics of the message delivery fail).
Enable non-dev builds to send custom messages.
Preserves 'dev-' for compat-enabled builds.
Changelog-Changed: JSON-RPC: moved dev-sendcustommsg to sendcustommsg
In fact, we make it compulsory, which means if you don't understand it
you'll hang up on us!
Add some logging for that in future.
Changelog-Changed: Protocol: All new invoices require a payment_secret (i.e. modern TLV format onion)
Changelog-Changed: Protocol: We can no longer connect to peers which don't support `payment_secret`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This makes it easier to access (rather than decoding bolt11).
Changelog-Added: JSON-RPC: `invoice` now outputs explicit `payment_secret` it its own field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Useful for regtest and testnet. Sure, you shouldn't use this on mainnet,
but I haven't restricted it because our users are usually pretty clever.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #1806
Changelog-Added: config: `force_feerates` option to allow overriding feerate estimates (mainly for regtest).
We don't do it for sendinvoice (yet?).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: `fetchinvoice` can take a payer note, and `listinvoice` will show the payer_notes received.
We don't support it (yet), but update the spec to include it.
We include the previous field (recurrence_signature) as a shim for the
moment, for compat with existing nodes. It's ugly, but next release
we'll stop *sending* it, then finally we'll stop accepting it!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
As per latest spec revision.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: BOLT12 offers can now be unsigned, for really short QR codes.
Discovered by the next test, oops.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSONRPC: `listinvoice` no longer crashes if given an invalid (or bolt12) `invstring` argument.
We were actually using the last commit tx's size, since we were
setting it in lightningd. Instead, hand the min and desired feerates
to closingd, and (as it knows the weight of the closing tx), and have
it start negotiation from there.
This can be significantly less when anchor outputs are enabled: for
example in test_closing.py, the commit tx weight is 1124 Sipa, the
close is 672 Sipa!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: Use a more accurate fee for mutual close negotiation.
Show amount they were trying to pay with, not invoice amount.
Also, show min fee in closing, not fee they offered.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Incoming HTLCs are rejected by the HTLC logic if the payload contains
an even type that `lightningd` doesn't recognize. This is to prevent
us from accidentally accepting a payment that has extra semantics
attached (for example if we get a keysend payment and don't know what
to do with the TLV field containing the message we should reject it,
otherwise the overall semantics of the message delivery fail).
This lets us transition (with a few supporting changes) to closingd,
which will happily let them mutual close with us.
We already handle the case where this mutual close is redundant (for
packet loss), so this is easy.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: We will now reestablish and negotiate mutual close on channels we've already closed (great if peer has lost their database).
This supports reestablish on a closed channel: we tell channeld to
respond to the reestablish message appropriately, then close the
channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It handles all the cases of retransmission, and in the normal case
retransmits shutdown and immediately returns for us to run closingd.
This is actually far simpler and reduces code duplication.
[ Includes fixup to stop warn_unused_result from Christian ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: We could get stuck on signature exchange if we needed to retransmit the final revoke_and_ack.
1. Hide the now-deprecated enable-autotor-v2-mode option.
2. Really don't print dev- options.
3. Don't print true and false as strings in some cases.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `listconfigs` would list some boolean options as strings `"true"` or `"false"` instead of using JSON booleans.
Makes it possible to write a decent JSON schema, but means we need to carry
additional data, so we create a `struct plugin_command`.
We remove the unused struct dynamic_plugin, too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In general, it's better to omit a field than put in a 'null', and
putting variable-named fields in an object is also a bad idea.
This is reflected in how hard this is to express in JSON schema, too.
Others:
1. Remove the obsolete "funding": "LOCAL" from unopened channels, but add
"opener": "local" as used in normal channels.
2. htlc cltv_expiry is a u16.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `listfunds` `channels` `funding_allocation_msat` and `funding_msat`: use `funding`.
Changelog-Deprecated: JSON-RPC: `listfunds` `channels` `last_tx_fee`: use `last_tx_fee_msat`.
Changelog-Deprecated: JSON-RPC: `listfunds` `channels` `closer` is now omitted if it does not apply, not JSON `null`.
https://github.com/lightningnetwork/lightning-rfc/pull/877 talks about
removing this restriction (only Electrum actually enforced it on
receive), so start by allowing creation of giant invoices, though
we mark them as requiring mpp.
Changelog-Changed: JSON-RPC: `invoice` now allows creation of giant invoices (>= 2^32 msat)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This mainly helps our CI under valgrind, which starts a fresh instance
and immediately calls the invoice command. This can cause the topology
plugin to try to access the gossmap file before it's created.
We can also move the gossmap reading in topology to init time.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This involves removing some fields from the now-misnamed routing.h
datastructures, and various internal messages.
One non-obvious change is to our "keepalive" logic which refreshes
channels every 13 days: instead of using the 'enabled' flag on the
last channel broadcast to decide whether to refresh it, we use the
local connected status directly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This turned into a more extensive cleanup than intended. The previous
warnings were overlapping and confusing, especially now MPP is the norm.
*warning_capacity* is now the "even under best circumstances, we don't
have enough incoming capacity", which is really what
warning_mpp_capacity was trying to say (no longer printed).
*warning_offline* and *warning_deadends* are only given if adding such
peers would have helped give capacity (i.e. not if *warning_capacity*
is set). The new *warning_private_unused* tells you that we would
have sufficient capacity, but we refused to expose private channels.
The test cases have been enhanced to cover the new warnings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `invoice` now gives `warning_private_unused` if unused unannounced channels could have provided sufficient capacity.
Changelog-Changed: JSON-RPC: `invoice` warnings are now better defined, and `warning_mpp_capacity` is no longer included (since `warning_capacity` covers that).
Temporarily rename old getroute to getrouteold (we will remove this).
Changelog-Changed: JSON-RPC: `getroute` is now implemented in a plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Prior to this, sending a v1 address (or, in fact, any random crap!)
would cause the unsupporting node to unilaterally close.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
An "active" channel may still be CHANNELD_AWAITING_LOCKIN, so have ->scid NULL.
You can only trigger this by trying to sendpay to the node using a
manual route, since routing would never use such a channel.
```
lightningd: FATAL SIGNAL 11 (version v0.10.0-319-g81cbc20-modded)
0x55e79d194e17 send_backtrace
common/daemon.c:39
0x55e79d194ec1 crashdump
common/daemon.c:52
0x7fce2d79920f ???
???:0
0x7fce2d8e16f7 ???
???:0
0x55e79d2019eb tal_dup_
ccan/ccan/tal/tal.c:801
0x55e79d14e1d9 immediate_routing_failure
lightningd/pay.c:365
0x55e79d14fe91 send_payment_core
lightningd/pay.c:1022
0x55e79d150995 send_payment
lightningd/pay.c:1180
0x55e79d151975 json_sendpay
lightningd/pay.c:1462
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Annotating the htlc in `listpeers` with their current status, and
which plugin is currently holding on to them with their
`htlc_accepted` hook can help us debug where plugins may go wrong.
Changelog-Added: jsonrpc: HTLCs in `listpeers` are now annotated with a status if they are waiting on an `htlc_accepted` hook of a plugin.
We don't actually set desired_type yet, but this handles it.
Changelog-EXPERIMENTAL: Protocol: we can now upgrade old channels to `option_static_remotekey` from https://github.com/lightningnetwork/lightning-rfc/pull/868
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Currently we abuse openingd and dualopend to do this, but connectd already
has the ability to talk to peers, so it's more efficient.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Undocumented (caught by json schema!) if we discard channel because it
wasn't open yet, then close returned the empty object. Make it return
a new type in this case.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSONRPC: `close` returns `type` "unopened" if it simply discards channel instead of empty object.
Tor v2 hidden services have been deprecated for a while:
https://blog.torproject.org/v2-deprecation-timeline .
This prevents user from being able to set them in the configuration
and to connect to them while still letting us be able to parse them
for gossip.
Changelog-Deprecated: lightningd: v2 Tor addresses. Use v3. See https://blog.torproject.org/v2-deprecation-timeline.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
If the peer is offline when we see the funding txid, we don't actually
update the channel's info. Here, we move it up to where the scid is set,
so that we always update the channel's funding_txid to the correct
(mined) information.
We set the timeout on first HTLC, but didn't clear it if that HTLC failed.
It's saner to have a per-HTLC timeout (since that's what it is!) and
also our timer infra is specially coded to scale approximately infinitely so
trying to optimize this is vastly premature.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: We would sometimes gratuitously disconnect 30 seconds after an HTLC failed.
1. We don't need to check for NULL before tal_count(NULL).
2. Use of json_for_each_arr iterator is probably better.
3. Weird indent fixed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Trying to put all the disconnect logic into the same path was a dumb
idea. If you asked to reconnect but passed in an 'unsaved' channel, we
would not call the 'reconnect' code.
Instead, we make a differentiation between "unsaved" channels
(ones that we haven't received commitment tx for) and handle the
disconnect for these separate from where we want to do a reconnect.
We were freeing the payload, which is then subsequently freed by the
plugin_hook caller. Whoops.
Now we pass through to the callback function and just clean up neatly.
------------------------------- Valgrind errors --------------------------------
Valgrind error file: valgrind-errors.406602
==406602== Invalid read of size 8
==406602== at 0x12AC93: openchannel2_hook_cb (dual_open_control.c:669)
==406602== by 0x12AF0A: openchannel2_hook_deserialize (dual_open_control.c:721)
==406602== by 0x16EF0E: plugin_hook_callback (plugin_hook.c:186)
==406602== by 0x169746: plugin_response_handle (plugin.c:514)
==406602== by 0x169959: plugin_read_json_one (plugin.c:620)
==406602== by 0x169B23: plugin_read_json (plugin.c:665)
==406602== by 0x1F4076: next_plan (io.c:59)
==406602== by 0x1F4C5B: do_plan (io.c:407)
==406602== by 0x1F4C9D: io_ready (io.c:417)
==406602== by 0x1F6F35: io_loop (poll.c:445)
==406602== by 0x13D48D: io_loop_with_timers (io_loop_with_timers.c:24)
==406602== by 0x143388: main (lightningd.c:1111)
==406602== Address 0x75e7418 is 56 bytes inside a block of size 3,520 free'd
==406602== at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==406602== by 0x204FB0: del_tree (tal.c:421)
==406602== by 0x20527E: tal_free (tal.c:486)
==406602== by 0x122D68: delete_channel (channel.c:124)
==406602== by 0x129291: channel_disconnect (dual_open_control.c:63)
==406602== by 0x129364: channel_close_conn (dual_open_control.c:82)
==406602== by 0x131CF6: peer_please_disconnect (connect_control.c:304)
==406602== by 0x131DEB: connectd_msg (connect_control.c:326)
==406602== by 0x172023: sd_msg_read (subd.c:509)
==406602== by 0x1F4076: next_plan (io.c:59)
==406602== by 0x1F4C5B: do_plan (io.c:407)
==406602== by 0x1F4C9D: io_ready (io.c:417)
==406602== Block was alloc'd at
==406602== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==406602== by 0x204A39: allocate (tal.c:250)
==406602== by 0x204FFA: tal_alloc_ (tal.c:428)
==406602== by 0x123165: new_unsaved_channel (channel.c:209)
==406602== by 0x130D34: peer_start_dualopend (dual_open_control.c:2985)
==406602== by 0x15BD2A: peer_connected_hook_final (peer_control.c:1105)
==406602== by 0x16F2E5: plugin_hook_call_ (plugin_hook.c:275)
==406602== by 0x15BF5C: plugin_hook_call_peer_connected (peer_control.c:1155)
==406602== by 0x15C16C: peer_connected (peer_control.c:1208)
==406602== by 0x131E3B: connectd_msg (connect_control.c:332)
==406602== by 0x172023: sd_msg_read (subd.c:509)
==406602== by 0x171842: read_fds (subd.c:310)
Fixes: #4494
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: config: New option `log-timestamps` allow disabling of timestamp prefix in logs.
Since plugins will start sending them soon, and they are likely to get
it wrong sometimes, be a bit more lenient, warn them in the logs
instead and then make sure it doesn't accidentally work anyway.
A plugin might subscribe to a notification topic that is only
registered by another plugin later, so push the check to that
consistency check phase where we do hook ordering as well.
We will eventually start emitting and dispatching custom notifications
from plugins just like we dispatch internal notifications. In order to
get reasonable error messages we need to make sure that the topics
plugins are asking for were correctly registered. When doing this we
don't really care about whether the plugin that registered the
notification is still alive or not (it might have died, but
subscribers should stay up and running), so we keep a list of all
topics attached to the `struct plugins` which gathers global plugin
information.
This avoids subdaemons complaining about malformed messages from us,
or doing the completely wrong thing, if they are really the wrong
version.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
You still shouldn't do this (you could get some transient failures),
but at least you have a decent chance if you reinstall over a running
daemon, instead of getting confusing internal errors if message
formats have changed.
Changelog-Added: lightningd: we now try to restart if subdaemons are upgraded underneath us.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #4346
We generally hang things off our JSON response (this pattern predates
tmpctx!) but sometimes it gets reported as a memleak. I'd prefer not
to mark JSON responses as "notleak", since they can be allocated for
a while), so use tmpctx here.
```
E ValueError:
E Node errors:
E Global errors:
E - Node /tmp/ltests-spnausnb/test_htlc_out_timeout_1/lightning-1/ has memory leaks: [
E {
E "backtrace": [
E "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
E "ccan/ccan/tal/tal.c:471 (tal_alloc_arr_)",
E "wallet/wallet.c:1775 (wallet_state_change_get)",
E "lightningd/peer_control.c:922 (json_add_channel)",
E "lightningd/peer_control.c:1424 (json_add_peer)",
E "lightningd/peer_control.c:1454 (json_listpeers)",
E "lightningd/jsonrpc.c:643 (command_exec)",
E "lightningd/jsonrpc.c:767 (rpc_command_hook_final)",
E "lightningd/plugin_hook.c:275 (plugin_hook_call_)",
E "lightningd/jsonrpc.c:855 (plugin_hook_call_rpc_command)",
E "lightningd/jsonrpc.c:942 (parse_request)",
E "lightningd/jsonrpc.c:1033 (read_json)",
E "ccan/ccan/io/io.c:59 (next_plan)",
E "ccan/ccan/io/io.c:435 (io_do_always)",
E "ccan/ccan/io/poll.c:300 (handle_always)",
E "ccan/ccan/io/poll.c:377 (io_loop)",
E "lightningd/io_loop_with_timers.c:24 (io_loop_with_timers)",
E "lightningd/lightningd.c:1097 (main)"
E ],
E "label": "wallet/wallet.c:1775:struct state_change_entry[]",
E "parents": [
E "common/json_stream.c:29:struct json_stream",
E "ccan/ccan/io/io.c:91:struct io_conn",
E "lightningd/lightningd.c:116:struct lightningd"
E ],
E "value": "0x55c6b02150b8"
E }
E ]
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The main change which affects us is that 2016 blocks to forget a channel
is a fixed number in the spec; we make this clear by renaming the
(developer-only) max_funding_unconfirmed to dev_max_funding_unconfirmed
and making it compile DEVELOPER only.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means remembering the connection direction. We also use the address to try
to reconnect, which we shouldn't bother with if they connect to us.
For peers from the database, we currently always save the addr: we shouldn't really
do this if they connected to us, since it's not useful for reconnecting (we don't
show the addr in JSON reply to listpeers unless we're connected, so it's only an
internal issue). This is left for future work.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This matters: if we connected, the address is probably usable for future connections.
But if they connected, the port is probably not (but the IP address may be).
Changelog-Added: JSON-RPC: `connect` returns "direction" ("in": they iniatated, or "out": we initiated)
Changelog-Added: plugins: `peer_connected` hook and `connect` notifications have "direction" field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
You can now activate dual-funded channels using the
`--experimental-dual-fund` flag
Changelog-Changed: Config: `--experimental-dual-fund` runtime flag will enable dual-funded protocol on this node
As @fiatjaf points out we were notifying before we were actually set
on accepting, since the hook could also still reject. Switched them
around does and calling the notification only once it's been decided
is the correct thing to do.
Changelog-Fixed: plugin: The `invoice_payment` notification was being sent before the hook was called, which could still abort it.
Suggested-by: Fiatjaf <@fiatjaf>
Signed-off-by: Christian Decker <@cdecker>
There are perfectly valid reasons for us to not have a command on return
(something went boom while sending them our sigs and we've now gotten
their sigs during a reconnect and subsequently broadcast the tx)
This can result in us logging a warning if we've 1) dropped their sigs
response, 2) only us (the opener) added inputs, 3) and we broadcast on
their reconnect (when they retransmit their sigs)
Otherwise, we might find an address other than the one given and
the user might think that address worked.
Fixes: #4185
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `connect` returns `address` it actually connected to
And update all the in-tree callers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `fundchannel_complete` `txid` and `txout` parameters (use `psbt`)
Requiring the user to calculate the txid of the PSBT is a horrible, bad,
no-good idea.
Doesn't deprecate yet, so I can test that this path works while
multifundchannel still uses it.
Fixes: #4416 (at least for future users!)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundchannel_complete` takes a psbt parameter.
Changelog-Added: lightningd: experimental-shutdown-wrong-funding to allow remote nodes to close incorrectly opened channels.
Changelog-Added: JSON-RPC: close has a new `wrong_funding` option to try to close out unused channels where we messed up the funding tx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's not unheard of for people to give the wrong funding tx to us,
getting their funds stuck. Interestingly, we can allow mutual close
using a different txid and output number as long as they (solely)
funded the channel, and the channel hasn't been used.
This defines a "play area" feature to do just that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Allows us to clean up an in-progress open that we won't be completing
Changelog-Added: EXPERIMENTAL JSON-RPC: Permit user-initiated aborting of in-progress opens. Only valid for not-yet-committed opens and RBF-attempts
The `rbf_channel` hook uses `our_funding_msat`, which is a nicer
and more easily understood than the `openchannel2`
`accepter_funding_msat`.
This updates the `openchannel2` hook to use the same nomenclature as
`rbf_channel`.
We were reporting the failure immediately but still continuing with
the startup. This could happen if an important plugin ends up in a
race with another plugin (important or not) for a contended
resource (CLI option or RPC method name). We would eventually notice
that we were supposed to abort, but at that point we already processed
a couple of blocks, loaded the entire state, etc.
This just aborts early with a sane error message.
Changelog-Added: plugin: If there is a misconfiguration with important plugins we now abort early with a more descriptive error message.
Reported-by: PsySc0rpi0n
Reported-by: Ján Sáreník <@jsarenik>
The receiving node: ...
- MUST fail the channel if:
- the `witness_stack` weight lowers the effective `feerate`
below the agreed upon transaction `feerate`
We consolidate to the latest/singular RFC patch for dual-funding, so
there's just a single patchfile for the change. Plus we move back to the
opener setting the desired feerate, the accepter merely declines to
participate if they disagree with the set rate.
Looks like #4394 treated a symptom but not the root cause. We were
actually sending the message framed with the WIRE_CUSTOMMSG_OUT and
the length prefix over the encrypted connection to the peer. It just
happened to be a valid custommsg...
This fixes the issue, and this time I made sure we actually send the
raw message over the wire. However for backward compatibility we
needed to imitate the faulty behavior which is 90% of this patch :-)
Changelog-Fixed: plugin: `dev-sendcustommsg` included the type and length prefix when sending a message.
We were getting a memleak error that the open_attempt isnt' being
cleaned up in test_rbf_reconnect_tx_construct. I had some trouble
reproducing it, so I removed the reliance on using `tmpctx` to clean it
up and was more surgical about cleaning it up inline.
Changelog-Added: JSON-RPC: `listpeers` now includes 'last_feerate', 'next_feerate', 'initial_feerate' and 'next_fee_step' for channels in state DUALOPEND_AWAITING_LOCKIN
fixup! listpeers: include feerate info for RBF-candidate channels
We move over to the new "warning" paradigm, instead of using
an "rbf_fail" message.
Every failure is either a warning or an error; on warnings we
hang up and reconnect later, effectively resetting the state.
We're *mostly* set up for both sides doing RBF, except that it reverses
the callback flow (using the plugin vs RPC calls) and we're not
currently smart enough to flip between them gracefully
Users have no idea what they would pay for unilateral closes.
At least this gives them a clue!
Reported-by: @az0re on IRC.
Changelog-Added: JSON-RPC: `listpeers` now shows latest feerate and unilaral close fee.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
only needed for moving the subd->channel from an uncommitted_channel to
a channel; we removed uncommitted_channel from dual_open so it's no
longer necessary
We're going to move over to "unsaved channels" instead of using a
separate "uncommitted_channel" struct. This means a peer might have a
channel that's not saved to the database yet in its channel list
We need to make sure that there's at least one input that's represented
in every single RBF-attempt for this channel, to prevent "parallel"
subsequent RBFs from succeeding/opening (the multi-channel backdoor?!)
When the funding tx reaches depth, update the channel's data to the
"correct" funding transaction info from inflights (if necessary).
This will be necessary if:
- the transaction has been successfully RBF'd and
- the lesser fee transaction is the one successfully mined, OR
- the channel is in the process of being RBF'd
We were always prefixing the `message` field with the internal type
prefix 0x0407, followed by the length prefix. Neither is needed since
the type being constant is of no interest to the plugin and the length
being implicit due to the JSON-encoding.
Reported-by: Ilya Evdokimov
Changelog-Fixed: plugin: The `custommsg` hook no longer includes the internal type prefix and length prefix in its `payload`
Changelog-Deprecated: plugin: The `message` field on the `custommsg` hook is deprecated in favor of the `payload` field, which skips the internal prefix.
This avoids spamming the logs. We also remove the duplicate debug
logs on self-disable (plugin_kill logs it for us).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We currently log every kill at INFO level, even if it's during shutdown.
Change those to debug, but lift those where we got a malformed response.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We use waitpid() manually for subdaemons, so we need to step
around that (otherwise we could simply ignore them).
We could destroy subdaemons only once they've exited, but
that works badly with the sd->conn, which will be freed
when error (i.e. close) is detected, so the current code
is probably the best compromise.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If a plugin died due to connection close, we'd always say
"Plugin exited before completing handshake.", which was often
wrong.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We used to only set it for single-use offers (where it's required),
but it's still interesting for multi-use offers, so let's keep it
there.
We also put this field in the documentation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since we turned many errors into warnings, we want our tests to fail
when they happen unexpectedly. We make WARNING clear in the strings
we print, too, to help out.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
No more sending "all-channel" errors; in particular, gossipd now only
sends warnings (which make us hang up), not errors, and peer_connected
rejections are warnings (and disconnect), not errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `peer_connected` rejections now send a warning, not an error, to the peer.
This is in line with the warnings draft, where all-zeroes in a
channel_id is no longer special (i.e. it will be ignored).
But gossipd would send these if it got upset with us, so it's best
practice to ignore them for now anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we treat error messages from peer which refer to "all channels" as warnings, not errors.
We fix up the test by using pay, instead of sendpay (and making pay log
the expected message).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: sendpay no longer extracts updates from errors, the caller should do it from the `raw_message`.
From #clightning:
(11:24:10) andytoshi: hiya, i'm trying to set up a new lightningd node, and when i run lightningd --network=bitcoin --log-level=debug --daemon
(11:24:17) andytoshi: i get errors of the form fetchinvoice: Malformed JSON reply '2021-01-25T00:51:16.655Z DEBUG plugin-offers: disabled itself at init: offers not enabled in config
(11:24:43) andytoshi: there are a couple variants of this, but always some form of "something: failed to parse <a log line> as json"
Indeed, we close stdout, and it ends up being reused for some plugin.
But the real problem is that we log to stdout by default, which doesn't
make sense. If they really want to discard logs, they can use
--log-file=/dev/null.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON failures when --daemon is used without --log-file.
rearranges the`peer_connected_hook_payload` definition to the location
where this is used in the file.
Fixes certain blanklines and linebreaks to make the code look nicer.
Nit: The underscore in "openchannel_hook" is wrong, bcause the name of
the hook is just "openchannel". The "_hook" implied this to be part of
the name.
Changelog-None
It doesn't make sense currently without EXPERIMENTAL_FEATURES, and
check-manpages complains about it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
payload is owned by the peer, which is freed in this case, then we
free payload (again).
==1404== Invalid read of size 8
==1404== at 0x1F39E8: to_tal_hdr (tal.c:174)
==1404== by 0x1F43A4: tal_free (tal.c:479)
==1404== by 0x14B3D1: peer_connected_hook_cb (peer_control.c:1087)
==1404== by 0x15D6E9: plugin_hook_call_ (plugin_hook.c:288)
==1404== by 0x14B40E: plugin_hook_call_peer_connected (peer_control.c:1090)
==1404== by 0x14B5B8: peer_connected (peer_control.c:1135)
==1404== by 0x122FCF: connectd_msg (connect_control.c:310)
==1404== by 0x160291: sd_msg_read (subd.c:480)
==1404== by 0x15FBE7: read_fds (subd.c:308)
==1404== by 0x1E37D1: next_plan (io.c:59)
==1404== by 0x1E434E: do_plan (io.c:407)
==1404== by 0x1E438C: io_ready (io.c:417)
==1404== Address 0x2fcd2268 is 24 bytes inside a block of size 336 free'd
==1404== at 0x4C32D3B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1404== by 0x1F416E: del_tree (tal.c:421)
==1404== by 0x1F40F2: del_tree (tal.c:412)
==1404== by 0x1F442C: tal_free (tal.c:486)
==1404== by 0x148816: delete_peer (peer_control.c:120)
==1404== by 0x148899: maybe_delete_peer (peer_control.c:136)
==1404== by 0x13A970: destroy_uncommitted_channel (opening_common.c:29)
==1404== by 0x1F3BB1: notify (tal.c:240)
==1404== by 0x1F40A0: del_tree (tal.c:402)
==1404== by 0x1F442C: tal_free (tal.c:486)
==1404== by 0x13D3E9: peer_start_openingd (opening_control.c:911)
==1404== by 0x14B3C2: peer_connected_hook_cb (peer_control.c:1086)
==1404== Block was alloc'd at
==1404== at 0x4C31B0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1404== by 0x1F3C1B: allocate (tal.c:250)
==1404== by 0x1F41B4: tal_alloc_ (tal.c:428)
==1404== by 0x14B454: peer_connected (peer_control.c:1105)
==1404== by 0x122FCF: connectd_msg (connect_control.c:310)
==1404== by 0x160291: sd_msg_read (subd.c:480)
==1404== by 0x15FBE7: read_fds (subd.c:308)
==1404== by 0x1E37D1: next_plan (io.c:59)
==1404== by 0x1E434E: do_plan (io.c:407)
==1404== by 0x1E438C: io_ready (io.c:417)
==1404== by 0x1E6552: io_loop (poll.c:445)
==1404== by 0x12E2AD: io_loop_with_timers (io_loop_with_timers.c:24)
Fixes: #4329
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The fetchinvoice and offers plugins disable themselves if the option
isn't enabled (it's enabled by default on EXPERIMENTAL_FEATURES).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-offers` enables fetch, payment and creation of (early draft) offers.
By returning 'disable: <reason>' inside getmanifest or init result.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: plugins: plugins can now disable themselves by returning `disable`, even if marked important.
Note that this also changes so the feature is not represented in channels,
reflecting the recent drafts.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-onion-messages` enables send, receive and relay of onion messages.
We weren't sending a channel_open notification for dual-funded channels.
This is only sent for the 'accepter' side. We send it as soon as both
funding_tx sigs have been exchanged, even though it's possible the
funding transaction might be published without this having been the case.
Since we fail the channel if this happens, only notifying for good/valid
channels reaching the broadcast state is the right way to handle this.
Allow a user to switch on dual-funding without needing to compile
as EXPERIMENTAL_FEATURES.
Doesn't work yet, since everything is still behind
'EXPERIMENTAL_FEATURES' compile time flags... but useful for testing
Let plugins know when a channel open has failed.
We need to notify accepters now too, so we remove the check on who's
funding the channel before sending the 'failed' message from
dualopend->master.
We already know what the channel id is, we should go ahead and pass it
on to any listening plugins -- this makes clean up easier/possible
if a open fails early on and we've got reserved utxos.
When we're the opener, we get the upfront shutdown scriptpubkey (if
there is one) from the `close_to` param of `openchannel_init`.
We were passing it through dualopend, but we need to break the update
chain so that our test_option_upfront_shutdown_script test works (same
as on the openingd flow.)
Prior to this, all reconnect logic lived in channeld. If you
disconnected before we finished building a funding transaction, that was
no big deal. Now, however, we're waiting for the funding to lock in in
dualopend, instead of handing straight to channeld to wait.
So we need a way to restart dualopend.
We delegate the decision about what to do about 'out of bound' feerates
to the plugin (if one exists), however in the case that the plugin
doesnt exist or doesnt want to figure it out, we default to using the
'best' as their intended feerate, and rejecting if it's out of bounds.
We need to know if they've sent us their sigs message yet. Ideally, we'd
be able to check the 'finalness' of the PSBT, however if the peer
doesn't have any inputs to the channel this doesn't work.
Back in the days before dual-funding, the `channel` struct on subd was
only every one type per daemon (either struct channel or struct
uncommitted_channel)
The RBF requirement on dualopend means that dualopend's channel,
however, can now be two different things -- either channel or
uncommitted_channel.
To track the difference/disambiguate, we now track the channel type on a
flag on the subd. It gets updated when we swap out the channel.
This will make it possible to do RBF, since we can re-start the opening
process in dualopend while waiting for lock-in.
Note the new channel states are being used, DUALOPEND_INIT and
DUALOPEND_AWAITING_LOCKIN, to differentiate from openingd/channeld opens
v2 channel opens are going to happen over in dualopend. In order
to make sure that these don't end up in the wrong place/to keep track of
the difference between "waiting for sigs" and "have merely initiatlized
a channel", we add two new states to the channel state machine.
A channel that 'originates' in dualopend will only ever arrive at
channeld in the state CHANNELD_NORMAL.
Since this all stays in dualopend/dual_open_control, we can hold
onto the openchannel_signed command to wait for a response here locally.
Previously we were splitting across the channeld/openingd boundary.
We used this for dual funded opens, to track the receipt of signatures.
We're moving all of this over to dualopend now, however, so we no longer
need the PSBT in channeld.
This is fixed when payload is freed, but I noted a leak in the case of
an invalid payload:
```
E ValueError:
E Node errors:
E Global errors:
E - Node /tmp/ltests-ipt6ab_y/test_sendinvoice_1/lightning-2/ has memory leaks: [
E {
E "backtrace": [
E "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
E "ccan/ccan/tal/tal.c:471 (tal_alloc_arr_)",
E "gossipd/gossipd_wiregen.c:925 (fromwire_gossipd_got_onionmsg_to_us)",
E "lightningd/onion_message.c:99 (handle_onionmsg_to_us)",
E "lightningd/gossip_control.c:166 (gossip_msg)",
E "lightningd/subd.c:480 (sd_msg_read)",
E "ccan/ccan/io/io.c:59 (next_plan)",
E "ccan/ccan/io/io.c:407 (do_plan)",
E "ccan/ccan/io/io.c:417 (io_ready)",
E "ccan/ccan/io/poll.c:445 (io_loop)",
E "lightningd/io_loop_with_timers.c:24 (io_loop_with_timers)",
E "lightningd/lightningd.c:1016 (main)"
E ],
E "label": "gossipd/gossipd_wiregen.c:925:u8[]",
E "parents": [
E "lightningd/onion_message.c:96:struct onion_message_hook_payload",
E "lightningd/plugin_hook.c:87:struct hook_instance *[]"
E ],
E "value": "0x560779438db8"
E }
E ]
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Our new "decode" command will also handle bolt11. We make a few cleanups:
1. Avoid type_to_string() in JSON, instead use format functions directly.
2. Don't need to escape description now that JSON core does that for us.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The previous onion_message code required a confirmed, not-shutting-down
channel, not just a connection. That's overkill; plus before widespread
adoption we will want to connect directly as a last resort.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is required if we want to create a "bouncer" plugin (in my copious free time!)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `invoice` now takes an optional `cltv` parameter.
This makes for more useful errors. It prints where it was up to in
the guide, but doesn't print the entire JSON it's scanning.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Interestingly, we required that "sendrawtx" include "errmsg" field even
on success, otherwise we crashed in broadcast_remainder.
We only actually insist on an "errmsg" if success is false. And this
logic here is weird (the !success) was added by darosior in
947f5ddde1, which makes the msg checks redundant.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
A user reported that it is sometimes cumbersome to search an invoice
based on the payment hash or the bolt11 string in the full list, which
may be required when we don't have the label available.
This adds support for querying / filtering based on the `payment_hash`
or `bolt11` string.
Changelog-Added: JSON-RPC: `listinvoices` can now query for an invoice matching a `payment_hash` or a `bolt11` string, in addition to `label`
This makes use of the constant defined in the previous commits to more
accurately detect plaintext, encrypted, and invalid seeds. We now error
on invalid seeds.
Changelog-changed: hsmd: we now error at startup on invalid hsm_secret
Changelog-changed: hsmtool: all commands now error on invalid hsm_secret
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
This avoids duplication of both logic and error-prone values, such as
the salt. Grouping all hsm encryption logic into a public API will also
allow us to fuzz it.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
Changelog-changed: lightningd: the `--encrypted-hsm` now asks you to confirm your password when first set
Changelog-changed: hsmtool: the `encrypt` now asks you to confirm your password
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
This is similar to the createinvoice API, except we don't need to save
invoice requests in the database. We may, however, have to look up
payment_key for recurring invoice requests, and sign the message with
the payment_key.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Invoices are signed with our own key, but we use a transient payer_key with a
tweak for invoice_requests (and refunds).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. Hoist 7200 constant into the bolt12 heade2.
2. Make preimage the last createinvoice arg, so we could make it optional.
3. Check the validity of the preimage in createinvoice.
4. Always output used flag in listoffers.
5. Rename wallet offer iterators to offer_id iterators.
6. Fix paramter typos.
7. Rename `local_offer_id` parameter to `localofferid`.
8. Add reference constraints on local_offer_id db fields.
9. Remove cut/paste comment.
10. Clarify source of fatal() messages in wallet.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This takes an unsigned bolt11 (or bolt12 if EXPERIMENTAL_FEATURES) string
and signs it and puts it in the database.
The invoice command could now be moved out to a plugin, in fact.
Changelog-Added: JSON-RPC: `createinvoice` new low-level invoice creation API.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is for offers which have `send_invoice`: we need to associate the
payment with the original offer, in (the usual) case where it is a single
use offer. We mark it used when it's paid, to avoid a race.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This allows us to mark an offer used when an invoice derived from it
is paid, and importantly, avoid any other invoices for the offer being
paid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
"multi" means that specifying a parameter twice will append, not override.
Multi args are always given as a JSON array, even if only one.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: new "multi" field allows an option to be specified multiple times.
This was fixed in 0.8.2.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: plugins: options to init are no longer given as strings if they are bool or int types (deprecated in 0.8.2).
Still asserts that it's the standard size, but makes it a dynamic
member. For simpliciy, changes the parse_onionpacket API (it must be
a tal object now, so we might as well allocate it here to catch all
the callers).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were masquerading errors when parsing the request by reporting only
a bogus malformed `id` field in the response, when the real issue was
that we were unable to parse the request in the first place (which
caused the null-id error to be returned).
Fixes#4238
Only way to be sure that plugins don't accidentally respond to onion_message
sent via reply path from another message (which would potentially leak our
identity!).
To quote BOLT #7 (Onion Messages) in the offers PR:
```markdown
The reader:
- MUST ignore any message which contains a `blinding` which it did not expect, or does not contain
a `blinding` when one is expected.
...
`blinding` is critical to the use of blinded paths: there are various
means by which a blinded path is passed to a node. The receipt of an
expected `blinding` indicates that blinded path has been used: it is
important that a node not accept unblinded messages when it is expecting
a blinded message, as this implies the sender is probing to detect if
the recipient is the terminus of the blinded path.
Similarly, since blinded paths don't expire, a node could try to use
a blinded path to send an unexpected message hoping for a response.
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Required to determine if this msg used expected reply path.
Also remove FIXME (om->enctlv is handled above).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Found in tests/test_connection.py::test_restart_many_payments:
`lightningd: outstanding taken(): lightningd/peer_htlcs.c:532:towire_temporary_channel_failure(((void *)0), ((void *)0))`
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There's actually a (very unlikely) race here: we would previously have
crashed with an assertion in invoices_resolve.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We register on it for offers, and without this nobody else can.
Changelog-Changed: plugins: more than one plugin can now register invoice_payment hook.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Avoids much cut & paste. Some tests don't need any of it, but most
want at least some of this infrastructure.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We already do some sanity checks, add this one.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: invalid UTF-8 strings now rejected.
This effectively reverts ac93b780d5.
Christian points out that plugins need time before we deprecate
the old options (probably 6 months) as they need to work with
both old and new.
Changelog-Deprecated: **UNDO** plugins: hooks should now be specified using objects, not raw names.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Incase we have been offline while a channel was force closed on us we
now set the 'closer' to 'remote' instead of null because this is by far
the most probable reason.
Changelog-None
This adds a missing plugins_send_getmanifest() call in the rescan function
that lead to a RPC hangup. Not sure though if this is the proper fix.
Changelog-None
We previously registered hooks up in who-replies-to-getmanifest-first
order, but then if any had dependencies it would scatter that order.
This allows users to manually set dependencies developers have
forgotten by specifying the plugins manually in their configuration or
cmdline. This was an excellent consideration by @mschmook.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now both python and c libraries are updated, we can officially
deprecate the old form.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: plugins: hooks should now be specified using objects, not raw names.
The next patch will use these to order the hooks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: plugins: hooks can now specify that they must be called 'before' or 'after' other plugins.
This adds a `state_change` 'cause' to a channel.
A 'cause' is some initial 'reason' a channel was created or closed by:
/* Anything other than the reasons below. Should not happen. */
REASON_UNKNOWN,
/* Unconscious internal reasons, e.g. dev fail of a channel. */
REASON_LOCAL,
/* The operator or a plugin opened or closed a channel by intention. */
REASON_USER,
/* The remote closed or funded a channel with us by intention. */
REASON_REMOTE,
/* E.g. We need to close a channel because of bad signatures and such. */
REASON_PROTOCOL,
/* A channel was closed onchain, while we were offline. */
/* Note: This is very likely a conscious remote decision. */
REASON_ONCHAIN
If a 'cause' is known and a subsequent state change is made with
`REASON_UNKNOWN` the preceding cause will be used as reason, since a lot
(all `REASON_UNKNOWN`) state changes are a subsequent consequences of a prior
cause: local, user, remote, protocol or onchain.
Changelog-Added: Plugins: Channel closure resaon/cause to channel_state_changed notification
This will allow nodes (with log-level=debug) to gather how many payments
are made without payment_secrets. We need to know this so we know when
we can make them compulsory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We let the plugin decide what feerate to accept/whether or not to add
funds to the open. To aid this decision, we also send the plugin what we
(c-lightning) currently have as our max and min acceptable feerates.
We also now use these as our default for max/min acceptable feerate
range when sending an openchannel offer to a peer.
In the future, it might be a good idea to make these more easily
changeable, either via a config setting (?) or a command param.
The previous dev-force-features forced you to explicitly declare every
desired feature bit in an array, for each set.
Here, we allow you to also denote adding/subtracing a feature bit
by just passing in the number of the bit to flip and the direction to
turn it. e.g.
'dev-force-features': '+223'
Will turn on opt_dual_fund/odd.
'dev-force-features': '-16'
Will flag off opt_basic_mpp.
"make clean" removes all object files listed in $(ALL_OBJS), which is derived from $(ALL_C_SOURCES).
Source files in $(LIGHTNINGD_SRC_NOHDR) are missing from $(ALL_C_SOURCES) therefore some object files are not removed by "make clean".
This commit fixes this point.
Signed-off-by: YOSHIDA Masanori <masanori.yoshida@gmail.com>
Changelog-None
For compatibility, we only do this if `allow-deprecated-apis` is false
for now. Otherwise scripts parsing should use `grep -v '^# '` or
start using `-N none`.
Changelog-Added: JSON-RPC: `close` now sends notifications for slow closes (if `allow-deprecated-apis`=false)
Changelog-Deprecated: cli: scripts should filter out '^# ' or use `-N none`, as commands will start returning notifications soon
Fixes: #3925
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This lets callers enable notifications; we won't send any if they don't.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `notifications` command to enable notifications.
And make caller of json_stream_forward_change_id use it, since
we're going to reuse that.
Also call json_out_finished here, so next object doesn't have a ","
prepended.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that check-whitespace and check-bolt already do this, so we
can eliminate redundant lines in common/Makefile and bitcoin/Makefile.
We also include the plugin headers in ALL_C_HEADERS so they get
checked.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This will allow us to build complex, multi-peer transactions, with
easeTM!
Changelog-Added: EXPERIMENTAL, Plugins: `openchannel_peer_sigs` notification, which contains a peer's signatures for the funding transaction (`opt_dual_fund`)
Prior to this patch update, we expected a client to call
`openchannel_signed` before checking for peer's tx-sigs messages on the
wire.
When moving to a 'multifundchannel' approach, we'll need to be able to
collect sigs from our peers before sending our tx_sigs message. There's
no strict ordering on when tx-sigs messages are sent/received, so this
is fine.
To do this, we go ahead and start up channeld as soon as
commitment_sigs are secured, so that we process incoming tx-sigs from
our peers as soon as we get them.
This is handy/necessary for getting multifundchannel to work, as we need
to know what output to tell all the other peers about.
Changelog-Added: Experimental!! JSON-RPC: openchannel_init returns a field `funding_serial` that indicates the serial_id of the funding output in the provided PSBT
There's a few structs/wire calls that only exist under experimental features.
These were in a common file that was shared/used a bunch of places but
this causes problems. Here we move one of the problematic methods back
into `openingd`, as it's only used locally and then isolate the
references to the `witness_stack` in a new `common/psbt_internal` file.
This lets us remove the iff EXP_FEATURES inclusion switches in most of
the Makefiles.
Rusty pointed out that having an empty channel_id is suboptimal; adding
another call is probably the right idea rather than re-using an existing
one.
Suggested-By: @rustyrussell
...right time.
We re-send the tx_sigs on start/init/reconnect until we've gotten a
funding_locked from our peer. We also build it in channeld now, instead
of in dualopend, and don't pass in a message for them anymore
`openchannel_signed` and `openchannel_update` which allow a user to
continue a openchannel or kick off the completion of a openchannel.
`openchannel_update` should be called until it returns with
`commitments_secured`.
`openchannel_signed` commands hang out across the openingd/channeld
boundary -- we don't return until we've successfully broadcast the
transaction (or timed out waiting for them to send a tx_sigs back).
We need the PSBT to create the finalized tx from once the peer's
tx_signatures are received. Since we're passing the PSBT, we no longer
need the secondary message to be passed, as it was derived from the
PSBT.
Also removes now unused witness serialization code
There are 3 commands for opening a channel with dualfunding.
`openchannel_init` is the first of these.
It initializes the open-channel dialog, and stops once we've run out of
updates (input/outputs) to send to the peer.
Great report from whitslack on this crash at startup:
```
2020-10-07T13:03:21.419Z **BROKEN** lightningd: FATAL SIGNAL 6 (version 0.9.1)
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: common/daemon.c:51 (crashdump) 0x559fb67bcc76
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0 ((null)) 0x7f61cdca8baf
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ../sysdeps/unix/sysv/linux/raise.c:50 (__GI_raise) 0x7f61cdca8b31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/stdlib/abort.c:79 (__GI_abort) 0x7f61cdc92535
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:92 (__assert_fail_base) 0x7f61cdc9241e
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:101 (__GI___assert_fail) 0x7f61cdca1241
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:750 (subd_send_msg) 0x559fb67a1c31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:745 (subd_send_msg) 0x559fb67a1c31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:252 (local_fail_in_htlc) 0x559fb6798f77
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1441 (onchain_failed_our_htlc) 0x559fb6798f77
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:339 (handle_missing_htlc_output) 0x559fb6786b9d
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:455 (onchain_msg) 0x559fb6786b9d
```
The problem is a channel with an onchaind can be in state FUNDING_STATE_SEEN,
because onchaind has started but not responded to init yet (which it does once it
has analyzed the commitment tx).
Channel B is onchain, and its onchaind fails the HTLC, and we try to send a msg
to channel A's onchaind as if it were channeld.
Explicitly check if it's channeld, rather than trying to see if it's onchaind.
Fixes: #4114
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: crash: assertion fail at restart when source and destination channels of an HTLC are both onchain.
The 100MB log buffer has been the biggest memory footprint for the daemon.
Keeping 10MB for emergency log dumps seems sufficient.
This has been mentioned in the last developer meeting.
Changelog-Changed: In-memory log buffer reduced from 100MB to 10MB
I got a corrupt file, which looked like multiple concurrent attempts
to build it. So instead, build it in one command, but also use
VERBOSE so we print correctly with V=1 (and --quiet).
Also move into plugins/ where it logically belongs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We force use of tal_wally_start/tal_wally_end around every wally
allocation, and with "end" make the caller choose where to reparent
everything.
This is particularly powerful where we allocate a tx or a psbt: we
want that tx or psbt to be the parent of the other allocations, so
this way we can reparent the tx or psbt, then reparent everything
else onto it.
Implementing psbt_finalize (which uses a behavior flag antipattern)
was tricky, so I ended up splitting that into 'psbt_finalize' and
'psbt_final_tx', which I think also makes the callers clearer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
wallet_commit_channel would fill in the old_remote_per_commit and
fee_states, which is weird since the caller doesn't care.
Make the caller set all the channel_info fields, so wallet_commit_channel
is a simple consumer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I mistakenly assumed the block would be freed after processing completed. That
is not true since chaintopology keeps headers and stubs around for reorgs. So
we need to remove the precomputed txids along with the full_txs.
1. Rename memleak_enter_allocations to memleak_find_allocations.
2. Unify scanning for pointers into memleak_remove_region / memleak_remove_pointer.
3. Document the functions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We promised this in 0.8.1!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Plugin: Relative plugin paths are not relative to startup (deprecated v0.7.2.1)
Changelog-Removed: JSON API: The hook `db_write` can no longer return `true` (deprecated in 0.8.1)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is simple, and we now can multifundchannel to every node on testnet
(one simply hangs once we connect).
Changelog-Fixed: Protocol: We now hang up if peer doesn't respond to init message after 60 seconds.
Changelog-Added: JSON-RPC: delpay a new method to delete the payment completed or failed.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
This notification will be raised whenever a channel state changes.
The payload includes the channel and peer identifiers and the
old and the new state.
Example payload:
```
{
"channel_state_changed": {
"peer_id": "03bc9337c7a28bb784d67742ebedd30a93bacdf7e4ca16436ef3798000242b2251",
"channel_id": "a2d0851832f0e30a0cf778a826d72f077ca86b69f72677e0267f23f63a0599b4",
"short_channel_id" : "561820x1020x1",
"old_state": "CHANNELD_NORMAL",
"new_state": "AWAITING_UNILATERAL"
}
}
```
Changelog-Added: Plugins: channel_state_changed notification
As the cmd gets freed on a received error, the node id in which we iterate in `process_check_funding_broadcast`
may gets freed while we are using it.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
Fixing the following error by changing 'enum feerate' to int.
lightningd/bitcoind.c:183:29: error: result of comparison of constant
8 with expression of type 'enum feerate' is always true [-Werror,
-Wtautological-constant-out-of-range-compare]
for (enum feerate f = 0; f < NUM_FEERATES; f++) {
Changelog-Fixed: compile error on macos
