The pattern of making tal-allocated copies of wally data to pass around
was made redundant after these calls were added by the use of
tal_wally_start/tal_wally_end to parent wally allocations. We can thus
just pass the data directly and avoid the allocations.
Removes redundant allocations when checking tx filters and computing fees.
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
Standardizes the is_xxx script function all take a script length, and changes
their first-level callers to pass it. This has several knock on benefits:
- We remove the repeated tal_count/tal_bytelen calls on the script, in
particular the redundant calls that result when we must check for multiple
types of script - which is almost all cases.
- We remove the dependency on the memory being tal-allocated (It is, in
all cases, but theres no reason we need to require that).
- We remove all cases where we create a copy of the script just to id it.
- We remove all allocations for non-interesting scripts while iterating block
txs in process_getfilteredblock_step1().
- We remove all allocations *including for potentially interesting scripts* in
topo_add_utxos().
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
We were extracting the output script for all outputs, and discarding
them immediately again if they were not P2WSH outputs which are the
ones of interest to us. This patch move the extraction until after we
have determined it is useful, and so we should save a couple thousand
`tal()` and `tal_free()` calls.
Changelog-Changed: lightningd: Speed up blocksync by not parsing unused parts of the transactions
Watchtowers changed the code so that we *always* have a channel->shutdown_scriptpubkey[LOCAL]
(see new_channel()). The previous code had several problems:
1. It tested this for NULL, unnecessarily.
2. It allowed overriding if it was a default, *even* if we were already using it.
3. If the peer opened without option_shutdown_anysegwit, but upgraded before we closed,
we would not recognize the default.
4. It set the final scriptpubkey (and other things!) even if the command failed.
Changelog-Fixed: JSON-RPC: `close` with `destination` works even if prior `destination` was rejected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If we accept a channel_update in state "NEED_SIGS" we should not set
refresh timer: we're simply holding it for the moment we get to that state
(which will happen as we mine the block).
```
0x7fd1cce39205 __assert_fail
./assert/assert.c:101
0x55c103cc6ee9 check_channel_gossip
lightningd/channel_gossip.c:128
0x55c103cc8a13 channel_gossip_update_from_gossipd
lightningd/channel_gossip.c:821
0x55c103cd752d handle_init_cupdate
lightningd/gossip_control.c:138
0x55c103cd79a3 gossip_msg
lightningd/gossip_control.c:190
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was triggered by the recover plugin tests (not yet merged!) and causes a crash
because we don't have signatures yet. It can only happen if we lost our database,
but at least don't crash!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We still want to test non-anchor channels, as we still support them, but
we've made it non-experimental. To test non-anchor channels, we
use dev-force-features: -23.
Changelog-Added: Protocol: `option_anchors_zero_fee_htlc_tx` enabled, no longer experimental.
Changelog-Changed: Config: `experimental-anchors` now does nothing (it's enabled by default).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Header from folded patch 'fixup!_options__make_anchors_enabled_by_default,_ignore_experimental-anchors.patch':
fixup! options: make anchors enabled by default, ignore experimental-anchors.
This simplifies our tests which will want to turn off anchors,
even though they won't be set for elements.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We tried to open a channel with feerate 0 in this case! Rework so it's
clear that we have two different feerates here.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's an unnecessary round-trip, and can cause us to complain in CI, in
the case where the channel has been closed by the time we ask.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It was an obscure dev command, as it never worked reliably.
It would be much easier to re-implement once this is done.
This turned out to reveal a tiny leak on
tests/test_gossip.py::test_gossip_store_load_amount_truncated where we
didn't immedately free chan_ann if it was dangling.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This caused a flake in test_gossip_lease_rates, since the peer would ignore
the node_announcement due to duplicate timestamps!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This isuseful to find completely dead channels which are worthy of
closing.
Changelog-Added: JSON-RPC: `listpeerchannels` field `last_stable_connection` showing when we last held an established channel for a minute or more.
Changelog-Added: JSON-RPC: `listclosedchannels` field `last_stable_connection` showing when we last held an established channel for a minute or more.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a nice reflection of channel stability: in particular, worse case
ping time is 45 seconds, so we should have had some traffic.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is useful to see if we've really made progress, or just bounced
off the peer (e.g. it doesn't know the channel, or we have fallen
behind somehow).
Changelog-Added: JSON-RPC: `listpeerchannels` new field `reestablished` set once we've exchanged `channel_reestablish` messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In particular, we were sending `announcement_signatures` before
`channel_reestablish`; we allow this because LND used to do it, but
it's not spec compliant.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It then waits 10 more seconds (for plugins to call setlease, especially)
before it will update a node_announcement.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rather than take the first two from peers with committed channels, use
the most common address given by at least 2 peers, and accept the majority
from non-committed peers if there are no committed peers.
This works well with the node_announcement rework, which waits until
everyone has a chance to connect before creating the node_announcement.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now lightningd just doesn't tell us about private channels, doesn't
expect us to generate channel gossip, and tells us about public channels
via the addgossip call, we don't need any of this in gossipd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This commit is a bit messy, but it tries to do the minimal switchover.
Some tests change, so those are included here.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Don't return false on db errors (we always fail on those), but return
false if they don't exist.
Also, add routine to clear them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Let's tell the caller what channel_type they got!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundchannel`, `multifundchannel`, `fundchannel_start` and `openchannel_init`: new field `channel_type`.
As suggested in https://github.com/lightning/bolts/pull/1092.
We still support channels opened without it, but you can no longer open new ones without it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_gossip_queries` is now required (advertized by all but 16 nodes)
We're about to make static_remotekey compulsory, but we still want to
do tests for pre-existing channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And add request schemas for openchannel_init and fundchannel_start.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundchannel_start` and `openchannel_init` now take an optional `channel_type` parameter.
As suggested in https://github.com/lightning/bolts/pull/1092.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_gossip_queries` is now required (advertized by all but 11 nodes)
As suggested in https://github.com/lightning/bolts/pull/1092.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_data_loss_protect` is now required (advertized by all but 11 nodes)
Unfortunately, this is awkward: we just copy through most requests,
so we can't easily add a "deprecation" field to each one. So we do
a notification if the next command has a different deprecation status
than the global one, but that requires opt-in from the plugin.
We didn't previously document the subscriptions array, so do that now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `deprecated_oneshot` notifiction subscription to change deprecated status for a single command.
I did some CHANGELOG and git digging to see when these were deprecated, and
some were very old (v0.8.2!). But since they didn't warn users loudly, I
chose to do so this release only.
I renamed ld's `deprecated_apis` to `deprecated_ok` to make sure I
caught them all.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We still accept boolean: the plugin may not want to commit to a deprecation schedule.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: rpcmethods and options can set `deprecated` to a pair of version strings, not just a boolean.
This command allows more fine-grained testing, without having to change the config of the
lightning node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `deprecations` to enable/disable deprecated APIs from this caller.
This allows the user to specify the feature *by name*, and hopefully
complain to the developer to fix their code, knowing it will be removed entirely
in the next release!
Changelog-Added: config: `i-promise-to-fix-broken-api-user` allows for a one-release re-enablement of long-deprecated features.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Config: `disable-ip-discovery` (deprecated in v23.02): use `announce-addr-discovered`
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The `struct notification` lost type-safety, but avoided a redundant
string. The string is better, I think.
Since all notifications now contain an object of same name (some have
deprecated fields outside that), we can add helpers to do that, too.
Also, add some const (easy to do now we're typesafe!)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: hsmd: Added hsmd_forget_channel to enable explicit channel deletion. ([#6987])
Motivation: Previously, a signer prematurely forgetting a channel led
to failures in unresolved channel requests. This update introduces
hsmd_forget_channel, allowing nodes to explicitly notify signers when
a channel is irrevocably resolved and can be safely forgotten. This
ensures synchronized channel cleanup between nodes and signers.
This change maintains backward and forward compatibility. Nodes
explicitly check whether a signer has `WIRE_HSMD_FORGET_CHANNEL`
capability before sending the message. Nodes without
`WIRE_HSMD_FORGET_CHANNEL` capability won't send this message. Signers
capable of handling this message but not receiving it will continue to
use conservative pruning methods.
Fixes#6987
Removes the tal_count checking overhead when iterating constant arrays.
Separated from the previous commit to make review easier.
Changelog-None
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
- Avoid overhead from tal checks when iterating block txs
- Skip pegin txs as well as coinbase txs while iterating
- Early-exit if the txout cannot possibly be p2wsh
- Don't re-calculate the txid when we already have it
- Don't allocate a script for non-policy asset outputs
- Don't copy txids for non-interesting UTXOs
Note the below -Changed line covers the previous wally and PSBT commits
which also provide general block processing speedups.
Changelog-Changed: core: Processing blocks should now be faster
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
Create a notification that is triggered when a `costummsg` is received.
Changelog-Added: Plugins: notification custommsg for receiving an unknown protocol message
In the original [$6173] pr `bind` was a typo for `bind-addr`.
`bind` never existed, so this commit updates uses of `bind` to `bind-addr`.
This links lightningd-config.5.md to configuration.md since `bind-addr` has options of interest such as ipv4/ipv6.
We temporarily use a second gossmap so we can just switch private info off
for listincoming and not listchannels.
Note that listchannels now uses the local alias (if no scid), so we have
to change that in the routehint caller.
Since we now *always* use a channel alias in hints if one exists, a
test broke, so fix that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is redundant if it's a public channel, but vital if it's not. Publishing unconditionally makes
it easier for gossmap: we create a local modification all the time, even if redundant (and we can
have the actual capacity ceiling accurate in this case, since we know it for local channels).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listpeerchannels` now shows gossip update contents (even if channel unannounced).
Not just when it's a private channel. This is useful for listpeerchannels in the next patch.
Most of this is renaming.
It also means that source can be NULL, so move it out of the struct and put it in the message,
where it logically belongs, and make it an optional field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
and stash in the database.
Rusty: I added the bad gossip message so we would see unknown updates in CI, and made sure we don't send our own generated updates to lightningd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Turns out we were sending feerate updates to daemons that do not
understand it. Don't do that!
Closes#6932
Changelog-Fixed: channeld: We could crash `closingd` by sending it a `channeld` message
We were (dumbly?) using the `onchain_hrp` for 'chain_mvts' and the
`lightning_hrp` for 'channel_mvts'.
This works fine everywhere *except* for on a signet, where we use
different prefixes.
Since the lightning-hrp set is more diversified (testnet btc
+ signet btc use the same HRP 'onchain'), let's use that.
Should have zero impact on anything other than nodes running on signet.
To preserve your current accounts database without needing to delete,
restart, execute the following: (note preferrably when your node isn't
running).
```
UPDATE chain_events SET currency = 'tbs' WHERE currency = 'tb';
```
Fixes#6534
Changelog-Fixed: `bkpr-listbalances` would crash for nodes on signet with payments in channels, because onchain events were using a different currency than inchannel events.
Reported-by: Shahana Farooqui
Changelog-Fixed: JSON-RPC: Plugin notification `msat` fields in `invoice_payment` and `invoice_created` hooks now a number, not a string with "msat" suffix.
Changelog-Fixed: JSON-RPC: Plugin hook `payment` `msat` field is now a number, not a string with "msat" suffix.
Adds tests for when the connection fails during
1) splice tx_signature
2) splice commitment_signed
Fleshed out the reestablish flow for these two cases and implemented the fixes to make these reestablish flows work.
Part of this work required changing commit process for splices: Now we send a single commit_part for the splice where previously we sent all commits, and accordingly, we no longer revoke in response.
Changelog-Fixed: Implemented splicing restart logic for tx_signature and commitment_signed. Splice commitments are reworked in a manner incompatible with the last version.
Since these worked in v23.08, we can't just rename them. So if they are
used and unclaimed, we should rename them internally (if they're claimed,
it's probably clightning-rest, and we should *NOT* touch them!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Plugins: `clnrest` parameters `rest-port`, `rest-protocol`, `rest-host` and `rest-certs`: prefix `cln` to them
If we disconnect, we lose the open_attempt record. Which is fine, but we
should prevent the user from starting another RBF if the last one isn't
done yet!
When we got our peer's sigs, if we were the remote, we would re-notify
the plugin, which in turn would re-send the tx-sigs to use.
In the case of CLN, we'd then
- break, because we'd re-forward the sigs to the `openchannel` plugin,
which was then in the wrong state (MULTIFUNDCHANNEL_SIGNED)
spenderp: plugins/spender/openchannel.c:598: json_peer_sigs: Assertion `dest->state == MULTIFUNDCHANNEL_SECURED' failed.
spenderp: FATAL SIGNAL 6 (version 5880d59-modded)
In the case of eclair, they'd just see our 2nd TX_SIGS message and
@t-bast would complain:
> This test works, with one minor issue: on reconnection, cln sends its tx_signatures twice (duplicate?).
This commit does two things:
- has the openchannel / spender plugin log a broken instead of
crashing when the state is not what we're expecting
- stops us from calling the `funder` plugin if this is a
replay/second receipt of commit-sigs.
We need to keep track of if we've gotten the last negotiation's
commitment sigs, for reconnect logic (helps us know what messages to
send in the reconnect case)
If an openchannel_update fails (due to disconnect etc) it's possible
that it could 'resolve' itself later due to the auto reconnect logic
If you call an openchannel_update and we've already got an inflight
record saved, go ahead and return the info from the inflight (including
info about whether or not the commitments are secured.)
This makes openchannel_update a bit more 'robust'/idempotent, in that
you can make repeat calls to it after the channel is inflight and get
the info you need back to continue (call openchannel_signed)
Changelog-Changed: RPC: `openchannel_update` will now echo back a result if there's a matching inflight record for this open.
Since we can now get a COMMITMENT_SIGNED message due to a reconnect,
in addition to the 'inline' open process, it's possible that we might
have cleaned up / lost the open_attempt object.
This is fine, we have (almost) all the data we need to round this off
successfully/send out a notice.
Note that the only exception is the `close_to` data is lost/forgotten in
the case of a restart; this is largely fine.
If the peer's disconnected but the caller sends us valid sigs for the
channel open, we should go ahead and store them to disk before we reject
the call based on the fact that the peer is disconnected.
This way if the peer reconnects later, the channel open will succeed
Changelog-Changed: RPC: `openchannel_signed` will now remember the details of a signed PSBT even if the peer is disconnected.
When we reconnect, if we get a note from the peer that they dont know
about a pending inflight, we need to be able to clean it up so we can
restart/re-negotiate a new RBF etc.
This adds a cleanup method to remove any inflights for a channel without
a last_tx (commitment tx)
Here, we split up what was "commit_received" into two phases:
- commit-ready, where we're about to send our commitment tx to
peer
- commit-received, when we've gotten the commitment tx from our
peer
This lets us do the right thing (as far as the spec is concerned) with
returning the correct 'next_funding_txid' on reconnect (later commits).
From the spec:
Once peers are ready to exchange commitment signatures, they must remember
the details of the funding transaction to allow resuming the signatures
exchange if a disconnection happens.
Basically this means we add channels to the database before we've gotten
commitments for them; it's nice that there's now a state for commitments
recevied but we now save the channel prior to that.
This commit makes it possible to track the pre-commit-rcvd but not quite
open-init state.
"Patrick, I'm sorry I doubted you."
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Config: `large-channels` is now the default, wumbology for all.
Adding a fee offset as the channel opener reduces the likelihood of a
disconnect by the peer do to slight variation in feerate calculation
between nodes.
Changelog-Fixed: Some peer disconnects due to update_fee disagreements are avoided.
This table doesn't have `id`, except as the implicit one in Sqlite3,
so we need to add it for postgres.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
peer_htlcs has become a bit of a dumping ground: move listforwards
etc to its own file.
Also move `struct channel_info` from peer_htlcs.h to channel.h where
it more logically belongs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Looking through logs I was surprise to see:
```
lightningd-1 2023-10-26T03:42:36.824Z INFO lightningd: Sending 200000000msat over 1 hops to deliver 200000000msat
```
On a re-payment where we simply returned from sendpay immediately! Move that log to later.
We used to have "unsaved" payments: now we don't we can use
our normal "iterator" pattern rather than returning arrays.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It used to be used for both `sendpay` and `waitsendpay` but now it's
only for the latter, so the name is confusing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We didn't write to db immediately, but waited until it the actual HTLC got
added (or failed). That way we didn't have a separate transaction to
write the payment into the db, but the complexity is not worth it: it
makes the next refactors harder, since we can't use the normal
iterator patterns like we do with the rest of the db (as we have to add
the unstored ones).
We might as well also make sendpay return immediately: we used to return
once the HTLC had been confirmed sent, since we entered it in the db
at that point, but we can keep it simple now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Adding an index means:
1. Add the new subsystem, and new updated_index field to the db, and
create xxx_index_deleted/created/updated APIs.
2. Hook up these functions to the points they need to be called.
3. Add index, start and limit fields to the list command.
4. Add created_index and updated_index into the list command.
This does #1.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means refactoring out some of the generic anchor info, from the
per-commitment-tx info (we can have at least two, perhaps more with
splicing!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's going to want to remember these, in case it encounters peers'
commitment tx and needs to boost it with CPFP on the anchor.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Tihis commit is implementing a 2-phase commit between
the signer the node and the peer.
The main reason for this is that everybody must agree on the lock,
otherwise one of them will want N signatures (on the splice candidates),
and another will produce only 1 signature.
check_outpoint is the "prepare" for the signer, and lock_outpoint is the
"commit". if check_outpoint returns true, lock_outpoint must not fail.
Link: https://github.com/ElementsProject/lightning/issues/6722
Suggested-by: @devrandom
Co-Developed-by: Ken Sedgwick <ken@bonsai.com>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-Added: JSON-RPC: `recover` command to force (unused) lightningd node to restart with `--recover` flag.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This makes `check` much more thorough, and useful.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `check` now does much more checking on every command (not just basic parameter types).
Put an assertion inside db.c, and run every command we do (in testing) through
a `check` variant.
I inserted a deliberate bug (made addpsbtoutput call wallet_get_newindex()
before returning when running `check`, and indeed, backtrace as expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We often want to do more parameter checks after param(), so allow a
new param_check(), with the proviso that the caller needs to also return
command_check_done() after other checks if command_check_only(cmd) is true.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We had a complaint that you can't CPFP a mutual close, which you
should be able to do.
Fixes: #6692
Changelog-Fixed: wallet: close change outputs show up immediately in `listfunds` so you can CPFP.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
datastoreusage returns the total_bytes that are stored under a given
{Key} or from root. {Key} is the entry point from which we begin to
traverse the datastore.
Changelog-Added: JSON-RPC: `datastoreusage`: returns the total bytes that are stored under a given key.
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
```
Already in transaction from lightningd/plugin.c:727
```
There are two callers, and one didn't disable transactions, so do it in plugin_exclusive_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now we've asserted that channeld would tell lightningd the same thing it
would do anyway, we can simply have channeld say "enable=True|False" and
lightningd fill in the other fields.
This means there's a pile of things channeld doesn't need to know any more!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
channeld used to talk directly to gossipd, so it made sense for it to
tell gossipd directly when it wanted it to make a new channel_update.
When that changed with v0.11, we simply directed the message via
lightningd.
But much of the information is actually told to channeld by lightningd!
So I applied this assertion and ran the test suite, before the next patch makes it redundant.
We got one assertion: test_setchannel_zero deliberately drives the
advertized htlc_max over the real htlc max in test_setchannel_zero for
testing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We have to work quite hard to do this, since we don't want to call
finish if the broadcast has been freed in the meantime.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Previously, every broadcast was attached to a channel, but we can
make it explicit, so when the context is freed, the re-broadcast stops
(if rebroadcast is set).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If the context is freed, the callback isn't called. This doesn't matter
yet, since our callbacks tend to be such that the callback itself is
required to free things, but it's clearer this way and allows more
flexible usage in following patches.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We remove it from the pending_requests strmap before calling it,
so it doesn't get called again by destroy_plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We should really unify the cases of a local request, vs a forwarded
request, but for now, don't steal the request onto the plugin, and
if we return from the plugin and the request is gone, don't get upset.
This uncovered a case where we weren't inside a transaction, in
test_hook_crash, so fix that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now the internal code will generate a "PLUGIN_TERMINATED" response
when the plugin dies, we can handle it in plugin_hook.
But we can also simplify it by turning the snapshot of hooks into
a simple array: this means we are robust against any combination of plugins
exiting at any time.
Note: this reveals an issue with test_rpc_command_hook where we run
the request hook again (unexpectedly), so we disable that for the next
patch.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We had special code to fail a forwarded request, but not for an
internally-generated request. Instead, we should pretend the (dead)
plugin responded with a PLUGIN_TERMINATED error, and handle the
request through the normal paths.
This breaks the case where a plugin crashes (or stops itself) in a
hook, so we handle that next.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It was always a bit weird they weren't, and it seems a premature
optimization to make the callbacks to this themselves.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's per-plugin, so why is there a single map for all plugins? It
works because we always make unique ids, but it's weird.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When using DEBUG_SUBD with pytest:
```
lightningd: Unknown decode for --dev-debugger=<subprocess>
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
During the changeset calculation after the `openchannel2_sign`
hook.
So this commit patch the problem with the following change:
- Addressed an issue where `psbt_get_changeset` was modifying the original PSBT unnecessarily.
- This modification led to problems with a different hsmd, as referenced in [Issue #6672](https://github.com/ElementsProject/lightning/issues/6672).
- Noted a potential optimization where only a subpart of the PSBT
needs to be cloned, as the mutation is specific to inputs.
Link: https://github.com/ElementsProject/lightning/issues/6672
Reported-by: @devrandom
Suggested-by: Ken Sedgwick <ken@bonsai.com>
Co-Developed-by: Ken Sedgwick <ken@bonsai.com>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-Added: Plugins: plugins can now specify (unknown) even messages we should accept from peers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We do it here, but it's not necessary, and we also deprive them of the
chance to do so (since we kill them).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Previously, we would forward the message to a subd, but now we have
the case where the subd is gone, but we're still connected. If the
peer anything but a reestablish in that state, we drop the connection.
Instead, an error should always make us fail the channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We generalize the current df-only "aborted" flag (and invert it) to a
"disconnected" flag in the peer status message.
We convert it back to the aborted flag for now inside subd.c, but that's
next.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Move the "no lease, return" to the top, to avoid testing twice. Also,
we won't spam now for most channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
As a node matures and is no longer new, it can take some time
to determine which version of `cln` it's running.
To address this, we now display the version earlier. This ensures
that even in the event of a crash, we're aware of the running version.
(cln-meta-project-py3.11) ➜ lightning git:(macros/log-version) ✗ ./lightningd/lightningd
2023-10-12T19:21:00.899Z INFO lightningd: v23.08.1-209-gae94be4-modded
2023-10-12T19:21:00.994Z INFO lightningd: Creating configuration directory /home/vincent/.lightning/bitcoin
2023-10-12T19:21:01.235Z INFO lightningd: Creating database
2023-10-12T19:21:01.279Z UNUSUAL hsmd: HSM: created new hsm_secret file
Could not connect to bitcoind using bitcoin-cli. Is bitcoind running?
Make sure you have bitcoind running and that bitcoin-cli is able to connect to bitcoind.
You can verify that your Bitcoin Core installation is ready for use by running:
$ bitcoin-cli echo 'hello world'
2023-10-12T19:21:01.349Z **BROKEN** plugin-bcli: \nCould not connect to bitcoind using bitcoin-cli. Is bitcoind running?\n\nMake sure you have bitcoind running and that bitcoin-cli is able to connect to bitcoind.\n\nYou can verify that your Bitcoin Core installation is ready for use by running:\n\n $ bitcoin-cli echo 'hello world'\n
2023-10-12T19:21:01.349Z INFO plugin-bcli: Killing plugin: exited before we sent init
The Bitcoin backend died.
Link: https://github.com/ElementsProject/lightning/issues/6374
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
This makes it easier to use outside simple subds, and now lightningd can
simply dump to log rather than returning JSON.
JSON formatting was a lot of work, and we only did it for lightningd, not for
subdaemons. Easier to use the logs in all cases.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We didn't apply the inflight to the channel struct before asserting, so
we can break test_rbf_non_last_mined:
```
lightningd: lightningd/dual_open_control.c:981: dualopend_tell_depth: Assertion `bitcoin_txid_eq(&channel->funding.txid, txid)' failed.
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now we're not always using the same functions to watch during
dual-funding opening, we need to make sure we're watching the close
(in particular, df close before the opening is confirmed).
So, keep a pointer, and if it's not set in drop_to_chain, set it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We used the original channel funding output number. I'm not sure if this
was true in the previous code, or a regression I introduced, but it
caused occasonal failures in test_splice_gossip!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We use the *same* callback for the funding tx, as well as for inflight dual-funding txs, as well as inflight splice txs. This is deeply confusing!
Instead, use explicit cbs for splicing and df. Once they're locked in, use the normal callback.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We never do this, but we're about to (we always watch before
we broadcast a tx).
We use a `depth` member to avoid calling the callback multiple times
for the same event, but we initialize it to 0. This means if we
register a watch, and the first thing that happens is that it
reorganizes out, we *don't* make the callback.
Use an impossible value at initialization, instead.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The latter is used when we're put in the db, the former is the uncommitted state.
Currently dbid == 0 is used in addition to the state, which is unwieldy.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Experimental: JSON-RPC: added new dual-funding state `DUALOPEND_OPEN_COMMITTED`
We usually hand times by copy, not by pointer (and if we did, they should
be const!). I noticed this particularly for the state changed code, but
it goes down to to json_add_timeiso, so I fixed that too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We should use capability tests for states (can you add htlcs?) rather than vague
descriptions (are you closing?).
And as much as possible, use switch () statements to force us to think
about all the cases, especially when we add new states!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Currently it's half done in funding_depth_cb, and half in
channeld_tell_depth. It's very confusing as a result,
with splicing, dual-funding and zeroconf.
This does introduce a behaviour change: if a channel is NORMAL and
it gets reorganized, we force close (unless we were the one who funded
it, or it's zeroconf anyway). This is safer than continuing to use
the channel in this case!
Some tests are changed to zeroconf to make them work, but v2 doesn't
support zeroconf, so that's removed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a workaround, the real fix is to use a different
callback for inflight splice attempts, which comes later.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Not just if htlc addition is too slow, make this the default. dual-open's txabort
is excluded, however.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If you previously configured with `--enable-developer` we turn that into `--enable-debugbuild`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: build: `--enable-developer` arg to configure (and DEVELOPER variables): use `./configure --enable-debugbuild` and `developer` setting at runtime.
And require --developer to use them.
Also refuse redirection to deprecated APIs if deprecated APIs are disabled!
Changelog-Removed: `dev-sendcustommsg` (use `sendcustommsg`, which was added in v0.10.1)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Also requires us to expose memleak when !DEVELOPER, however we only
ever used the memleak tracking when the LIGHTNINGD_DEV_MEMLEAK
environment variable was set, so keep that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Currently it just defaults to the DEVELOPER compile option, but we'll
move over to this.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: `--developer` enables developer options and changes default to be "disable deprecated APIs".
We check for list_empty, so it's always actually set.
```
lightningd/peer_control.c: In function ‘drop_to_chain’:
lightningd/peer_control.c:353:17: error: ‘tx’ may be used uninitialized [-Werror=maybe-uninitialized]
353 | resolve_close_command(ld, channel, cooperative, tx);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lightningd/peer_control.c:341:36: note: ‘tx’ was declared here
341 | struct bitcoin_tx *tx;
| ^~
cc1: all warnings being treated as errors
make: *** [Makefile:298: lightningd/peer_control.o] Error 1
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Added a test for splicing out that exposed some behavior and code glitches that are addressed in this commit.
Added test for splice gossip.
Also added documentation for how to do a splice out.
ChangeLog-Fixed: Added docs, testing, and some fixes related to splicing out, insufficent balance handling, and restarting during a splice.
json_add_timeabs only printed in milliseconds and json_add_time outputs a string which is weird
Changelog-Changed: JSON-RPC time fields now have full nanosecond precision (i.e. 9 decimals not 3): `listfowards` `received_time` `resolved_time` `listpays`/`listsendpays` `created_at`.
Explicitly allow all-zero in the onion_hash: we didn't do anything except log if it was unexpected anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were allowed to, but the spec removed that. So we handle warnings
differently from errors now.
This also means the LND "internal error" workaround is done in
lightningd (we still disconnect, but we don't want to close channel).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: we no longer disconnect every time we receive a warning message.
This seems to be a cut & paste bug (mine, AFAICT!) from the command code:
```
rune = rune_derive_start(cmd, master_rune,
tal_fmt(tmpctx, "%"PRIu64,
rune_counter ? *rune_counter : 0));
```
In that case, rune_counter was a pointer, which could be NULL.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It always is for runes we create, but in theory you can take our secret key
and make our own runes with your own tools.
(We correctly refuse runes without uniqueids if they're *not* ours
anyway: uniqueid is only used for our own runes).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
nodeid is only useful when we know the peer we're talking to (e.g. commando).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
No-schema-diff-check: We're simply making optional, not deprecating!
During tests we can see that the subdaemon can be restarted unnecessarily if we're slow enough; we don't need to do so if it's still running.
Reported-by: Matt Morehouse <mattmorehouse@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We determine whether they are allowed or not based on the hook return
value of `mindepth`. To do so we need to pass that value down to
`openingd` and verify that the `channel_type` and our permissions
match up.
1. announce-addr-discovered-port takes a port option.
2. accept-htlc-tlv-types was deprecated in favor of multiple accept-htlc-tlv-type.
3. Document clnrest.py options.
4. Don't list --version twice in lightningd --help (initial_config_opts calls
opt_register_version() already).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: we clean up properly if a plugin fails to start, and we don't kill all processes if it's from `plugin startdir`.
Don’t send the funding spend to onchaind if we detect it in inflights (aka. a splice). While we already prevented onchaind_funding_spent from being called directly, the call to wallet_channeltxs_add meant onchaind_funding_spent would be called *anyway* on restart. This is now fixed.
Additionally there was a potential for a race problem depending on the firing order of the channel depth and and funding spent events.
Instead of requiring these events fire in a specific order, we make a special “memory only” inflight object to prevent the race regardless of firing order.
Changelog-Fixed: Splice: bugfix for restart related race condition interacting with adversarial close detection.
Indeed, we can fall through this if it's not a valid enum value.
gcc-12 (Ubuntu 12.2.0-17ubuntu1) 12.2.0
```
In file included from plugins/commando.c:10:
ccan/ccan/tal/str/str.h: In function ‘rune_altern_to_english’:
ccan/ccan/tal/str/str.h:43:9: error: ‘cond_str’ may be used uninitialized [-Werror=maybe-uninitialized]
43 | tal_fmt_(ctx, TAL_LABEL(char, "[]"), __VA_ARGS__)
| ^~~~~~~~
plugins/commando.c:97:21: note: ‘cond_str’ was declared here
97 | const char *cond_str;
| ^~~~~~~~
cc1: all warnings being treated as errors
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In spec commit 498f104fd399488c77f449d05cb21c0b604636a2 (August 2021),
Bastien Teinturier removed the requirement that the mutual close fee be
less than or equal the final commitment tx.
We adopted that change in v0.10.2, but we made sure to never offer a fee
under the final commitment tx's fee, so we didn't break older nodes.
However, the closing tx can actually be larger than the final commitment tx!
The final commit tx has a 22-byte P2WKH output and a 34-byte P2WSH output;
the closing can have two 34-byte outputs, making it 4*8 = 32 Sipa heavier.
Previously this would only happen if both sides asked for P2WSH outputs,
but now it happens with P2TR, which we now do.
The result is that we create a tx which is below the finally commitment
tx fee, and may be below minrelayfee (as it was in regtest).
So it's time to remove that backwards-compatibility hack.
Changelog-Fixed: Protocol: We may propose mutual close transaction which has a slightly higher fee than the final commitment tx (depending on the outputs, e.g. two taproot outputs).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #6545
Avoids a gratuitous "ctx" field, and the simplified declaration
is now understood by `make update-mocks`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was recommended by @t-bast: if the final spec commits to something
compatible, we can simply advertize and accept both features, but if it
does change in incompatible ways we won't cause problems for nodes
who implement the official spec.
(I split this, so first, we remove the OPT_SPLICE entirely, to make
sure we caught them all. --RR)
Suggested-by: @t-bast
Changelog-None
The nomenclature confusion mean that we were ANDING a capability
with a message number (29) which always returned non-zero. We really
do need a new capability which we can hand to channeld to make these
splice txs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I obviously like the word "capabilities" since I reused it to refer
to the HSM's overall features :(
Suggested-by: @ksedgwic
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Apparently MacOS doesn't always have fdatasync, so use fsync. Even more importantly
check whether it succeeds!
Fixes: #6516
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I noticed this while debugging an issue with ACINQ, that we got upset,
but didn't trigger a reconnect cycle.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: We now close connection with a peer if adding an HTLC times out (which may be a TCP connectivity issue).
In this case, the user's default was info, but they specifically asked for debug
from one plugin. Since there were no per-file filters, it set filtering to the
default level, info, and rejected it. Since it's been explicitly filtered in,
we need to pass it at this point.
Reported-by: @wtogami
Fixes: #6503
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was strongly recommended by Russell O'Connor: the "ms" implies that
it's a BIP-32 master secret, and this is CLN specific.
If we changed the hrp to "cln" it would be better, but apparently that
means we no longer fit in a "standard billfold metal wallet" (and
our code assumes a 2-byte prefix anyway).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
