Commit graph

13102 commits

Author SHA1 Message Date
Rusty Russell
7894d7136f pytest: adapt all the anchor-iff-EXPERIMENTAL tests to --experimental-anchors.
We use parameterization here.  The old `anchor_expected()` was for
non-zero-fee anchors, and have bitrotted so there are some other
changes as well.

Unfortunately, all the anchor accounting seems to be broken, but I
cannot understand these tests at all.  I had to simply disable them
for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
f64188f925 pytest: make dual-funding tests using option_anchors_zero_fee_htlc_tx.
They used to force option_anchor_outputs, so switch them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
825173f1b4 pytest: test CPFP using anchors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
17821da80b lightningd: turn on experimental support for zero-fee htlc anchors.
We disabled experimental support for opening non-zero-fee anchor
channels (though old nodes may still have such channels if they turned
that on!).

So we simply call this `experimental-anchors`, since this is the variant
which we expect to be used widely.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: protocol: added support for zero-fee-htlc anchors (`option_anchors_zero_fee_htlc_tx`), using `--experimental-anchors`.
2023-06-29 11:28:47 -04:00
Rusty Russell
af6d7c0779 global: thread zero fee option everywhere.
In most cases, it's the same as option_anchor_outputs, but for
fees it's different.  This transformation is the simplest:
pass it as a pair, and test it explicitly.

In future we could rationalize some paths, but this was nice
and mechanical.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
e1a9e25412 common/channel_type: add zero-fee-anchor type.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
73691100bf onchaind: don't grind for htlc fees if option_anchors_zero_fee_htlc_tx set.
The answer, it's right in the name of the option!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
a2039472c1 onchaind: use our own inputs to add to HTLC txs (iff using anchors).
This lets us RBF htlc txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
662b2687e2 hsmd: routine to sign HTLC tx merged with our own tx.
Since HTLC txs when using anchors are
SIGHASH_SINGLE|SIGHASH_ANYONECANPAY, we can attach other inputs to
give it a higher feerate.  But we need the HSMd to actually sign the
combo.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
a9d8f84c7e channeld: for anchors, use minrelayfee as minimum commit tx fee to allow.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
6087decec3 dualopend: use a lower feerate for first commitment tx for anchors.
We need to know both, because in theory we could negotiate a
non-anchor channel even if they support it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
96e198f751 openingd: use a lower feerate for first commitment tx for anchors.
We need to know both, because in theory we could negotiate a
non-anchor channel even if they support it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
e45bf14300 lightningd: use lowball feerate for commit_tx on anchor channels.
Since we can CPFP, we don't have to track the feerate as closely.  But
it still needs to get in the mempool, so we use 10 sat/byte, or the
100 block estimate if that is higher.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `feerates` has new fields `unilateral_anchor_close` to show the feerate used for anchor channels (currently experimental), and `unilateral_close_nonanchor_satoshis`.
Changelog-Changed: JSON-RPC: `feerates` `unilateral_close_satoshis` now assumes anchor channels if enabled (currently experimental).
2023-06-29 11:28:47 -04:00
Rusty Russell
dd76d60b0d dual-funding: don't override default if multifundchannel doesn't set feerate.
mfc->feerate_str is *never* NULL, since we set it in getfeerate; this is
confusing, as many places check for NULL.

Indeed, the logic in perform_fundpsbt() was *wrong* in this case: it used
`normal` (if it was NULL, which it never was) instead of `opening` to fundpsbt.

And the correct thing is for multifundchannel to not use a string here at
all, but to use the exact feerate it is counting on (even the same
string may have different values now if a block has come in).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
5332a8a67a lightningd: code to spend anchor outputs if we want to boost commitment tx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
e036080fc5 common: simple helpers to test for either anchor feature.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
0587cf324e lightningd/chaintopology: change form of finished callback.
We don't actually use it anywhere, but we actually want to now for
CPFP.  So give it more parameters and make it return bool so it can
be set without necessarily suppressing rexmit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
313354329d bitcoin/psbt: handle anchor spends.
Turns out it's a single sig, identical to the already-handled
case where we spend a to_remote output.

We also close a temporary memleak: stack was unused, but
tallocated off the psbt, so it lives as long as the PSBT.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
c2dffcab0f hsmd: command to sign anchor spends.
This is kind of a withdrawl to ourselves, except we also spend a
channel to-local anchor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
d895ef4fee pytest: test for commitment feerate.
Interesting we didn't actually test that the feerate we use is
actually delivered.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
e47e51edf2 lightningd: don't make htlc_timeout_satoshis/htlc_success_satoshis zero if we support anchors.
It depends on whether we negotiated anchors: just document that this
field doesn't apply for anchors (it becomes zero by the end of this
patch series, which is weird).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
47fd31e8b4 lightningd: don't spam with RBF messages if fee hasn't changed.
Reported-by: @19710405 on GitHub
Fixes: #6283
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
cba310b983 bitcoin: set PSBT amount inside tx for bitcoin_tx_output_set_amount.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
a5232659bf wallet: fix psbt_using_utxos when used with base PSBT.
We were setting the wrong input number: don't assume it's the
same as the UTXO number, but simply the last-appended input.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
3b1652842e msggen: add new version string.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
7f265300c7 lightningd: ignore any new options we add in deprecated output.
It's weird to add new options into the deprecated section.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
5c6e706026 lightningd: initialize channel_type field on dualopend channel creation.
Otherwise a badly-timed listpeerchannels will crash.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
336dcef541 wallet: don't try to set a timer past 2038 on 32-bit platforms.
It'll wrap, probably be in the past, and infinite loop.  This was caused by an invoice
with expiry set at 2076.  This wrap caused us to think the expiry has already
passed, and keep looping!

Reported-by: @telelvis
Fixes: #6339
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: lightnind: don't infinite loop on 32 bit platforms if only invoices are expiring after 2038.
2023-06-29 06:54:58 -04:00
Bitkarrot
910630c497
add m1 native lib paths to Makefile
add m1 native lib paths to Makefile (also need PKG_CONFIG_PATH)

[ Squashed into a single commit --RR ]
2023-06-26 08:25:21 +09:30
avatar4d
856808aa2d Correct INSTALL.md requirements for OpenBSD
GNU xgettext is required to compile. It is found in the gettext-tools package in OpenBSD. Tested on OpenBSD 7.3
2023-06-26 08:18:28 +09:30
Christian Decker
6507d34b50 cln-grpc: Add some missing states to HtlcState 2023-06-23 14:38:23 +09:30
Rusty Russell
b5845afd43 pytest: fix another flake in test_restorefrompeer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-23 14:36:23 +09:30
Rusty Russell
71adeccfbf pytest: fix reconnect flake in test_plugin_connected_hook_chaining
```
         l1.rpc.reject(l3.info['id'])
     
         l2.connect(l1)
         l1.daemon.wait_for_logs([
             f"peer_connected_logger_a {l2id}",
             f"{l2id} is allowed",
             f"peer_connected_logger_b {l2id}"
         ])
         assert len(l1.rpc.listpeers(l2id)['peers']) == 1
     
 >       l3.connect(l1)
 tests/test_plugin.py:468: 
... 
 >           raise RpcError(method, payload, resp['error'])
 E           pyln.client.lightning.RpcError: RPC call failed: method: connect, payload: {'id': '0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518', 'host': '127.0.0.1', 'port': 42391}, error: {'code': 402, 'message': 'disconnected during connection'}
 
 contrib/pyln-client/pyln/client/lightning.py:422: RpcError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-23 14:36:23 +09:30
Rusty Russell
ffb0f03e1a lightningd: fix crash on shutdown while if channel being cancelled at same time.
```
 DEBUG   lightningd: Command returned result after jcon close
 DEBUG   connectd: Shutting down
 DEBUG   gossipd: Shutting down
 DEBUG   hsmd: Shutting down
 **BROKEN** lightningd: FATAL SIGNAL 6 (version b29955a-modded)
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/common/daemon.c:38 (send_backtrace) 0x55fea4a13a08
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/common/daemon.c:75 (crashdump) 0x55fea4a140a1
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f9ea680651f
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f9ea685aa7c
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f9ea6806475
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f9ea67ec7f2
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:95 (call_error) 0x55fea4bbef68
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:169 (check_bounds) 0x55fea4bbfa68
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:180 (to_tal_hdr) 0x55fea4bbdefe
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:193 (to_tal_hdr_or_null) 0x55fea4bbd9cf
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:461 (tal_alloc_) 0x55fea4bbd893
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:506 (tal_alloc_arr_) 0x55fea4bbdce6
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/str/str.c:91 (tal_vfmt_) 0x55fea4bbc95e
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/lightningd/jsonrpc.c:502 (command_fail) 0x55fea499c427
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/lightningd/channel.c:81 (destroy_channel) 0x55fea4976a31
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:246 (notify) 0x55fea4bbdc18
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:437 (del_tree) 0x55fea4bbe028
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:521 (tal_free) 0x55fea4bbde84
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/lightningd/lightningd.c:577 (free_all_channels) 0x55fea49a2660
 **BROKEN** lightningd: backtrace: /home/runner/work/lightning/lightning/lightningd/lightningd.c:1280 (main) 0x55fea49a1530
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f9ea67edd8f
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f9ea67ede3f
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x55fea496ef04
 **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-23 14:36:23 +09:30
Rusty Russell
f0a9f1100a poetry, CI: insist in protobuf v21.12 in both Python and CI.
And re-ran poetry update which updated the lock file ofc.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-23 13:58:31 +09:30
Rusty Russell
69fc9c28bc Makefile: correctly erase generated contrib/pyln-testing/pyln/testing/ in distclean, and rebuild by default.
And add contrib/pyln-testing/pyln/testing/grpc2py.py since we didn't previously know
how to build it!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-23 13:58:31 +09:30
Matt Whitlock
8d737cc4bf Makefile: use grouped targets for recipes with multiple fixed outputs
See the section headed "Rules with Grouped Targets" on the Texinfo page
`(make)Multiple Targets`.

Without this fix, Make does not know that these recipes unconditionally
make *all* of their named targets regardless of which target triggers
their execution, and Make will blissfully execute multiple instances of
any such recipe in parallel, not only wasting CPU cycles but potentially
producing incorrect results if the recipe is not atomic in its effects
on the file system.  With this fix, Make understands that it need only
execute such a recipe once to make all of its targets.

In pursuit of the above, move and combine two redundant msggen recipes
into the top-level Makefile, and populate its grouped targets from the
subordinate Makefiles.

Changelog-None
2023-06-23 13:58:31 +09:30
Vincenzo Palazzo
fdb676491f pyln-proto: expose the tlv types
Exposing the tlv types to allow public access to it.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-06-23 13:32:25 +09:30
Matt Morehouse
8991f27497 fuzz: initial fuzz-cryptomsg seeds
Corpus generated by 300+ CPU hours of fuzzing and then minimized.
2023-06-23 13:12:42 +09:30
Matt Morehouse
1d94d2cd5d fuzz: test message encryption and decryption
Add a fuzz test for BOLT 8 message encryption and decryption. The fuzz
test is based on the unit test at common/test/run-cryptomsg.c and uses a
static initial state with fuzzer-generated messages to encrypt or
decrypt.
2023-06-23 13:12:42 +09:30
Matt Morehouse
fca50eaebc common: make hdr parameter const
cryptomsg_decrypt_header doesn't modify hdr, so we should make it const.
2023-06-23 13:12:42 +09:30
Matt Morehouse
8d9f033954 fuzz: initial handshake target seeds
Corpora generated for fuzz-connectd-handshake-act{1,2,3} from 300+ CPU
hours of fuzzing and then minimized.
2023-06-22 13:40:45 +09:30
Matt Morehouse
0e2173f1fd fuzz: add handshake act 3 target
The fuzz target uses static keys with a fuzzer-generated Act 3 packet.
2023-06-22 13:40:45 +09:30
Matt Morehouse
a203db00ca fuzz: add handshake act 2 target
The fuzz target uses static keys with a fuzzer-generated Act 2 packet.
2023-06-22 13:40:45 +09:30
Matt Morehouse
4bb216a611 fuzz: add handshake act 1 target
The fuzz target uses static keys with a fuzzer-generated Act 1 packet.
2023-06-22 13:40:45 +09:30
Matt Morehouse
2b629b2509 fuzz: add connectd_handshake.h
This header will be used by multiple fuzz targets to fuzz Acts 1, 2, and
3 of the BOLT 8 handshake.

We could make this header into a full library, but considering its
narrow use let's try not to over-engineer things.
2023-06-22 13:40:45 +09:30
Rusty Russell
55d6eb6762 CI: run 2, not 3 UBSAN/ASAN pytests at once.
It seems to reliably be getting a SIGTERM after 17-18 minutes:

```
[gw1] [ 91%] PASSED tests/test_plugin.py::test_forward_event_notification
Error: Process completed with exit code 143.
```

Perhaps this is out of mem?  So, try -n2.  We also make the configure
line explicit, rather than relying on environment vars (we should probably
do this for the other cases, too)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-22 13:23:21 +09:30
Rusty Russell
ca9a6b15b5 pay: don't require description for hashdesc invoices (i.e. undeprecate).
Since we didn't hash the descriptions properly (see previous commit), we
cannot immediately deprecate omitting the descriptions (since you'd
have to omit them for backwards compat!).

And move the "must have description or hash" test into bolt11.c core.
Changelog-Deprecated: `pay` has *undeprecated* paying a description-hash invoice without providing the description.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-22 13:23:21 +09:30
Rusty Russell
92ff0fd8c0 pay, decodepay: handle descriptions with " inside them where we use hashed descriptions.
This means we need to push off requring this for another full deprecation cycle!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `pay` and `decodepay` with description now correctly handle JSON escapes (e.g " inside description)
2023-06-22 13:23:21 +09:30
Bryan Ramos
7a57f7024c Added sample config file 2023-06-21 14:57:34 +09:30