Commit Graph

13902 Commits

Author SHA1 Message Date
Vincenzo Palazzo
a6e1f19e6d libhsmd: removing assertion on push_value
The assertion may not make sense if we change the
meaning of `push_value` to be relative, especially since
negative values appear as large positive.

Suggested-by: Ken Sedgwick <ken@bonsai.com>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-10-23 16:08:26 +10:30
Ken Sedgwick
4649bccbea splice: signer must be informed of splice params
The signer needs to know when the splice operation starts and the
splice parameters for each splice transaction candidate.

The channel establishment v2 (dual funding) code path already
notifies the signer via the hsmd API hsmd_ready_channel calls
However, the splicing code path does not.

Link: https://github.com/ElementsProject/lightning/issues/6723
Suggested-by: @devrandom
Co-Developed-by: @devrandom
Co-Developed-by: Ken Sedgwick <ken@bonsai.com>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-10-23 16:08:26 +10:30
Vincenzo Palazzo
f4bf89b315 hsmd: name the hsmd_ready_channel to hsmd_setup_channel
Originally VLS used hsmd_ready_channel as an early call during channel
setup, but later the BOLT-2 spec changed the name of funding_locked to channel_ready.

This is very confusing because the hsmd_ready_channel is not directly
related to the new channel_ready.

This commit is renaming the hsmd_ready_channel to hsmd_setup_channel.

Link: https://github.com/ElementsProject/lightning/issues/6717
Suggested-by: Ken Sedgwick
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-10-23 16:04:30 +10:30
ShahanaFarooqui
86f13d4d18 wallet: db_get_runes will only get rune from DB select
wallet_get_runes is used to return runes with last_used_nsec
2023-10-23 15:53:30 +10:30
ShahanaFarooqui
d9e3d3e6d1 runes: bug fix for db failure to upgrade due to runes table alteration #6770
While rebasing the PR for per rune restriction, I unintentionally merged the
`{SQL("ALTER TABLE runes ADD last_used_nsec BIGINT DEFAULT NULL"), NULL}` database
alteration command ahead of {NULL, migrate_runes_idfix} (commit eacf0b502c (diff-1abcdf1b9d822b30079d6450b790274bdfb7c7fa04baa43ad2d9bd449865d4c9R978)).

`migrate_runes_idfix` was the 234th change (deployed with version 23.08.1)
and adding the `last_used_nsec` column should have been the next
(235th, added in current release) change. Due to this incorrect ordering,
nodes updating from version 23.08.1 to the master branch will not add the
`last_used_nsec` column as they should, and instead execute `migrate_runes_idfix`
again, leading to the error in issue #6770.

After the reordering, db_get_runes method also has to be fixed for only
selecting rune NOT last_used_nsec. Because this column was added after
`migrate_runes_idfix` calls it. I am tempted to change the method name from
`db_get_runes` to `db_migrate_runes` for more clarity on its functionality though.

Changelog-None.
2023-10-23 15:53:30 +10:30
Rusty Russell
1d61edfe0c connectd: use shutdown() not close() on TCP sockets for dev-disconnect.
close() is allowed to lose data, and I saw this in CI:

```
2023-10-22T05:12:36.6576005Z ____________________________ test_permfail_htlc_out ____________________________
2023-10-22T05:12:36.6608511Z [gw2] linux -- Python 3.8.18 /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/bin/python
2023-10-22T05:12:36.6611663Z 
2023-10-22T05:12:36.6614768Z node_factory = <pyln.testing.utils.NodeFactory object at 0x7f381039a5e0>
2023-10-22T05:12:36.6623694Z bitcoind = <pyln.testing.utils.BitcoinD object at 0x7f38103c0400>
2023-10-22T05:12:36.6627092Z executor = <concurrent.futures.thread.ThreadPoolExecutor object at 0x7f38103c0ee0>
2023-10-22T05:12:36.6627701Z 
2023-10-22T05:12:36.6628051Z     def test_permfail_htlc_out(node_factory, bitcoind, executor):
2023-10-22T05:12:36.6631192Z         # Test case where we fail with unsettled outgoing HTLC.
2023-10-22T05:12:36.6634154Z         disconnects = ['+WIRE_REVOKE_AND_ACK', 'permfail']
2023-10-22T05:12:36.6635106Z         l1 = node_factory.get_node(options={'dev-no-reconnect': None})
2023-10-22T05:12:36.6637321Z         # Feerates identical so we don't get gratuitous commit to update them
2023-10-22T05:12:36.6642691Z         l2 = node_factory.get_node(disconnect=disconnects,
2023-10-22T05:12:36.6644734Z                                    feerates=(7500, 7500, 7500, 7500))
2023-10-22T05:12:36.6647205Z     
2023-10-22T05:12:36.6649671Z         l1.rpc.connect(l2.info['id'], 'localhost', l2.port)
2023-10-22T05:12:36.6650460Z         l2.daemon.wait_for_log('Handed peer, entering loop')
2023-10-22T05:12:36.6654865Z         l2.fundchannel(l1, 10**6)
2023-10-22T05:12:36.6655305Z     
2023-10-22T05:12:36.6657810Z         # This will fail at l2's end.
2023-10-22T05:12:36.6660554Z         t = executor.submit(l2.pay, l1, 200000000)
2023-10-22T05:12:36.6662947Z     
2023-10-22T05:12:36.6665147Z         l2.daemon.wait_for_log('dev_disconnect permfail')
2023-10-22T05:12:36.6668530Z         l2.wait_for_channel_onchain(l1.info['id'])
2023-10-22T05:12:36.6671588Z         bitcoind.generate_block(1)
2023-10-22T05:12:36.6674510Z >       l1.daemon.wait_for_log('Their unilateral tx, old commit point')
2023-10-22T05:12:36.6675001Z 
2023-10-22T05:12:36.6675212Z tests/test_closing.py:3027: 
...
2023-10-22T05:12:36.8784390Z lightningd-2 2023-10-22T04:41:04.448Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-connectd: dev_disconnect: +WIRE_REVOKE_AND_ACK (WIRE_REVOKE_AND_ACK)
2023-10-22T05:12:36.8786260Z lightningd-2 2023-10-22T04:41:04.452Z INFO    0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-channeld-chan#1: Peer connection lost
2023-10-22T05:12:36.8788076Z lightningd-2 2023-10-22T04:41:04.453Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-channeld-chan#1: Status closed, but not exited. Killing
2023-10-22T05:12:36.8789915Z lightningd-1 2023-10-22T04:41:04.454Z INFO    022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-channeld-chan#1: Peer connection lost
```

Note that l1 doesn't receive WIRE_REVOKE_AND_ACK!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell
93189724dd lightningd: let channeld/dual_openingd send error to peer.
We do it here, but it's not necessary, and we also deprive them of the
chance to do so (since we kill them).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell
6fdaec313d pytest: be sure that we receive error on datalose failure.
It was intermittant before: I added a sleep(1) in the code before
sending the error (temporarily) to make it always triggers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell
f44c1971a5 channeld: don't hang up immediately after sending bad reestablish warning.
This gives the peer a chance to send an error, which will make us drop
to chain.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5818
2023-10-23 15:48:50 +10:30
Rusty Russell
cf2ebed567 lightningd: close channel ourselves, if we receive an error.
Previously, we would forward the message to a subd, but now we have
the case where the subd is gone, but we're still connected.  If the
peer anything but a reestablish in that state, we drop the connection.

Instead, an error should always make us fail the channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell
6a67615274 pyln-testing: make disconnects continue after restart.
We truncate the file on stop(), but don't re-created it on start().

We didn't notice it before, but the net

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell
b968b18534 common: move is_peer_error/is_peer_warning from read_peer_msg.c to wire_error.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell
285595ada7 pytest: revert warning delivery failure workaround
Reverts 6203d250bf "pytest: fix flake in upfront warning." now we've actually fixed the cause.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell
9abf9c2924 lightningd: pass disconnect flag to subd's errcb.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell
443082ba98 common: add peer_failed_warn_nodisconnect routine for non-disconnecting warnings
We generalize the current df-only "aborted" flag (and invert it) to a
"disconnected" flag in the peer status message.

We convert it back to the aborted flag for now inside subd.c, but that's
next.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell
08f0a54fdc connectd: don't disconnect automatically on sending warning or error.
This changes some tests which expected us to disconnect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell
798cf27cb4 connectd: give subds a chance to drain when lightningd says to disconnect.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell
a32b3b68e2 lightningd: stop all subds when we want to disconnect.
This prepares us for connectd disconnecting more gently: it can
wait for these to all disconnect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell
64495cf166 tools: make sure we don't use optimization when building mocks.
Don't want it to optimize something out!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell
2ea3c6155f Makefile: fix update-mocks dependencies.
It definitely needs generated source files.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Dusty Daemon
e1ac2410b0 connectd: Ignore sodium fd for Mac tests
On Mac most tests report BROKEN because sodium creating an untracked fd pointing to /dev/random. dev_report_fd’s finds it at tear down and reports a BROKEN message.

We allow a single “char special” fd without reporting it as broken improving QOL for Mac developers.

While we’re here we added the fd mode to the log to help with future rogue fd issues.

ChangeLog-None
2023-10-19 14:31:25 +10:30
Matt Morehouse
87f4907bb4 bolt11: don't abort on invalid pubkey
Rather than crashing the entire node on invalid pubkey, check the
validity of the pubkey in decode_n, and return an error if invalid.

Detected by libFuzzer:
==265599== ERROR: libFuzzer: deadly signal
    #7 abort
    #8 bolt11_decode common/bolt11.c:999:4
2023-10-18 09:53:22 +10:30
Matt Morehouse
c1f20687a6 bolt11: validate recovery ID
Invalid recovery IDs cause
secp256k1_ecdsa_recoverable_signature_parse_compact to abort, which
crashes the entire node. We should return an error instead.

Detected by libFuzzer:
[libsecp256k1] illegal argument: recid >= 0 && recid <= 3
2023-10-18 09:53:22 +10:30
Matt Morehouse
4b29502098 fuzz: check invoice signatures
There may be bugs in signature validation, so we should fuzz that too.
2023-10-18 09:53:22 +10:30
Matt Morehouse
ee8cf69f28 bolt11: return error on unexpected lightning prefix
Remove the assertion so that an error is returned for invalid bech32.
An error is preferable to crashing the entire node if there's an extra
"lightning:" prefix:

  $ lightning-cli pay "lightning:lightning:"

Node log:
  pay: common/bolt11.c:718: bolt11_decode_nosig: Assertion `!has_lightning_prefix(str)' failed.
  pay: FATAL SIGNAL 6
  ...
  INFO    plugin-pay: Killing plugin: exited during normal operation
  **BROKEN** plugin-pay: Plugin marked as important, shutting down lightningd
2023-10-18 09:53:22 +10:30
Matt Morehouse
74b8fff3d4 github: re-enable ASan and UBSan for check-fuzz
ASan and UBSan were disabled by
364de00947, which has already prevented
multiple bugs from being discovered prior to merging.
2023-10-18 09:53:22 +10:30
Matt Morehouse
ee501b035b bolt11: avoid reading uninitialized memory
If both databits and *data_len are 0, pull_uint return uninitialized
stack memory in *val.

Detected by valgrind and UBSan.

valgrind:
==173904== Use of uninitialised value of size 8
==173904==    __sanitizer_cov_trace_cmp8
==173904==    decode_c (bolt11.c:292)
==173904==    bolt11_decode_nosig (bolt11.c:877)

UBSan:
common/bolt11.c:79:29: runtime error: shift exponent 64 is too large for 64-bit type 'uint64_t' (aka 'unsigned long')

Corpus input e6f7b9744a7d79b2aa4f7c477707bdd3483f40fa triggers the UBSan
report, but we didn't previously realize this because UBSan has been
disabled in the CI run. We rename the input to indicate its usefulness
as a permanent regression test.
2023-10-18 09:53:22 +10:30
Matt Morehouse
eeec529031 bolt11: check return value of pull_all
Otherwise, if pull_all fails, we attempt to create a script from NULL,
causing a UBSan report:

bitcoin/script.c:29:28: runtime error: null pointer passed as argument 2, which is declared to never be null

Corpus input bf703c2c20c0818af70a8c4caad6e6fd8cfd1ac6 triggers the UBSan
report, but we didn't previously realize this because UBSan has been
disabled in the CI run. We rename the input to indicate its usefulness
as a permanent regression test.
2023-10-18 09:53:22 +10:30
Rusty Russell
19c581c33f lightningd: simplify lease update blockheight code.
Move the "no lease, return" to the top, to avoid testing twice.  Also,
we won't spam now for most channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-17 14:59:54 +10:30
Rusty Russell
91699d8483 lightningd: fix crash caused by sending channeld message to closingd
```
Sending closingd an invalid message 03f40000006d
...
0x564b55bb7378 fatal
	lightningd/log.c:1035
0x564b55bf7841 subd_send_msg
	lightningd/subd.c:841
0x564b55b87d08 try_update_blockheight
	lightningd/channel_control.c:143
0x564b55b8c497 channel_notify_new_block
	lightningd/channel_control.c:1687
0x564b55bb26bb notify_new_block
	lightningd/lightningd.c:743
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-17 14:59:54 +10:30
Rusty Russell
03e51ae879 Revert "runes: fix checkrune when method parameter is the empty string."
This reverts commit ae94be4ce2.
2023-10-17 14:58:40 +10:30
Rusty Russell
c0782ee841 clnrest: correctly omit method to checkrune call if none is supplied.
Specifying the empty string is very different from not specifying it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-17 14:58:40 +10:30
Rusty Russell
cdd58e2fe3 pytest: correctly test missing method for runes.
This makes it clear exactly what the expectation is/

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-17 14:58:40 +10:30
Michael Schmoock
8ac9c73ea8 pytests: make pay helper able to route (optionaly)
This makes the pay helper function being able to route
a payment using the optional `route` paramter that defaults to `False`.

I added this, as some plugins maintained their own version of `pay` that
needed routed payment helper.

Changelog-None
2023-10-17 14:53:53 +10:30
Dusty Daemon
e938999027 channeld: comment typo fix
ChangeLog-None
2023-10-16 00:55:47 +02:00
Vincenzo Palazzo
e936eea9cb build: fix compilation issue on OpenBSD
During our development, we modified the way
we report backtraces.

On a minimal configuration in OpenBSD, it seems that we
no longer compile from commit a9f26b7d07 because
our conditional code is buggy.

With the following compiler

vultr# cc -v
OpenBSD clang version 13.0.0
Target: amd64-unknown-openbsd7.3
Thread model: posix
InstalledDir: /usr/bin

We have the following error

cc common/channel_id.c
cc common/daemon.c
common/daemon.c:218:2: error: implicit declaration of function 'add_steal_notifiers' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        add_steal_notifiers(NULL);
        ^
1 error generated.
gmake: *** [Makefile:298: common/daemon.o] Error 1

Reported-by: @grubles
Fixes a9f26b7d07
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-10-15 14:01:45 +02:00
Tony Aldon
8f6a02c182 doc: fix display of lists and code blocks in manpages.
The conversion from markdown to manpages does not display lists and
code blocks correctly if they are not preceded by an empty line.

Changelog-None
2023-10-14 17:55:51 +02:00
Vincenzo Palazzo
9b69c56cfb lightningd: print the version of cln early
As a node matures and is no longer new, it can take some time
to determine which version of `cln` it's running.

To address this, we now display the version earlier. This ensures
that even in the event of a crash, we're aware of the running version.

(cln-meta-project-py3.11) ➜  lightning git:(macros/log-version) ✗ ./lightningd/lightningd
2023-10-12T19:21:00.899Z INFO    lightningd: v23.08.1-209-gae94be4-modded
2023-10-12T19:21:00.994Z INFO    lightningd: Creating configuration directory /home/vincent/.lightning/bitcoin
2023-10-12T19:21:01.235Z INFO    lightningd: Creating database
2023-10-12T19:21:01.279Z UNUSUAL hsmd: HSM: created new hsm_secret file

Could not connect to bitcoind using bitcoin-cli. Is bitcoind running?

Make sure you have bitcoind running and that bitcoin-cli is able to connect to bitcoind.

You can verify that your Bitcoin Core installation is ready for use by running:

    $ bitcoin-cli echo 'hello world'
2023-10-12T19:21:01.349Z **BROKEN** plugin-bcli: \nCould not connect to bitcoind using bitcoin-cli. Is bitcoind running?\n\nMake sure you have bitcoind running and that bitcoin-cli is able to connect to bitcoind.\n\nYou can verify that your Bitcoin Core installation is ready for use by running:\n\n    $ bitcoin-cli echo 'hello world'\n
2023-10-12T19:21:01.349Z INFO    plugin-bcli: Killing plugin: exited before we sent init
The Bitcoin backend died.

Link: https://github.com/ElementsProject/lightning/issues/6374
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-10-14 17:52:15 +02:00
Tony Aldon
7bea339f2f cln-grpc: do not generate cerficates when no autostart.
Changelog-None
2023-10-13 10:25:36 +02:00
Tony Aldon
340a7316d6 pytest: test that cln-grpc certificates are not generated when cln-grpc is not started.
We update `test_grpc_no_auto_start` test to check that we do not
generate certificates when the cln-grpc plugin is not started.

This fails currently, so next commit fix it up.
2023-10-13 10:25:36 +02:00
Tony Aldon
ae94be4ce2 runes: fix checkrune when method parameter is the empty string.
This fix #6725.

Changelog-Fixed: fix `checkrune` when `method` parameter is the empty string.
2023-10-12 20:42:43 +02:00
Tony Aldon
ebee48ca8c pytest: test checkrune when method parameter is the empty string.
Add test `test_rune_method_not_equal_and_method_empty` that reproduces
issue #6725.

This fails currently, so next commit fix it up.

Error:

```
>       with pytest.raises(RpcError, match='Not permitted: method not present'):
E       Failed: DID NOT RAISE <class 'pyln.client.lightning.RpcError'>

tests/test_runes.py:605: Failed
```
2023-10-12 20:42:43 +02:00
dergoegge
02717c73dc fuzz: bolt11 inputs 2023-10-12 19:28:28 +02:00
dergoegge
0fc9fe1a81 fuzz: test bolt11 decoding 2023-10-12 19:28:28 +02:00
Tony Aldon
1a46b37609 contrib: add developer option to config file in startup_regtest.sh.
Without `developer` option, after compiling CLN and sourcing
`contrib/startup_regtest.sh`, if we try to start 2 nodes on regtest
running the command `start_ln` we get the following error:

```
lightningd: Config file /tmp/l1-regtest/config line 6: dev-fast-gossip: requires --developer
...
lightningd: Config file /tmp/l2-regtest/config line 6: dev-fast-gossip: requires --developer
```

Changelog-None
2023-10-10 23:04:31 +10:30
Rusty Russell
176a58f9e0 pytest: wean many tests off the assumption that listchannels shows private channels.
We will be changing this, or at least deprecating it, so get our
tests ready.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-04 08:02:33 +10:30
Rusty Russell
7bb6609f9a pytest: remove old test_tlv_or_legacy
We no longer support legacy at all, so this test doesn't really
test anything.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-04 08:02:33 +10:30
Rusty Russell
7d3f13ec0d renepay: work around change in fundchannel in tests.
The tests will wait until it's locally enabled, but it might not have
the update in the gossip store.  So have renepay enhance its local
view even if it already knows about the channel (this is correct
anyway, it just isn't very important usually).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-04 08:02:33 +10:30
Rusty Russell
bb60c77859 plugins/topology: convert listpeerchannels into local overlay.
This prepares us for when the gossmap doesn't contain private channel info.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-04 08:02:33 +10:30
Rusty Russell
8c0932e46a plugins/topology: split getroute logic.
Mechanical change to bundle into struct getroute_info for next
patch which uses callback.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-04 08:02:33 +10:30