Commit graph

13154 commits

Author SHA1 Message Date
Rusty Russell
0d1c46a5e5 Makefile: rebuild ccan when ccan_compat.h changes.
As it did recently:

```
/usr/bin/ld: bitcoin/pubkey.o: in function `pubkey_to_hash160':
/home/rusty/devel/cvs/lightning/bitcoin/pubkey.c:101: undefined reference to `ccan_ripemd160'
/usr/bin/ld: bitcoin/script.o: in function `hash160':
/home/rusty/devel/cvs/lightning/bitcoin/script.c:22: undefined reference to `ccan_ripemd160'
/usr/bin/ld: bitcoin/script.o: in function `bitcoin_wscript_htlc_offer':
/home/rusty/devel/cvs/lightning/bitcoin/script.c:662: undefined reference to `ccan_ripemd160'
/usr/bin/ld: bitcoin/script.o: in function `bitcoin_wscript_htlc_receive':
/home/rusty/devel/cvs/lightning/bitcoin/script.c:780: undefined reference to `ccan_ripemd160'
collect2: error: ld returned 1 exit status
make: *** [Makefile:660: devtools/create-gossipstore] Error 1
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-10 13:37:22 +02:00
Rusty Russell
e289695012 lightningd: fix rebase clash.
`deprecated_apis` is now inside `ld`.

```
lightningd/notification.c: In function ‘connect_notification_serialize’:
lightningd/notification.c:60:13: error: ‘deprecated_apis’ undeclared (first use in this function)
   60 |         if (deprecated_apis)
      |             ^~~~~~~~~~~~~~~
lightningd/notification.c:60:13: note: each undeclared identifier is reported only once for each function it appears in
lightningd/notification.c: In function ‘disconnect_notification_serialize’:
lightningd/notification.c:97:13: error: ‘deprecated_apis’ undeclared (first use in this function)
   97 |         if (deprecated_apis)
      |             ^~~~~~~~~~~~~~~
lightningd/notification.c: In function ‘block_added_notification_serialize’:
lightningd/notification.c:612:13: error: ‘deprecated_apis’ undeclared (first use in this function)
  612 |         if (deprecated_apis) {
      |             ^~~~~~~~~~~~~~~
make: *** [Makefile:299: lightningd/notification.o] Error 1
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-10 13:37:22 +02:00
Vincenzo Palazzo
36ff6aa910 log: increase the log for ignore fee to unsual
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-07-10 16:52:17 +09:30
Vincenzo Palazzo
77d787b975 This commit fixes a bug introduced in 64b1ddd761 that does not
ignore the min fee as specified by the user setting.

We explicitly allow the user to ignore the fee limits, although this comes with inherent risks.

By enabling this option, users are explicitly
I was aware of the potential dangers.
There are situations, such as the one described in [1], where it
becomes necessary to bypass the fee limits to resolve issues like a stuck channel.

BTW experimental-anchors should fix this.

[1] https://github.com/ElementsProject/lightning/issues/6362

Reported-by: @pabpas
Fixes: 64b1ddd761
Link: https://github.com/ElementsProject/lightning/issues/6362
Changelog-Fixes: do not ignore the ignore-fee-limit option during
update_fee
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-07-10 16:52:17 +09:30
Matt Morehouse
a264e10564 fuzz: initial sha256 seeds
Corpus generated from 100+ CPU hours of fuzzing and then minimized.
2023-07-10 14:56:50 +09:30
Matt Morehouse
8547912708 fuzz: initial ripemd160 seeds
Corpus generated from 100+ CPU hours of fuzzing and then minimized.
2023-07-10 14:56:50 +09:30
Matt Morehouse
f0f043c742 fuzz: add sha256 target
The target fuzzes CCAN's implementation of sha256 and compares the hash
results against OpenSSL's implementation.
2023-07-10 14:56:50 +09:30
Matt Morehouse
8d7885abc1 fuzz: add ripemd160 target
The target fuzzes CCAN's implementation of ripemd160 and compares the
hash results against OpenSSL's implementation.
2023-07-10 14:56:50 +09:30
Matt Morehouse
33da575fd0 ccan: rename ripemd160 functions
The symbols were being pulled in from libwally instead of the from our
own CCAN. Do the same trick we use for sha256.
2023-07-10 14:56:50 +09:30
Shahana Farooqui
1606ef1487 lightningd: actually deprecate old notification fields now tests and plugins are all updated.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `connect`, `disconnect` and `block_added` notification fields outside the same-named object (use .connect/.disconnect/.block_added sub-objects)
2023-07-10 14:51:11 +09:30
Rusty Russell
99b885c84c plugins: update for new deprecation API where all fields are nicely wrapped.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-10 14:51:11 +09:30
Shahana Farooqui
4e5e38f4b0 tests: notification response fixes
tests: notification response fixes
2023-07-10 14:51:11 +09:30
Shahana Farooqui
8e14b3ff8f doc: Typo in balance_snapshot notification object 2023-07-10 14:51:11 +09:30
Shahana Farooqui
b9fba1c0b5 notification: block_added json key updated from block to block_added
Changelog-Added: JSON-RPC: `block_added` notification wraps fields in `block_added` object (notification consistency)
2023-07-10 14:51:11 +09:30
Shahana Farooqui
79092c16b1 notification: disconnect with json key disconnect
Changelog-Added: JSON-RPC: `connect` and `disconnect` notifications now wrap `id` field in a `connect`/`disconnect` object (consistency with other notifications)
2023-07-10 14:51:11 +09:30
Shahana Farooqui
e247d16536 notification: connect with json key connect 2023-07-10 14:51:11 +09:30
Vincenzo Palazzo
d58414b062 docs: remove the ppa reference from the docs
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-07-10 13:48:36 +09:30
Vincenzo Palazzo
b3f72f9eed fix(docs): Fix instructions for installing cln on FreeBSD
This commit is adding the instructions for compiling cln on FreeBSD,
because it looks that we not longer compile with the FreeBSD package
manager, and I have no idea who the mantainer of this package is.

Link: https://github.com/ElementsProject/lightning/issues/6301
Reported-by: @bektar
Changelog-None
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-07-10 13:48:36 +09:30
Christian Decker
706b47587f cln-rpc: Remove wildcard import from model
There is a name clash between request and response for the delinvoicestatus
param.
2023-07-10 13:47:38 +09:30
Christian Decker
b81212826f ci: Add pyln-grpc-proto to auto-publication workflow 2023-07-10 13:47:38 +09:30
Christian Decker
6583f0dead pyln: Create a dedicated pyln-grpc-proto package
Having the grpc bindings in the pyln-testing package was always a bit
strange, however it came with additional issues. Due to the way that
protos are handled by protobuf, any name clash, independently of where
we import the protos, would cause an import error. This usually
happens in diamond-pattern dependencies, and so pull out the generated
files into their own package that everyone else can rely on.

Changelog-None
2023-07-10 13:47:38 +09:30
Shahana Farooqui
5053999ff3 doc: Updated instructions
Updated release instruction after build-release.sh fixes.
Add Docker buildx and QEMU instruction in reproducible
2023-07-10 13:24:51 +09:30
Shahana Farooqui
4bdae80486 build: updated build-release script
Updated build release for:
- force-version and force-mtime checks
- zipfile is not optional anymore, it will be created before all other action
- CLN's amd64, arm32v7 and arm64v8 docker setup
- Verify release feature
2023-07-10 13:24:51 +09:30
Shahana Farooqui
fe39047a1b build: passing release version for make
Passing release version for make command to ensure that refresh submodule (from external makefile) runs with --force-version option as well.
2023-07-10 13:24:51 +09:30
Shahana Farooqui
2f489b97d1 docker: cl-repro Dockerfile update
Updating bionic, focal and jammy cl-repro builds:
- Rust version from 1.62 to 1.65
- Removed git clone, only mount option
- Removed sha256sums generation
2023-07-10 13:24:51 +09:30
Shahana Farooqui
e752840849 docker: Core lightning Dockerfile updates
- Updated Dockerfile for amd64
- Removed linuxarm32v7.Dockerfile
- Removed linuxarm64v8.Dockerfile
- Added Dockerfile.arm32v7 for grpc enabled image
- Added Dockerfile.arm64v8 for grpc enabled image
2023-07-10 13:24:51 +09:30
Shahana Farooqui
7dda91f7f6 gitignore: Ignore SHA256SUMS from release verify
Ignore SHA256SUMS and SHA256SUMS.asc used for release verification
2023-07-10 13:24:51 +09:30
Rusty Russell
212c4dba76 pytest: fix flake in test_onchain_middleman_simple with anchors.
Greg Sanders helped debug this:

```
        # Payment should succeed.
>       l1.bitcoin.generate_block(1, wait_for_mempool=txid1)

tests/test_closing.py:2145: 
...
>               raise ValueError("Timeout while waiting for {}".format(success))
E               ValueError: Timeout while waiting for <function BitcoinD.generate_block.<locals>.<lambda> at 0x7f7cd7271560>
```

The lgos show the HTLC tx doesn't go through because it double-spent an input but didn't spend enough:

```
2023-07-06T03:05:54.3424456Z lightningd-2 2023-07-06T02:57:37.490Z DEBUG   plugin-bcli: sendrawtx exit 26 (bitcoin-cli -regtest -datadir=/tmp/ltests-yihsd7f4/test_onchain_middleman_simple_1/lightning-2/ -rpcport=39033 -rpcuser=... -stdinrpcpass sendrawtransaction 0200000000010220f632933db174def3b4bd879ad892e28f4422ee6e844bda27c4e5bdbc754fda030000000001000000975e48aeaced953f7c2b85f20e85b5c241258cb9dbd2ba13de3038daba6316330100000000fdffffff02a1860100000000002200202df545ea882889846c52fc5e111ac07ce07e0c09418ac15743a6f6284c2a4fa739391e000000000016001461ed06c0632af284ec9e192e97758fc04692c8290500473044022064446978d7f15d923237d44d7701e4a09a2d03ebb0a7e2c42e22c67435ad2fcc02201c51002fb72920978c79872e427acd90a13423e841b4198717c1771e7355ba4683473044022059ed9e2c536a71fac3cd63c7349c64a4445f0f936270295518a8aa03607d1e9d022064d318669a7602f585c84fe80aa95487816920c2a8ac26836601fbb369068ff38320478171dbde0e1c243e5c1ae23bcce446ab361197ca10d57c1d8f030cc9ff52158c76a914f5fb7361abefe39af0c3ab31d5be71096deeb4198763ac6721034bbdd4e5a933a3a83f6c3d22714cf23c452cb0c5ac8e429eea14992262787e687c8201208763a9147a3d3592e3d93a525beed57acce3eb70ba1b514288527c21039a62150fd6808d6c68aabd1e9d144c93e84a8f54f4e0f9ec5b3c37eee0a051c752ae6775017eb175ac6851b275680247304402200fde6a943a2f7f0287af2f5c5872326a4b8ad7020a098c4e05c28d2c2a0f2018022053a1263f982b98bb8dfcbb3a66641fe9f298e7ab432a78d21a2e79190d74753f012102397b0449a60d9d35634401bceaf3beb6118fc229b8552bd7122f735808b28aee00000000) error code: -26\nerror message:\ninsufficient fee, rejecting replacement 76f438f176d8f9beabb286f53c81aa7dcb4948d12f034f51753f4dd9071d6a74; new feerate 0.00029576 BTC/kvB <= old feerate 0.00054659 BTC/kvB
```

This is because sometimes we reuse the same UTXO for the anchor push spend as we do for the HTLC.  That would be fine, except that we can have bitcoind mine the commitment tx and not the anchor push, and then we fail to replace it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Rusty Russell
b08791d4fa Makefile: add dependencies to fix parallel build.
```
cc tools/headerversions.c
tools/headerversions.c:3:10: fatal error: 'config.h' file not found
#include "config.h"
         ^~~~~~~~~~
1 error generated.
make: *** [Makefile:299: tools/headerversions.o] Error 1
```

And then:

```
tools/test/enum.c:1:10: fatal error: 'config.h' file not found
#include "config.h"
         ^~~~~~~~~~
1 error generated.
make: *** [Makefile:299: tools/test/enum.o] Error 1
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Rusty Russell
016e48f8d3 CI: use prebuild for (almost all) ci steps.
With the exception of fuzzing, make all builds in the `compile` job,
and simply download them in the other steps.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Rusty Russell
364de00947 CI: clean up workflow files.
We used to have to use environment variables to pass through to
github/scripts/build.sh, but now we run ./configure directly it's
clearer to use explicit flags (though some matrixes still use env vars
for simplicity).

We also don't need to set COMPAT, as it's the default (MacOS tests
that we build without it, but otherwise we assume it's on).

And we make `gather` actually depend on all the other steps!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Rusty Russell
fa596a9446 configure: --enable-debugbuild flag for debug builds.
This controls debug flags for the build, rather than --developer,
which is going away.

I thought about making this flag control the RUST_PROFILE too, but
it seems that we want that set to "release" for CI, whereas for the
C code we want --enable-debugbuild.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Rusty Russell
0c4426a349 lightningd: remove deprecated_apis global, put into lightningd.
We usually have access to `ld`, so avoid the global.

The only place generic code needs it is for the json command struct,
and that already has accessors: add one for libplugin and lightningd
to tell it if deprecated apis are OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Rusty Russell
db7c608e2d common/utils: remove now-unused softref facility.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Rusty Russell
3ce2f28907 gossipd: don't use softref in the seeker.
We keep several peer pointers, but we just add a hook to NULL them
manually when a peer dies, rather than using voodoo.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Rusty Russell
17e0c057aa gossipd: don't use peer softrefs for gossip credit.
We use a "softref" which is a magic pointer which gets NULL'ed when
the object is freed.  But it's heavy, and a bit tricky to use, and we
only use it in gossipd.

Instead, keep the nodeid, and do a lookup (now that's fast) if we want
to credit the sender for valid gossip.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Rusty Russell
6e2a34373c gossipd: put ->daemon pointer into routing_state.
They're closely tied, and duplicating fields was really ugly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Rusty Russell
01670d5e5e gossipd: use htable, not linked list for peers.
This speeds up nodeid lookups, which is useful for the next simplification.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Rusty Russell
9bc1a020d0 connectd: don't try to keep going if gossipd dies.
We will access the freed connection to gossipd.  This is weird to track
down when the *actual* issue is that gossipd died!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Rusty Russell
04f485aee0 db: add generic warn/error function.
This avoids the mess where we override db_fatal for teqsts, and keeps it
generic.

Also allows us to get rid of one #if DEVELOPER, and an ugly global for
bookkeeper.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Rusty Russell
bef137209d libplugin: expose plugin_logv.
If you have your own logging infra, you'll want this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Erik De Smedt
db9dbc5f31 Remove --daemon from lightningd.service
When the `--daemon` flag is used, `lightningd` requires that `--log-file`
is used as well. By consequence, the `lightningd.service` didn't work
out-of-the-box for me.

This changes also sends the logs to `journald`.
The new approach is consistent with the `bitcoind.service` file in the bitcoin-core repository.

I prefer this approach because it comes with automatic log-rotation.
2023-07-09 13:02:10 +09:30
Tony Aldon
dae6a0bcf2 contrib: fix id parsing in IO loop of cowsay.sh plugin 2023-07-06 16:18:46 +09:30
Rusty Russell
e6d23b5677 pytest: test for emergency minimum fund violations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
3e57496af9 plugins: don't let multifundchannel open a new anchor channel without reserves.
If we're opening a channel with a peer which support anchors (and
we do), we tell fundpsbt/utxopsbt to enforce the emergency reserve;
this matters, as it doesn't know about the channel yet, and thus
won't (if it's our first anchor channel).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `fundchannel` and `multifundchannel` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels (or are opening one).
2023-06-29 11:28:47 -04:00
Rusty Russell
0402e645f0 lightningd: add option opening_anchor_channel to fundpsbt, utxopsbt.
This is needed when we know we're *opening* an anchor channel, to
override the "do we already have an anchor channel open?" logic.

Also, document the nonwrapped arg added in v23.02.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundpsbt` and `utxopsbt` new parameter `opening_anchor_channel` so lightningd knowns it needs emergency reserve for anchors.
2023-06-29 11:28:47 -04:00
Rusty Russell
391da2f440 lightningd: don't let them fundpsbt below emergency reserve.
This is the simple version which always tries to keep some sats if we
have an anchor channel.  Turns out that we need something more
sophisticated for multifundchannel, so that's next.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `withdraw` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels (and `all` will be reduced appropriately).
Changelog-Changed: JSON-RPC: `fundpsbt` and `utxopsbt` will refuse to spend funds below `min-emergency-msat` if we have any anchor channels.
2023-06-29 11:28:47 -04:00
Rusty Russell
75aca3cbb6 config: add min-emergency-msat option.
For anchors, we need some sats sitting around in case we need to CPFP
a close.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: `min-emergency-msat` setting for (currently experimental!) anchor channels, to keep funds in reserve for forced closes.
2023-06-29 11:28:47 -04:00
Rusty Russell
2f0e13e793 multifundchannel: use excess_as_change flag to simplify logic.
This was added to fundpsbt/utxopsbt in v0.10, but the spender plugin
didn't take advantage of it, instead calculating its own change amount
and output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
37005b93d3 txprepare: use excess_as_change flag to simplify withdraw/txprepare logic.
This was added to fundpsbt/utxopsbt in v0.10, but the txprepare plugin
didn't take advantage of it, instead calculating its own change amount
and output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00