Commit Graph

12342 Commits

Author SHA1 Message Date
Michael Schmoock
29f81baac9 wireaddr: is_dnsaddr allow underscore in hostname
The hostname part of a DNS FQDN can allow for additional characters
other than specified in `man 7 hostname`.

This extends is_dnsaddr and the test issue #5657.
Also fixes a typo in a comment.

Changelog-Fixed: wireaddr: #5657 allow '_' underscore in hostname part of DNS FQDN
2022-12-06 14:26:19 +01:00
nicolas.dorier
a96ff3b097 Update the contrib arm32v7 and arm64v8 dockerfiles 2022-12-06 10:48:57 +01:00
Rusty Russell
e778ebb9af wallet: only log broken if we have duplicate scids in channels.
This was reported, but the channel was closed.  So however we ended
up with a duplicate, we're no *worse* off than we were before migration?

Fixes: #5760
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-12-06 10:45:20 +01:00
Rusty Russell
37590eeeb2 common: fix arm32 compile breakage.
It's obviously the incorrect type, while our CI didn't catch it, Nicholas did:

```
plugins/fetchinvoice.c:1362:30: error: conversion from 'long long unsigned int' to 'size_t' {aka 'unsigned int'} changes value from '18446744073709551615' to '4294967295' [-Werror=overflow]
 1362 |      || tlv_span(wire, 1001, UINT64_MAX, NULL) != 0) {
```

Reported-by: @NicholasDorier
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Build: arm32 compiler error in fetchinvoice, due to bad types on 32-bit platforms.
Fixes: #5776
2022-12-06 10:44:39 +01:00
Rusty Russell
1d4f7f023d Revert "lightningd: always require "jsonrpc": "2.0" in request."
This reverts commit 43b037ab0b.

Nicholas Dorier says BTC Payserver still wants this for another year
or so.

Changelog-Added: JSON-RPC: reverts requirement for "jsonrpc" "2.0" inside requests (still deprecated though, just for a while longer!)
2022-12-06 10:43:54 +01:00
Alex Myers
70fc702ee4 reckless: further verbosity/squelch of pip output 2022-12-06 10:42:29 +01:00
Alex Myers
8653d1200f reckless: avoid redundant include statement in config 2022-12-06 10:42:29 +01:00
Alex Myers
3f2e923e81 reckless: fix verbose option
The switch to logging enabled verbose output regardless of the option
flag. Here the functionality is restored.

Changelog-Fixed: reckless verbosity properly applied.
2022-12-06 10:42:29 +01:00
Rusty Russell
9751502ff5 jsonrpc: fix error when we abort batching due to timeout.
The read_json() call expects len_read to be the amount of *new*
data read.  If we call this back without resetting, it will parse
this much random junk in the buffer.

Fixes: #5766
Changelog-Fixed: Plugin: `autoclean` could misperform or get killed due to lightningd's invalid handling of JSON batching.
2022-12-06 10:42:05 +01:00
Rusty Russell
24d7aad3d0 autoclean: fix uncleaned stats when we don't clean due to being too new.
We only incremented this if it was the wrong state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `autoclean-once` response `uncleaned` count is now correct.
2022-12-06 10:42:05 +01:00
Rusty Russell
4613b29815 doc: document autoclean-once command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-12-06 10:42:05 +01:00
Christian Decker
a31575ca0b tools: Add multi-language support to blockreplace.py
Suggested-by: Rusty Russell <@rustyrussell>
2022-12-06 11:39:53 +10:30
Christian Decker
744d111cea doc: Create a blockreplace tool to update generated blocks in docs
We introduced a minor issue in #5757 that was causing the manpages to
be added every time we regenerate instead of replacing them. The
script was a bit unscrutable, and we do this block-replacement in
several places I thought it might be a good idea to have a dedicated
tool to do it.

This allows us to have simpler Makefiles whenever we update a
generated block inside another file. It's python, but does not have
dependencies :-)

Changelog-None
2022-12-06 11:39:53 +10:30
Christian Decker
d7cd3e1cb5 pyln: Fix an issue with the LightningConnection short-reading
We may end up with a short read, that is not empty, when a message
gets fragmented during transport, so we need to loop until we either
reach the full size or we have an empty read indicating a dropped
connection.

Changelog-None
2022-12-02 12:20:24 +01:00
Christian Decker
280b49a677 meta: Update changelog for v22.11 final 2022-11-30 20:42:59 +01:00
Rusty Russell
3ca1c70c44 lightningd: don't cap spendable_msat/receivable_msat for wumbo channels.
If we both support large channels, we can actually send giant HTLCs.

This, in turn, fixes pay (which relies on this field!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: `pay` now knows it can use locally-connected wumbo channels for large payments.
Fixes: #5250
Fixes: #5417
2022-11-30 19:51:22 +01:00
Rusty Russell
b6555dccaf pytest: test for wumbo direct payments.
We should be able to make larger payments if we're directly connected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-30 19:51:22 +01:00
Rusty Russell
15d0a8bec8 connectd: don't spam logs when we're under load.
This happens a lot with my node with rc2, so drop it to debug.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-30 19:31:38 +01:00
Vincenzo Palazzo
2064982006 lightningd: do not abort while parsing hsm pwd.
This is a mistake that I introduced while I implemented the lightningd custom error for hsmd.

In particular, when the command line parser check
if the file is encrypted make an additional check regarding the existence of the file.

This can be a not useful check, but due to the delicate nature of the hsm file, it is better to check if exist and if it doesn't exist an informative line inside the log is emitted that notifies the user that the file does not exist.

This log may return useful while debugging disaster
(that can happen) to understand that there is something strange (eg. the user moves the hsm file somewhere else).

Fixes https://github.com/ElementsProject/lightning/issues/5719

Changelog-Fixed: lightningd: do not abort while parsing hsm pwd

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-11-30 19:06:13 +01:00
Christian Decker
dbb38e2c7d docs: Add the reckless manpage to the readthedocs generation
Changelog-None
2022-11-30 17:03:08 +01:00
Rusty Russell
19300de58f lightningd: correctly exit when an important-plugin fails to start.
This was found by tests/test_plugin.py::test_important_plugin and
was NOT a flake!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-None: only just committed
2022-11-30 15:47:31 +01:00
Christian Decker
110ed3b1a9 submod: Switch lnprototest to clone from github.com/rustyrussell/lnprototest
This is the official root repository, and some tools (pip for example) doesn't like
pointing to repos that don't actually contain the commit that we point to. Funny
that Github actually shows non-existent commits in clones (bitcoin/bitcoin has been
impersonated before too).
2022-11-30 15:16:49 +01:00
Christian Decker
090facd79b ci: Temporarily disable lnprototest tests
They are broken, and not a good signal.
2022-11-30 14:24:12 +01:00
Christian Decker
dfb963e249 db: Backfill missing HTLC IDs in the forwards table
We have a primary key that is spanning the `in_channel_id` and the
`in_htcl_id`. The latter gets set to NULL when the HTLC and channel
gets deleted, so we coalesce with a random large number that is
unlikely to collide for the primary key.
2022-11-28 14:47:43 +01:00
Christian Decker
22798b80b3 meta: Adjust changelog for v22.11rc3 2022-11-27 15:00:57 +01:00
Rusty Russell
626998efce lightningd: don't timeout plugins if init is slow!
This is a minimal fix: we wait until all plugins reply from init before
continuing.  Really large or busy nodes can have other things monopolize
lightningd, then the timer goes off and we blame the plugin (which has
responded, we just haven't read it yet!).

The real answer is to have some timeouts only advance when we're idle,
or have them low-priority so we only activate them when we're idle (this
doesn't apply to all timers: some are probably important!).  But
this is a minimal fix for -rc3.

Fixes: https://github.com/ElementsProject/lightning/issues/5736
Changelog-Fixed: plugins: on large/slow nodes we could blame plugins for failing to answer init in time, when we were just slow.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-27 14:58:42 +01:00
Christian Decker
3d311c96b1 cln-plugin: Adjust visibility of some internals 2022-11-25 16:48:33 +01:00
Christian Decker
db62d542e1 cln-plugin: Make the configuration in init public
It was set to crate level visibility for some reason, not all that
helpful :-)

Suggested-by: Sergi Delgado Segura <@sr-gi>
2022-11-25 16:48:33 +01:00
Rusty Russell
8ebde4574d CHANGELOG.md: include the v0.12.1 CHANGELOG entries!
This should have been merged into master earlier!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-25 16:30:07 +01:00
Rusty Russell
47e9e3d399 CHANGELOG.md: order into a more user-first ordering.
And internal changes don't get CHANGELOG entries, so removed
"- Added interactive transaction building routine ([#5287])".

We need to come up with some way of marking rust crate changes,
as it's not immediately obvious what "- cln-plugin" means?

Maybe we switch from JSON-RPC to Control as a more general
prefix?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-25 16:30:07 +01:00
Christian Decker
64aa42e85b doc: Add a readme to the cln-grpc proxy
These are some common questions that come up from time to time, so
let's make sure we address them.
2022-11-25 16:06:41 +01:00
Christian Decker
02f9c2df24 autoclean: Fix a null-pointer derefence when checking HTLC age
The autoclean plugin would assume we have a `resolved_time` which may
not be true for oldish nodes that predate our annotations.

Changelog-None Unreleased change

Reported-by: <@devastgh>
2022-11-24 12:00:24 +10:30
Rusty Russell
ece77840f9 pyln-client, libplugin, rust cln-plugin: explicitly flag that we allow non-numeric JSON ids.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-21 11:23:54 +01:00
Rusty Russell
d5ce5cbab3 lightningd: only use non-numeric JSON ids if plugin says we can.
We also remember whether the id is a string or not, for replacement in
JSON passthrough.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-21 11:23:54 +01:00
Rusty Russell
24651f57ad plugins: set non_numeric_ids flag based on getmanifest nonnumericids field.
And document support for it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `getmanfest` response can contain `nonnumericids` to indicate support for modern string-based JSON request ids.
Changelog-Deprecated: Plugins: numeric JSON request ids: modern ones will be strings (see doc/lightningd-rpc.7.md!)
2022-11-21 11:23:54 +01:00
Rusty Russell
b1f50c825f doc: document how to construct JSON ids in modern plugins and utilities.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-21 11:23:54 +01:00
Christian Decker
94928de479 meta: Update CHANGELOG for release candidate v22.11rc2 2022-11-18 16:14:58 +01:00
Christian Decker
2e270ea7d3 rs: Bump cln crate versions to 0.1.1 2022-11-18 15:10:32 +01:00
Christian Decker
d2dae46de9 docker: Fix the dockerfile
Turns out that we were once again mixing non-venv and venv python, so
let's go full venv.

Changelog-None
2022-11-18 15:10:32 +01:00
Christian Decker
9481fb8815 msggen: Be less magic in detecting the repo root
We were assuming that we are running in a git repo, which is not true
for the PPA
2022-11-18 15:10:32 +01:00
Christian Decker
4f5ab638df grpc: Add the experimental optional flag to protoc
The `optional` keyword was first removed, and then re-added again, and
then it was made into an experimental flag. Come on people, decide
already which one it is...
2022-11-18 15:10:32 +01:00
Christian Decker
00dc0f082e make: Fix external/lowdown clean targets 2022-11-18 15:10:32 +01:00
Christian Decker
6518f6f26a make: Make the Makefile make 4.4 compatible
Fixes #5693

Changelog-None
2022-11-18 15:10:32 +01:00
Christian Decker
d27591ff54 ld: Replace list of outgoing_txs with a hash table
We had a quadratic check when processing a block, and this just
reduces it to be linear. Combined with the previous fix of adding txs
multiple times, this should speed up block processing considerably.
2022-11-18 13:39:09 +01:00
Christian Decker
351b8999c9 ld: Add an outgoing_txs_map htable to avoid costly lookups
We were using a quadratic lookup to locate the transactions we
broadcast, so let's use an `htable` instead, allowing for constant
lookups during block processing.
2022-11-18 13:39:09 +01:00
Christian Decker
c7ff3c4288 ld: Do not blindly add rebroadcasts to outgoing_tx set
The list of outgoing transactions was growing with every
rebroadcast. Now we just check whether it is already in the list and
don't add it anymore

Changelog-Fixed: ld: Reduce identification of own transactions to not slow down over time, reducing block processing time

Suggested-by: Matt Whitlock <@whitslack>
2022-11-18 13:39:09 +01:00
Rusty Russell
f2291c44d6 onchaind: cap RBF penalty fee for testnet/regtest
On testnet I noticed if we can't reach bitcoind for some reason, we'll
keep RBFing our penalty tx ("it didn't go in, RBF harder!!").  Makes
no sense to grossly exceed the amount needed for next block, so simply
cap penalty at 2x "estimatesmartfee 2 CONSERVATIVE".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-18 13:38:42 +01:00
Rusty Russell
9e41754e7d wallet: fix typo in debug message.
"coinbase" test is backwards!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-18 13:38:42 +01:00
Rusty Russell
d94b715bcf wallet/wallet.c: don't declare unused variable.
Ubuntu clang 15.0.2-1 complains:

```
wallet/wallet.c:280:6: error: variable 'i' set but not used
      [-Werror,-Wunused-but-set-variable]
        int i;
            ^
wallet/wallet.c:339:6: error: variable 'i' set but not used
      [-Werror,-Wunused-but-set-variable]
        int i;
            ^
wallet/wallet.c:4768:9: error: variable 'count' set but not used
      [-Werror,-Wunused-but-set-variable]
        size_t count;
               ^
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-18 12:26:59 +01:00
Rusty Russell
5e76c74622 tools/generate_wire.py: don't declare unused for variable.
Ubuntu clang 15.0.2-1 complains:

```
wire/peer_exp_wiregen.c:257:14: error: variable 'i' set but not used [-Werror,-Wunused-but-set-variable]
        for (size_t i = 0; *plen != 0; i++) {
                    ^
wire/peer_exp_wiregen.c:1373:14: error: variable 'i' set but not used [-Werror,-Wunused-but-set-variable]
        for (size_t i = 0; *plen != 0; i++) {
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-18 12:26:59 +01:00