Commit graph

15278 commits

Author SHA1 Message Date
Rusty Russell
b748c1845c pytest: fix flake in test_closing.py::test_penalty_htlc_tx_timeout
lightningd-3 penalizes lightningd-2 but then it can see the preimage for the HTLC which
has already been timed out:

```
2024-06-24T02:41:29.4633900Z lightningd-3 2024-06-24T02:33:54.073Z **BROKEN** 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-onchaind-chan#1: HTLC already resolved by THEIR_HTLC_TIMEOUT_TO_THEM when we found preimage
```

This is fair: the test deliberately takes l3 offline for long enough
that the HTLC can get timed out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
80cde517c9 lightningd: fix race in initialization.
Found by very slow CI: the two io_breaks() can combine into one, so we
block waiting for the second initialization which never happens.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
9da66100c1 chaintopology: free outstanding requests so we don't get responses while shutting down.
I've never seen this, but the race looks possible so we should close it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
928b7c83fc lightningd: pass explicit context for bitcoind request functions.
That way if you free the context, you free the call (i.e. the callback
will not be called).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
492ed84dbf lightningd: move checkchain_timer out of bitcoind, remove stopped flag.
checkchain_timer is run by chaintopology, so why have the pointer in
bitcoind?

And since we free the timers, we don't need them to self-disable by
checking the stopped flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
f2b033d7e1 lightningd: move all chainheight logic into setup_topology.
Handling half in main() and half here was a mess.  And the name
"max_blockheight" was poor: it was the max in the db, or UINT32_MAX,
but then we changed it depending on what height we wanted to start at.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
ff15b943fe lightningd: remove unused "min_blockheight" from chain_topology struct.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
f643fb88f5 lightningd: wait for bitcoind at startup if it's fallen behind.
Fixes: https://github.com/ElementsProject/lightning/issues/6924
Changelog-Changed: lightningd: we wait for bitcoind if it has somehow gone backwards (as long as header height is still ok).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
59afbac7cf lightningd: remove first_call param for getchaininfo.
It was a weird arbitrary bool passed through from the caller, which we no longer need.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
eb70476fb3 lightningd: split chaintopology init from normal loops.
The current code is confusing: there are polling loops, but we wait for
them to run once. 

Be explicit: make the calls once, then start the loops in begin_topology.

This removes various chain_topology struct members which only exist for
startup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
f318d08e94 lightningd: add arg to bitcoind_estimate_fees callback.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
436bf6c9e1 lightningd: pass height back from getrawblockbyheight.
Several callers stash this, but we have it, so make it explicit:
here's the block you asked for.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
ShahanaFarooqui
b803609247 docs: Removed example usage
Example usage has been either merged with examples or usage keys. This heading was not required separately.

This PR edits & rearranges Examples, Example Notifications and Example Usage data.

Changelog-None.
2024-06-22 17:02:33 +02:00
ShahanaFarooqui
e94b793f05 docs: Example notifications heading
- Added missing Example Notifications Heading
- Renamed `example_json_notifications` to `example_notifications`
- Moved `example_notifications` to the end of the page
2024-06-22 17:02:33 +02:00
ShahanaFarooqui
976c0c90bc docs: Changes examples format
- Changed ALL `doc/schemas/lightning-*.json` file's `json_example` to `examples`
- Change the heading from example to examples
- Bring shell command before the json command
- Move Example to the end of the page
- Remove horizontal line from Example
2024-06-22 17:02:33 +02:00
Christian Decker
1347e86671 wallet: Log to chan->log rather than wallet->log
Suggested-by: Rusty Russell <@rustyrussell>
2024-06-22 14:50:21 +02:00
Christian Decker
001b190c61 feat(main): Print channel status when loading it from DB
I was trying to debug a node with several multiplexed channels, and
was finding it a bit difficult to determine which channel index
matches which result of `listpeerchannels` as well as figuring out
what their status was. This just prints the status in string format
when loading the channel from the DB.

Changelog-Changed: wallet: The channel status is printed when loading it from the DB
2024-06-22 14:50:21 +02:00
Rusty Russell
e21b70cad0 pytest: fix default/description order in zeroconf test plugin.
Noticed as I was debugging.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 17:20:06 +09:30
Christian Decker
31cf9225f4 pyln: Turn the plugin options into a real Type (uppercase T)
[ Fix not to include 'value' and 'default' (if None) in getmanifest response --RR ]
[ Fix to support [] operator for existing plugins (including our test ones!) --RR ]
2024-06-20 17:20:06 +09:30
Rusty Russell
1eb1db0e24 pyln-client: allow dynamic option setter to throw exceptions
And don't set the value unless it passes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 17:20:06 +09:30
Christian Decker
457d25c1ce pyln: Drive-by type fixed 2024-06-20 17:20:06 +09:30
Christian Decker
2737f2e97e pyln: Add and test a callback based setconfig listener
I was looking into using the `threading.Condition` but since we're
already rather heavily using callbacks, this allows us to stay
single-threaded, and not having to completely hook the `setconfig`
function.

Changelog-Added: pyln-client: Added a notification mechanism for config changes
2024-06-20 17:20:06 +09:30
Rusty Russell
572ccac0e5 pyln-client: allow dynamic=True add_option to actually allow updates.
We didn't actually *change* the value you'd see, when we got a setconfig call!

Changelog-Added: pyln-client: implement setconfig hook for plugins so you can see changes in `dynamic` options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 17:20:06 +09:30
Rusty Russell
ead211e5e4 autoclean: call list in easy stages.
listforwards on a large node can easily run out of memory.  Sip, don't
gulp!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
c62e1f432f autoclean: use filter to reduce processing.
We can filter down to only the list* fields we need.  In the case of a
node with 1M forwards, this reduces listforwards time from 5 seconds
to 4 seconds.  It will also reduce memory consumption.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
c7ce9c55df libplugin: add plugin_option_dev_dynamic (our first dynamic dev option coming).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
1f9ddd24b2 libplugin: support filtering of outgoing commands.
This is crude, handing a raw JSON string, but it works.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
d2465d4498 autoclean: rework logic to clarify subsystems.
There are really three subsystems: invoices, forwards and sendpays,
each of which has two variants we care about (successes and failures).
If we split the code that way, we can extract the core differences in
each of these cases and share most of the logic.

It's a bit awkward to iterate over each "subsystem" in the JSON
parameter sense, so we have some iteration code to do that where we
need to.

The result is going to be much easier to paginate!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
83eed5ce8e autoclean: use a pointer for our global clean_info.
We're about to make this a more complex struct, so introducing a
new_clean_info() function unifies the code paths.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
84b6601bb3 autoclean: remove autocleaninvoice command.
Changelog-Removed: JSON-RPC: `autocleaninvoice` command (deprecated v22.11, EOL v24.02)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
b525e26fa5 lightningd: don't load all forwardings into memory at once.
Use modern-style iterators: this can be huge and we will run out of
memory!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
daywalker90
6ad8cfdffa CI: upgrade CI actions to latest releases
Changelog-None
2024-06-19 21:21:00 -07:00
fanquake
2614dfc104 ci: use Bitcoin Core 27.1
Changelog-None
2024-06-20 11:28:21 +09:30
Niklas Gögge
259015720e fuzz: Add input for from_bech32_charset off-by-one bug 2024-06-20 10:53:50 +09:30
Rusty Russell
1a1696f2f9 lightningd: don't try to "save" incoming channel by closing early if incoming already onchain.
From test_penalty_htlc_tx_timeout[False] flake (with keys replaced by l2 and l5, for clarity)

```
lightningd-3 2024-06-04T04:34:00.942Z UNUSUAL l2-chan#1: Peer permanent failure in CHANNELD_NORMAL: Funding transaction spent
lightningd-3 2024-06-04T04:34:01.570Z UNUSUAL l5-chan#3: Peer permanent failure in CHANNELD_NORMAL: Funding transaction spent
lightningd-3 2024-06-04T04:34:01.655Z UNUSUAL l5-chan#3: Abandoning unresolved onchain HTLC at block 132 (expired at 125) to avoid peer closing incoming HTLC at block 131
lightningd-3 2024-06-04T04:34:02.802Z **BROKEN** l5-chan#3: FUNDS LOSS of 50000000msat: peer took funds onchain before we could time out the HTLC, but we abandoned incoming HTLC to save the incoming channel
```

So, we were already closing l2, no reason to abandon it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 23:58:50 +02:00
Rusty Russell
db67df3364 lightningd: log BROKEN if we lost funds, only unusual if we closed abandoned HTLC.
Started seeing these in my logs, and it's the wrong way around!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 23:58:50 +02:00
Rusty Russell
0afaf2f3ca msggen: don't spam every time we rebuild.
2923 lines of output?  Even with warnings-only, it's 59 lines, so only
enable that with `make V=1`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 12:34:41 +02:00
Rusty Russell
7c3aa9477f pyproject.toml: Update ancient flake8
v4.0 no longer works for me (see below, and widely reported elsewhere).
v5.0 doesn't understand f strings, and creates a flood of complaints.
v6.0 requires python >= 3.8.1, so we need to update that.
v7.0 is the latest, but why push it.

```
make check-python-flake8
Traceback (most recent call last):
  File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/bin/flake8", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/main/cli.py", line 22, in main
    app.run(argv)
  File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/main/application.py", line 375, in run
    self._run(argv)
  File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/main/application.py", line 363, in _run
    self.initialize(argv)
  File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/main/application.py", line 343, in initialize
    self.find_plugins(config_finder)
  File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/main/application.py", line 157, in find_plugins
    self.check_plugins = plugin_manager.Checkers(local_plugins.extension)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/plugins/manager.py", line 363, in __init__
    self.manager = PluginManager(
                   ^^^^^^^^^^^^^^
  File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/plugins/manager.py", line 243, in __init__
    self._load_entrypoint_plugins()
  File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/plugins/manager.py", line 261, in _load_entrypoint_plugins
    eps = importlib_metadata.entry_points().get(self.namespace, ())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'EntryPoints' object has no attribute 'get'
make: *** [Makefile:535: check-python-flake8] Error 1
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 12:34:41 +02:00
ShahanaFarooqui
3d841d5b8a doc: Added Miscellaneous section for Docker users
Added CLN image execution and testing hints for Docker users.

Changelog-None.
2024-06-19 18:41:53 +09:30
ShahanaFarooqui
aba767114c doc: fixed broken link to setting-up-multiarchqemu-user-static
Changelog-None.
2024-06-19 18:41:07 +09:30
ShahanaFarooqui
7e46eb8bc4 docker: Update github actions for Docker releases
This PR is adding github action for auto Docker releases. This will streamline Docker image builds & releases for enhanced efficiency, reliability, and consistency in managing Docker-based deployments.

The build & release event will be triggered:
- On any new tag creation without rc suffix.
- Manually for testing purposes.

Changelog-None.
2024-06-19 18:40:07 +09:30
ShahanaFarooqui
44de5023cb docs: Updated scripts for rpc examples
This update will add lightning-cli examples and make examples heading collapsible.

- The JSON EXAMPLES heading is collapsible now

- Examples in lightning-cli format also

- manpage examples have better formatting

Changelog-None.
2024-06-19 15:56:22 +09:30
ShahanaFarooqui
a4e7079fc1 docs: Removed null values from example request params 2024-06-19 15:56:22 +09:30
ShahanaFarooqui
ec250069b9 docs: Updated req params type from string to object 2024-06-19 15:56:22 +09:30
Rusty Russell
2175cd7db3 onchaind: remove support for rescuing non-static channels when we've lost data.
We could get the current key from the reestablish message even if we'd
lost our db, but there are very few of these channels left: we upgraded to use them
in the 2019-01-09 release.

We will eventually remove support altogether, but this is a nice removal of
some ugly code for something which "never happens".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 15:54:24 +09:30
Rusty Russell
002dc60b33 Gossip: BOLT catch, remove initial_routing_sync.
Everyone sends a gossip_timestamp_filter message these days to start gossip.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 15:54:24 +09:30
Rusty Russell
06cf5ac841 Doc: update bolts to assume gossip_queries under the new meaning.
Everyone understands gossip_queries now, but peers leave it unset to indicate
they have nothing useful to say.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 15:54:24 +09:30
Rusty Russell
65ef909bd3 Doc: update to BOLT which assumes option_var_onion_optin.
We have assumed this for a long time, so nothing changes.

Confusingly, this BOLT commit also cleaned up one reamining `option_anchors_zero_fee_htlc_tx`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 15:54:24 +09:30
Rusty Russell
7f2aedc76f common: BOLT update: option_anchors_zero_fee_htlc_tx is now simply "option_anchors".
This is a difficult transition for us: this string appears in channel
types.  We make the transition now in the understanding that it will
be more difficult in future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Changelog-Deprecated: JSON-RPC: `listpeers` `features` array string "option_anchors_zero_fee_htlc_tx": use "option_anchors" (spec renamed it).
Changelog-Added: JSON-RPC: `listpeers` `features` array string uses "option_anchors" for feature 22/23, following renaming in BOLT 9.
Changelog-Changed: JSON-RPC: `listclosedchannels`, `listpeerchannels`, `openchannel_update`, `openchannel_init`, `fundchannel`, `fundchannel_start` and `multifundchannel`: `channel_type` array `names` now contains "anchors" instead of "anchors_zero_fee_htlc_tx".
Changelog-Changed: lightningd: `--list-features-only` now lists "option_anchors" instead of "option_anchors_zero_fee_htlc_tx".
2024-06-19 15:54:24 +09:30
Rusty Russell
7d3d763b96 channeld: BOLT update, no longer allow creation of old (experimental-only!) non-zero-fee anchor channels.
These were removed from the spec.

We still support existing ones, though we were the only implementation
which ever did, and only in experimental mode, so we should be able to
upgrade them and avoid a forced close, with a bit of engineering...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 15:54:24 +09:30