Commit graph

15055 commits

Author SHA1 Message Date
Rusty Russell
d922c89b4d doc: don't include exact version in manpages.
This slows compilation somewhat when we make small changes (e.g. making the tree dirty).
Instead, simply mark them as "pre-XXX" or "XXX".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-02 10:00:55 +02:00
Rusty Russell
d6ae456309 doc: speed "noop" compilation times by not force-building markdown and man pages every time.
Their dependencies seem correct, so we shouldn't have to build them every time.

Before:
```shell
$ time make -s
real	0m5.183s
user	0m5.134s
sys	0m11.715s
```

After:

```shell
$ time make -s
real	0m0.784s
user	0m0.775s
sys	0m1.159s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-02 10:00:55 +02:00
Erik De Smedt
62e82c20d7 Fix: Add notifications.rs to $(CHECK_GEN_ALL) target
The file `cln-rpc/src/notifications.rs` is generated by `msggen`.

The `Makefile` has tooling to update the file when needed
- `make check-gen-updated` should error if file isn't properly updated
- `make gen` updates the file

This comit fixes both behaviors mentioned above.
2024-07-02 11:42:14 +09:30
ShahanaFarooqui
7783658489 ci: Unify bitcoin versions for whole project
- Updated all Bitcoin version 27.1
- Updated all Elements versions 23.2.1

Changelog-Changed: Updated Bitcoin to v27.1 and Elements to v23.2.1
2024-07-02 11:41:36 +09:30
ShahanaFarooqui
545d67eaf6 doc: Updated warning box for wss-proxy
Bug Fix: The wss-proxy plugin throws error `plugin-wss-proxy: Killing plugin: disabled itself at init: Error in parsing options: 'NoneType' object is not subscriptable` if CLN is not listening on websocket.

Reference: https://github.com/ElementsProject/lightning/issues/7386#issuecomment-2175766309

Changelog-None.
2024-06-28 11:03:07 +09:30
ShahanaFarooqui
39626b9f8f clnrest: Fixes ssl_version deprecated warning
Reference Issue #6931: Fixes `Warning: option `ssl_version` is deprecated and it is ignored. Use ssl_context instead.`

Changelog-None.
2024-06-28 10:54:44 +09:30
ShahanaFarooqui
9f599365f9 clnrest: Updated requirements.txt (without coincurve)
As suggested by @cdecker, I compared `poetry show --tree` results from before/after `cd plugins/clnrest/ && poetry export --output requirements.txt && pip install -r requirements.txt` to check for any significant difference. There is no difference in the tree.

But poetry export removed coincurve & other dependencies from clnrest's requirements.txt.

Changelog-None.
2024-06-28 10:46:14 +09:30
daywalker90
c251686c6c pyln-proto: update coincurve to v20
Changelog-None
2024-06-28 10:46:14 +09:30
ShahanaFarooqui
7baa1e93ca docs: Fixes typo in library name flask-restx
clnrest installation instruction on the markdown tries to install `flask_restx` not `flask-restx`.

Reference: clnrest plugin complains of flask_restx missing (#7383)

Changelog-None.
2024-06-26 21:31:49 -07:00
ShahanaFarooqui
cd07ce83f7 docs: Moved python plugins after CLN instructions
Changelog-None.
2024-06-26 21:31:49 -07:00
Epic Curious
6c338d6173 fix(deps): add pytest dependency for make check 2024-06-26 21:31:49 -07:00
Epic Curious
41c766882b docs: chore: bump lightning tag in installation.md 2024-06-26 21:31:49 -07:00
Epic Curious
5c43d44800 fix(deps): add python3-pip to clnrest dependencies 2024-06-26 21:31:49 -07:00
Epic Curious
1cb749ecbb fix(deps): add libsecp256k1-dev to clnrest plugin instructions 2024-06-26 21:31:49 -07:00
Epic Curious
02262586ec fix(deps): add libffi-dev build dependency
Fix the fatal error  `ffi.h: No such file or directory` during pip3 install requirements.txt
2024-06-26 21:31:49 -07:00
ShahanaFarooqui
51e46339a7 docs: notifications schema update
Changelog-None.
2024-06-26 21:06:16 -07:00
ShahanaFarooqui
72079cc920 script: Adding Delete action for readme docs
Documents which are deleted from lightning schema were still listed on readme server.

This script was adding or updating the list but delete action was missing.

Changelog-None.
2024-06-25 12:34:07 -07:00
Rusty Russell
c30502779e CI: set elements and bitcoind versions in one place.
We have them spread everywhere: unify them, and make setup.sh use install-bitcoind.sh

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-25 09:36:00 +09:30
Rusty Russell
cb4dd7489c lightningd: check command should return as much detail as possible.
If they're explicitly calling "check", don't obfuscate the result!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-25 08:47:50 +09:30
Rusty Russell
9fd29d35dd common: move now-enlarged command_fail_badparam into its own source file.
It's getting a bit awkward to inline now: it's non-trivial.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-25 08:47:50 +09:30
Rusty Russell
f33c5188ef common: don't mention the contents of invalid parameters when in non-dev mode.
Shahana decided this was the optimal UX path, though I insisted that we still
report the actual problem directly when in dev mode, as a compromise.

Suggested-by: https://github.com/Amperstrand
Changelog-Changed: JSON-RPC: Do not return the contents of invalid parameters in error messages, refer to logs (use 'check' to get full error messages)
Fixes: https://github.com/ElementsProject/lightning/issues/7338
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-25 08:47:50 +09:30
Rusty Russell
e0e38c2cd5 common: new function command_log to log something about a specific command.
Needs implementations for lightningd and libplugin, since they both use this
infrastructure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-25 08:47:50 +09:30
Rusty Russell
3531414d18 lightningd: rename command_log() to command_logger()
It doesn't actually log, just gets the `struct logger`, so this name is better.

We're also going to implement command_log() as an actual logging
function in next commit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-25 08:47:50 +09:30
Rusty Russell
8288603bab pytest: fix flake in test_emergencyrecover
```
2024-06-24T05:09:32.5233603Z         assert l1.rpc.listfunds()["channels"][0]["state"] == "ONCHAIN"
2024-06-24T05:09:32.5234187Z >       assert l2.rpc.listfunds()["channels"][0]["state"] == "ONCHAIN"
2024-06-24T05:09:32.5234917Z E       AssertionError: assert 'FUNDING_SPEND_SEEN' == 'ONCHAIN'
2024-06-24T05:09:32.5235464Z E         - ONCHAIN
2024-06-24T05:09:32.5235773Z E         + FUNDING_SPEND_SEEN
2024-06-24T05:09:32.5236096Z 
2024-06-24T05:09:32.5236242Z tests/test_misc.py:2907: AssertionError
```

It's possible that l2 hasn't seen the onchain tx yet.  What we really want to do is
wait for it, then test that l1 really can spend the output it has recovered.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
f642e4e551 pytest: fix test_feerate_stress race
```
2024-06-24T05:14:14.9939033Z >       l1.rpc.call('dev-feerate', [l2.info['id'], rate - 5])
2024-06-24T05:14:14.9939354Z 
2024-06-24T05:14:14.9939466Z tests/test_connection.py:3439: 
...
2024-06-24T05:14:14.9967617Z >           raise RpcError(method, payload, resp['error'])
2024-06-24T05:14:14.9968833Z E           pyln.client.lightning.RpcError: RPC call failed: method: dev-feerate, payload: ['022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 2290], error: {'code': -1, 'message': 'Peer bad state'}
```

The disconnect can actually take a while: wait for both sides to
believe they're reconnected.  Also wait a little to make sure the
feerate change doesn't create a bad signature.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
bb64fc8ddc pytest: fix flake in test_feerate_stress.
If randomly l1done or l2done are zero, we will create invalid hex:

```
with pytest.raises(RpcError, match='WIRE_INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS|WIRE_TEMPORARY_CHANNEL_FAILURE'):
>           l2.rpc.waitsendpay("{:064x}".format(l2done - 1), timeout=TIMEOUT)

tests/test_connection.py:3438: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
contrib/pyln-client/pyln/client/lightning.py:1361: in waitsendpay
    return self.call("waitsendpay", payload)
contrib/pyln-testing/pyln/testing/utils.py:715: in call
    schemas[0].validate(testpayload)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = Validator(schema={'pairedWith': [['partid', 'groupid']], 'properties': {'groupid': {'description': ['Grouping key...pa... {'description': ['A timeout in... the payment.'], 'type': 'u32'}}, 'required': ['payment_hash']}, format_checker=None)
args = ({'payment_hash': '-000000000000000000000000000000000000000000000000000000000000001', 'timeout': 180},)
kwargs = {}
error = <ValidationError: "'-000000000000000000000000000000000000000000000000000000000000001' is not of type 'hash'">

    def validate(self, *args, **kwargs):
        for error in self.iter_errors(*args, **kwargs):
>           raise error
E           jsonschema.exceptions.ValidationError: '-000000000000000000000000000000000000000000000000000000000000001' is not of type 'hash'
E           
E           Failed validating 'type' in schema['properties']['payment_hash']:
E               {'description': ['The hash of the *payment_preimage*.'], 'type': 'hash'}
E           
E           On instance['payment_hash']:
E               '-000000000000000000000000000000000000000000000000000000000000001'
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
fd9975c288 reckless: don't use time-based directories.
This always fails on my test machine, which runs many tests in parallel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
b748c1845c pytest: fix flake in test_closing.py::test_penalty_htlc_tx_timeout
lightningd-3 penalizes lightningd-2 but then it can see the preimage for the HTLC which
has already been timed out:

```
2024-06-24T02:41:29.4633900Z lightningd-3 2024-06-24T02:33:54.073Z **BROKEN** 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-onchaind-chan#1: HTLC already resolved by THEIR_HTLC_TIMEOUT_TO_THEM when we found preimage
```

This is fair: the test deliberately takes l3 offline for long enough
that the HTLC can get timed out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
80cde517c9 lightningd: fix race in initialization.
Found by very slow CI: the two io_breaks() can combine into one, so we
block waiting for the second initialization which never happens.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
9da66100c1 chaintopology: free outstanding requests so we don't get responses while shutting down.
I've never seen this, but the race looks possible so we should close it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
928b7c83fc lightningd: pass explicit context for bitcoind request functions.
That way if you free the context, you free the call (i.e. the callback
will not be called).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
492ed84dbf lightningd: move checkchain_timer out of bitcoind, remove stopped flag.
checkchain_timer is run by chaintopology, so why have the pointer in
bitcoind?

And since we free the timers, we don't need them to self-disable by
checking the stopped flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
f2b033d7e1 lightningd: move all chainheight logic into setup_topology.
Handling half in main() and half here was a mess.  And the name
"max_blockheight" was poor: it was the max in the db, or UINT32_MAX,
but then we changed it depending on what height we wanted to start at.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
ff15b943fe lightningd: remove unused "min_blockheight" from chain_topology struct.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
f643fb88f5 lightningd: wait for bitcoind at startup if it's fallen behind.
Fixes: https://github.com/ElementsProject/lightning/issues/6924
Changelog-Changed: lightningd: we wait for bitcoind if it has somehow gone backwards (as long as header height is still ok).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
59afbac7cf lightningd: remove first_call param for getchaininfo.
It was a weird arbitrary bool passed through from the caller, which we no longer need.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
eb70476fb3 lightningd: split chaintopology init from normal loops.
The current code is confusing: there are polling loops, but we wait for
them to run once. 

Be explicit: make the calls once, then start the loops in begin_topology.

This removes various chain_topology struct members which only exist for
startup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
f318d08e94 lightningd: add arg to bitcoind_estimate_fees callback.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
Rusty Russell
436bf6c9e1 lightningd: pass height back from getrawblockbyheight.
Several callers stash this, but we have it, so make it explicit:
here's the block you asked for.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 17:26:11 +09:30
ShahanaFarooqui
b803609247 docs: Removed example usage
Example usage has been either merged with examples or usage keys. This heading was not required separately.

This PR edits & rearranges Examples, Example Notifications and Example Usage data.

Changelog-None.
2024-06-22 17:02:33 +02:00
ShahanaFarooqui
e94b793f05 docs: Example notifications heading
- Added missing Example Notifications Heading
- Renamed `example_json_notifications` to `example_notifications`
- Moved `example_notifications` to the end of the page
2024-06-22 17:02:33 +02:00
ShahanaFarooqui
976c0c90bc docs: Changes examples format
- Changed ALL `doc/schemas/lightning-*.json` file's `json_example` to `examples`
- Change the heading from example to examples
- Bring shell command before the json command
- Move Example to the end of the page
- Remove horizontal line from Example
2024-06-22 17:02:33 +02:00
Christian Decker
1347e86671 wallet: Log to chan->log rather than wallet->log
Suggested-by: Rusty Russell <@rustyrussell>
2024-06-22 14:50:21 +02:00
Christian Decker
001b190c61 feat(main): Print channel status when loading it from DB
I was trying to debug a node with several multiplexed channels, and
was finding it a bit difficult to determine which channel index
matches which result of `listpeerchannels` as well as figuring out
what their status was. This just prints the status in string format
when loading the channel from the DB.

Changelog-Changed: wallet: The channel status is printed when loading it from the DB
2024-06-22 14:50:21 +02:00
Rusty Russell
e21b70cad0 pytest: fix default/description order in zeroconf test plugin.
Noticed as I was debugging.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 17:20:06 +09:30
Christian Decker
31cf9225f4 pyln: Turn the plugin options into a real Type (uppercase T)
[ Fix not to include 'value' and 'default' (if None) in getmanifest response --RR ]
[ Fix to support [] operator for existing plugins (including our test ones!) --RR ]
2024-06-20 17:20:06 +09:30
Rusty Russell
1eb1db0e24 pyln-client: allow dynamic option setter to throw exceptions
And don't set the value unless it passes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 17:20:06 +09:30
Christian Decker
457d25c1ce pyln: Drive-by type fixed 2024-06-20 17:20:06 +09:30
Christian Decker
2737f2e97e pyln: Add and test a callback based setconfig listener
I was looking into using the `threading.Condition` but since we're
already rather heavily using callbacks, this allows us to stay
single-threaded, and not having to completely hook the `setconfig`
function.

Changelog-Added: pyln-client: Added a notification mechanism for config changes
2024-06-20 17:20:06 +09:30
Rusty Russell
572ccac0e5 pyln-client: allow dynamic=True add_option to actually allow updates.
We didn't actually *change* the value you'd see, when we got a setconfig call!

Changelog-Added: pyln-client: implement setconfig hook for plugins so you can see changes in `dynamic` options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 17:20:06 +09:30