Commit graph

943 commits

Author SHA1 Message Date
Rusty Russell
ea928bfca1 pytest: use modern listconfigs.
Use the configs object, as the others are about to be deprecated.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
0df97547dd lightningd: don't simply ignore defaults on flags, deprecate.
Changelog-Deprecated: Plugins: `default` no longer accepted on `flag` type parameters (it was silently ignored, so just don't set it).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
d348554ff4 pyln-client: don't try to convert objects to Millisatoshi.
The new listconfigs fields will be objects, and this messes us up!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
45e16180bc lightningd: remove deprecated null for missing plugin options.
We leave the code in contrib/pyln-client/pyln/client/lightning.py to handle
msat null fields for now, though, for a bit more compatibility.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
56eb24280c pyln-testing: allow protobuf v4
I need to update gprcio-tools, but it needs protobuf v4.  Christian
added this restriction in a99509db36 to
"Update protobuf dependency to silence dependabot", but perhaps it's
time to actually update?

```
$ poetry update
Updating dependencies
Resolving dependencies... (1.0s)

Because pyln-testing (23.05rc2) @ file:///home/rusty/devel/cvs/lightning/contrib/pyln-testing depends on protobuf (>=3.20.3,<4)
 and grpcio-tools (1.54.0) depends on protobuf (>=4.21.6,<5.0dev), pyln-testing (23.05rc2) @ file:///home/rusty/devel/cvs/lightning/contrib/pyln-testing is incompatible with grpcio-tools (1.54.0).
So, because cln-meta-project depends on both grpcio-tools (1.54.0) and pyln-testing (23.05rc2) @ file:///home/rusty/devel/cvs/lightning/contrib/pyln-testing, version solving failed.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 06:40:47 +09:30
Rusty Russell
a7786ca742 Revert "Put bitcoind logging in stdout for pytest"
This reverts commit 1037bf3c43.
(It broke elementsd: we no longer see "Done loading")
2023-05-29 18:41:46 +09:30
Christian Decker
c029de7d56 msggen: Add preapproveinvoice and preapprovekeysend to msggen 2023-05-29 14:50:13 +09:30
Greg Sanders
1037bf3c43 Put bitcoind logging in stdout for pytest 2023-05-25 11:07:45 +09:30
Jon Griffiths
8b4136bad5 build: remove gmp
Core and secp have not used gmp for a very long time now (core disabled
it in 2015).

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2023-05-24 18:21:43 +09:30
Rusty Russell
e7d4c3175a build: remove --enable-experimental-features / EXPERIMENTAL_FEATURES
Changelog-EXPERIMENTAL: Build: all experimental features are now runtime-enabled; no more ./configure --enable-experimental-features
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-05-23 09:34:08 +09:30
Rusty Russell
b8aa3a579e listtransactions: remove annotations, which were only in EXPERIMENTAL_FEATURES.
I don't know if anyone was using them, they seem half-hearted.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-05-23 09:34:08 +09:30
Shahana Farooqui
d1cf88c62e meta: update changelog and pyln version for 23.05 release
Changelog-None
2023-05-10 09:50:38 -07:00
Shahana Farooqui
4f258a9354 meta: Add changelog for 23.05rc4 2023-05-09 14:04:14 -07:00
Matt Morehouse
fe5f3cef51 pyln: remove unused variable
Variable inroduced in 708fb17 is making CI fail.
2023-05-09 13:15:36 -07:00
Shahana Farooqui
ea23122880 meta: Add changelog for 23.05rc3 2023-05-05 10:20:27 -07:00
Christian Decker
708fb17fa2 pytest: Add helper to get a grpc stub and test decode 2023-05-05 09:24:07 -07:00
Christian Decker
acc3bb2276 msggen: Switch signatures to string instead of bytes 2023-05-05 09:24:07 -07:00
Christian Decker
ef9e0fcf60 pyln: Set the correct envvar for logging for rust plugins 2023-05-05 09:24:07 -07:00
Christian Decker
fc26675336 msggen: Add DecodePay to the mappings
Changelog-Added: grpc: Added mapping for `listpeerchannels`, `listclosedchannels`, `decode` and `decodepay` RPC method
2023-05-05 11:54:41 +09:30
Christian Decker
0031f1160b msggen: Map arrays of hashes and add HtlcState enum 2023-05-05 11:54:41 +09:30
Christian Decker
db843159ea msggen: Move overrides into the model itself
We were using per-type overrides which caused some asymmetries, where
conversions could end up dropping fields as we went along. Essentially
each conversion would need to override a superset of the previous one,
which then caused issues when attempting to close the loop. By
overriding on the model level we ensure that all representations are
equivalent and convertible into one another, at the expense of
overriding a bit more aggressively, which should be fine anyway.
2023-05-05 11:54:41 +09:30
Christian Decker
b41cb2f005 cln-grpc: Rename the ChannelSide variants
IN/OUT => LOCAL/REMOTE
2023-05-05 11:54:41 +09:30
Christian Decker
bff3b1ca8c msggen: Add ListClosedChannels and overrides 2023-05-05 11:54:41 +09:30
Christian Decker
d28815f7b8 msggen: Disable grpc response -> json response temporarily
We use overrides that omit fields in some cases, which makes the
conversion lossy. This also means that until we complete the mapping
we can't reconvert back.
2023-05-05 11:54:41 +09:30
Christian Decker
e7a96cac11 msggen: Normalize enum and field names if they contain a '/' 2023-05-05 11:54:41 +09:30
Christian Decker
65f5134643 msggen: Add ListPeerChannels to generated interfaces 2023-05-05 11:54:41 +09:30
Christian Decker
2a52e52015 jsonrpc: Add versioning annotation to listpeerchannels 2023-05-05 11:54:41 +09:30
Shahana
21cc16fb5b meta: Add changelog for 23.05rc2 2023-04-26 15:46:02 -07:00
Shahana Farooqui
15795c969a meta: Add changelog for 23.05rc1
Typo correction

Updated delpay changelog
2023-04-15 12:33:06 -07:00
Christian Decker
2e5ad0f417 pyln: Exclude all cln- plugins from valgrind 2023-04-13 18:13:28 -07:00
Joel Klabo
c0d3eeb789 Fix Typo in startup_regtest.sh 2023-04-11 13:39:51 +09:30
Rusty Russell
89b96e8ac0 pyln-testing: add support to tell bitcoind not to include txs if fee is too low.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 17:26:47 +09:30
Rusty Russell
cf80f0520a connectd: dev-report-fds to do file descriptor audit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 09:41:56 +09:30
Rusty Russell
a000ee015a lightningd: do RBF again for all the txs.
Now we've set everything up, the replacement code is quite simple.

Some tests now have to deal with RBF though, and our rbf tests need work
since they look for the old onchaind messages.

In particular, when we can't afford the fee we want, we back off to
the next blockcount estimate, rather than spending all on fees
(necessarily).  So test_penalty_rbf_burn no longer applies.

Changelog-Changed: Protocol: spending unilateral close transactions now use dynamic fees based on deadlines (and RBF), instead of fixed fees.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell
3a3370f4c1 feerates: add floor field for the current minimum feerate bitcoind will accept
Changelog-Added: JSON-RPC: `feerates`: added `floor` field for current minimum feerate bitcoind will accept
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell
c46473e615 lightningd: allow "NNblocks" and "minimum" as feerates.
And consolidate descriptions into lightning-feerates().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `close`, `fundchannel`, `fundpsbt`, `multifundchannel`, `multiwithdraw`, `txprepare`, `upgradewallet`, `withdraw` now allow "minimum" and NN"blocks" as `feerate` (`feerange` for `close`).
2023-04-10 07:31:12 +09:30
Rusty Russell
64b1ddd761 lightningd: clean up feerate handling, deprecate old terms.
Drop try_get_feerate() in favor of explicit feerate_for_deadline() and
smoothed_feerate_for_deadline().

This shows us everywhere we deal with old-style feerates by names.

`delayed_to_us` and `htlc_resolution` will be moving to dynamic fees,
so deprecate those.

Note that "penalty" is still used for generating penalty txs for
watchtowers, and "unilateral_close" still used until we get zero-fee
anchors.

Changelog-Added: JSON-RPC: `feerates` `estimates` array shows fee estimates by blockcount from underlying plugin (usually *bcli*).
Changelog-Changed: JSON-RPC: `close`, `fundchannel`, `fundpsbt`, `multifundchannel`, `multiwithdraw`, `txprepare`, `upgradewallet`, `withdraw` `feerate` (`feerange` for `close`) value *slow* is now 100 block-estimate, not half of 100-block estimate.
Changelog-Deprecated: JSON-RPC: `close`, `fundchannel`, `fundpsbt`, `multifundchannel`, `multiwithdraw`, `txprepare`, `upgradewallet`, `withdraw` `feerate` (`feerange` for `close`) expressed as, "delayed_to_us", "htlc_resolution", "max_acceptable" or "min_acceptable".  Use explicit block counts or *slow*/*normal*/*urgent*/*minimum*.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell
cdb85d5618 lightningd: handle fees as blockcount + range.
Rather than have specific-purpose levels, have an array of
[blockcount, feerate], and rebuild the specific-purpose levels
for now on top.

We also keep a *separate* smoothed feerate, so you can ask for that
explicitly.

Since all the plugins used the same formula to derive the different
named fee levels, we apply the reverse to return to the underlying
estimates: updating the interface comes next.

This is ugly for now, but various specific-purpose levels will be
going away, as we shift to deadline-driven fees.

This temporarily breaks the floor calculation, so that test is
disabled.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell
b53cc69cfd msggen: fix incorrect assertion.
Adding a new field with `added` fails:

```
AssertionError: Field Feerates.perkb.estimates[] does not have an `added` annotation
```

Looks like this assertion is wrong: we should get an added from the field itself or
from the .msggen.json file.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell
c5b7dbcd98 pytest: clean up wait_for_onchaind_tx interface, remove wait_for_onchaind_broadcast
Using single tuples in Python is ugly, so:
1. Rename wait_for_onchaind_tx to wait_for_onchaind_txs.
2. Make it take tuples explicitly.
3. Make wait_for_onchaind_tx a simpler wrapper/unwrapper.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell
3e83bed460 pyln-testing: adapt wait_for_onchaind_broadcast function for when onchaind uses lightningd for broadcast.
We can no longer grab the tx in one line as we did with
wait_for_onchaind_broadcast, we need to track the broadcast from
lightningd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Christian Decker
168bc54700 msggen: Add VersioningCheck
This is a visitor that ensures every new field has at least an `added`
field, and that we don't change the `added` or `deprecated` annotation
after the fact.
2023-04-06 10:16:19 +09:30
Christian Decker
60b12ec096 msggen: Use the inferred optional field
Changelog-Changed: msggen: The generated interfaces `cln-rpc` anc `cln-grpc` can now work with a range of versions rather than having to match the CLN version
2023-04-06 10:16:19 +09:30
Christian Decker
392cacac81 msggen: Add an optional patch
This patch annotates the fields with a new `optional` attribute which
determines whether the field should be considered an inferred optional
due to being added or deprecated.
2023-04-06 10:16:19 +09:30
Christian Decker
5df469cfca msggen: Add patching system, add added and deprecated to Field
The patching system allows us to enrich the raw schema with some
additional information. In this specific case we want to backfill the
`added` and `deprecated` fields for the multiversion support.
2023-04-06 10:16:19 +09:30
Rusty Russell
13ae1a5168 pyln.testing: remove Throttler.
CI seems to block; Christian suggests the throttler may be to blame somehow?

Since trying to fix it made it worse, let's just remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-06 09:10:53 +09:30
Chris Guida
1507e87197 fix helloworld.py example in README for pyln-client 2023-04-05 15:28:32 +09:30
Michael Schmoock
04ea37d88f pygossmap: rename GossipStoreHeader to GossipStoreMsgHeader
Changelog-Added: pyln-client: Improvements on the gossmap implementation
2023-04-05 06:13:08 +09:30
Michael Schmoock
6e46a63c57 pygossmap: adds statistic and filter module
Includes a lot of useful filters and statistical methods.

To see a gossip_store summary:
```
s = GossmapStats(g)
s.print_stats()
```
2023-04-05 06:13:08 +09:30
Michael Schmoock
f1b6047d69 pygossmap: store features for nodes and channels
also makes them acessible using bitmask functions
2023-04-05 06:13:08 +09:30