Commit Graph

795 Commits

Author SHA1 Message Date
Christian Decker
8ae0af7962 pyln: Bump pyln-client dependency in pyln-testing
They were contradicting each other.
2022-09-22 11:41:11 +02:00
Rusty Russell
7c2c100145 pyln: results of make update-pyln-versions NEW_VERSION=0.12.0
But I had to make the pyln-proto requirement of pyln-client less
strict.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 11:41:11 +02:00
Rusty Russell
3c80b22d6f pyln: use poetry version, add target to check version, use poetry publish.
Remove non-working test-release target.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 11:41:11 +02:00
Christian Decker
b9a7f36ab3 msggen: Add conversion from cln-rpc to cln-grpc for Option<Outpoint>
We didn't have optional Outpoints as arguments so far, so let's
backfill that.

Changelog-Changed: cln-rpc: The `wrong_funding` argument for `close` was changed from `bytes` to `outpoint`
2022-09-22 10:41:05 +02:00
Rusty Russell
5a1c2447cb pyln-client: use f strings to concatenate JSON ids, handle older integer ids.
We don't really support using pyln-client with older Core Lightning versions,
but this is neater anyway.  I checked: f-strings go back to python 3.6, so
we can use them (I think this may be the first!).

Suggested-by: @MiWCryptAnalytics
Fixes: #5609
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-21 12:15:48 +02:00
Christian Decker
5c1de8029a openingd: Add reserve to fundchannel and multifundchannel
Changelog-Added: JSON-RPC: `fundchannel`, `multifundchannel` and `fundchannel_start` now accept a `reserve` parameter to indicate the absolute reserve to impose on the peer.
2022-09-21 11:25:47 +02:00
Christian Decker
ab95d2718f pyln: Reduce dependency strictness for pyln-testing 2022-09-20 10:33:40 +02:00
Rusty Russell
d2edeff698 contrib/giantnode.py: populate three regtest nodes with many invoices/forwards/payments.
If you don't want to run this (and it's v slow without the coming
optimizations!) you can download a copy from:

	https://ozlabs.org/~rusty/giantnodes.tar.xz

(Unpack in /, then just `start_ln 3`).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-18 16:21:54 +02:00
Rusty Russell
a99a72be9b contrib/startup_regtest.sh: misc fixes and add destroy_ln, print usage.
We weren't consistent with passing through regtest, and we should be.
Also, destroy_ln is useful for getting rid of all dirs.  Finally, use
eatmydata if it's available.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-18 16:21:54 +02:00
niftynei
37fc0d8c6f docker: use pip install + poetry export instead of poetry update
I dunno man, poetry just seems like a mess.
2022-09-18 09:48:50 +09:30
Christian Decker
37c07ddfe5 pyln: Add grpcio and protobuf dependencies to pyln-testing 2022-09-17 13:45:51 +02:00
Rusty Russell
fdc59dc94a contrib/pyln-testing: pass through id correctly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
f1f2c1322d contrib/pyln-client: construct JSON ID correctly.
They can set their name explicitly, but if they don't we extract it from argv[0].

We also set it around callbacks, so it will be expanded by default.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
ed3f700991 lightningd: use string as json req ids when we create them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
bd18fbc488 contrib/pyln-client: allow lightningd to give us non-numeric ids.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Christian Decker
cb3ee0ac2e wallet: Load and value completed_at timestamp from DB 2022-09-14 13:14:10 +02:00
niftynei
c143914ebf bkpr: migration to delete any duplicate lease_fee entries
Clean up for #5557.

If you've got duplicate 'lease_fee' entries, we delete them!
2022-09-14 15:59:37 +09:30
niftynei
7df530d184 builds: cleanup duplicate and unused code, fix spelling
Few extremely minor updates to the ubuntu dockerfile and ci builds
2022-09-10 11:24:31 +09:30
niftynei
34a0d7083a build: use ubuntu 22.04 LTS
Requires us to update to latest lnproto which is now using the most up
to date python-bitcoinlib, as well as updating our python lock files
(which pin the grpcio deps, because of locking problems h/t @cdecker)
2022-09-10 11:24:31 +09:30
Rusty Russell
cde93ab703 doc: document that we can build lowdown, remove from Alpine.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-08 00:36:51 +03:00
Rusty Russell
50056ce918 doc: remove mrkd requirement, add lowdown requirement.
I guessed it's called "lowdown" for everyone?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-08 00:36:51 +03:00
Rusty Russell
c4203e7de6 pyln-client: allow 'msat' fields to be 'null'
This happens with deprecated-apis and listconfigs, breaking some
python plugins!

Fixes: #5546
Fixes: #5563
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-07 11:01:36 +09:30
Rusty Russell
9543204b79 pytest: don't use bogus scids for first hop of route.
This was a legacy from when it was redundant: with multiple channels, it
no longer is!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-09 16:33:24 -05:00
niftynei
071b1bc4f1 pyln: update versions to v0.12.0 2022-08-02 10:00:26 -07:00
niftynei
4e503f7d0a bkpr/listpeeers: add lease_fees back to funds; separate out in listpeers
First, how we record "our_funds" and then apply pushes vs lease_fees
(for liquidity ad buys/sales) was exactly opposite.

For pushes we were reporting the total funded into the channel, with the
push representing how much we'd later moved to the peer.

For lease_fees we were rerporting the total in the channel, with the
push representing how much was already moved to the peer.

We fix this (from a view perspective) by re-adding lease fees to what's
reported in the channel funding totals. Since this is now new behavior
(for leased channel values), we added new fields so we can take the old
field names thru a deprecation cycle.

We also make it possible to differentiate btw a push and a lease_fee
(before they were all the same), by adding to new fields to `listpeers`:
`fee_paid_msat` and `fee_rcvd_msat`.

This allows us to avoid math in the bookkeeper, instead we just pick
the numbers out directly and record them.

Fixes #5472

Changelog-Added: JSON-RPC: `listpeers` now has a few new fields for `funding` (`remote_funds_msat`, `local_funds_msat`, `fee_paid_msat`, `fee_rcvd_msat`).
Changelog-Deprecated: JSON-RPC: `listpeers`.`funded` fields `local_msat` and `remote_msat` are now deprecated.
2022-07-31 21:53:05 +09:30
niftynei
3c79a456c0 test-db-provider: if postgres in tests, startup a bookkeeper db
FXIME: Has a edge case where if you disable the bookkeeper, it'll
blowup because you've got an option that isn't present anywhere...
2022-07-28 12:08:18 +09:30
niftynei
5c45939acf test-utils: add the bolt11 invoice
nice to have for making tests that assert on description/bolt data
2022-07-28 12:08:18 +09:30
Christian Decker
b55df5c626 msggen: Use tempfile + rename to make changes to .msggen.json atomic
This was causing issues when multiple instances of msggen were running
in parallel on CI.

Changelog-None
2022-07-26 09:49:20 -07:00
niftynei
b6bf352503 contrib startup-regtest: turn off deprecated apis, update deprecated
Update an out of date config, turn off deprecated apis by default for
all regtest tests. (Makes sense, this is a devtool)
2022-07-23 16:22:38 +09:30
Rusty Russell
4cada557ba pytest: don't redirect stderr by default.
Some tests need to inspect it, but most don't, and I suspect I'm missing some
error messages due to this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-21 15:37:05 -05:00
Christian Decker
b48ae58b56 repro: Update ubuntu jammy reprobuild 2022-07-21 15:21:12 -05:00
Christian Decker
9c945dbc68 reprobuild: Add Rust compiler to repro build docker images
Changelog-Added: build: Reproducible builds now include rust binaries such as the `cln-grpc` plugin
2022-07-21 15:21:12 -05:00
Christian Decker
76d05483fa pyln-testing: Add listinvoices to grpc shim 2022-07-21 14:19:06 +09:30
Christian Decker
217ce4c03c schema: Add missing mindepth argument to fundchannel
This was likely missed during the zeroconf PR.

Changelog-None
2022-07-21 14:19:06 +09:30
Christian Decker
20b2f0af85 pyln: Ignore generated files when linting 2022-07-21 14:19:06 +09:30
Christian Decker
ed51c164c0 pyln-testing: Add invoice RPC method 2022-07-21 14:19:06 +09:30
Christian Decker
3f79536437 msggen: Ignore state_changes in grpc2py
It's being skipped in grpc so we don't have it later anyway.
2022-07-21 14:19:06 +09:30
Christian Decker
b6a4cbbf98 cln-rpc: Add mindepth after rebase on master 2022-07-21 14:19:06 +09:30
Christian Decker
6df0a9281f pyln-testing: Add a couple of methods used in tests
These are the most used methods in tests, so we can start getting our
test coverage up.
2022-07-21 14:19:06 +09:30
Christian Decker
18a9eb2feb msggen: Add stop method to generators
We'll need this when testing the grpc interface in pyln-testing,
otherwise tests just slowly die and wither.
2022-07-21 14:19:06 +09:30
Christian Decker
12275d0bfe cln-grpc: Skip serializing fields when Option<Vec<T>> is empty too
The CLN API is rather strict about the fact that we should skip
providing a field whenever it is empty. Checking for `is_none` would
still include empty arrays.

Changelog-Fixed cln-rpc: Optional empty arrays will no longer be serialized in requests
2022-07-21 14:19:06 +09:30
Christian Decker
77f5eb556b msggen: Add fundchannel request 2022-07-21 14:19:06 +09:30
Christian Decker
b8bcc7d13f pytest: Add a new RPC interface to talk to grpc
This allows us to re-use existing tests (assuming the call and fields
are covered by `cln-rpc` and `cln-grpc`) to test the full roundtrip
from test over the grpc interface to the json-rpc interface and back
again.

You can switch to the grpc interface by setting the `CLN_TEST_GRPC`
environment variable to 1, but for now only very few shims are
implemented (due to the non-generated nature of LightningRpc).
2022-07-21 14:19:06 +09:30
Christian Decker
5307586d4d msggen: Add a new generator for grpc -> python converter
To test the grpc interface we'll want to emulate the JSON-RPC
interface as best we can, hence when talking to the grpc interface we
want to convert back into a parsed JSON format as LightningRpc would
have returned it. This is just the simplest way of closing the loop
here:

```
  pyln-testing --grpc-> cln-grpc --grpc2json
     ^                                  |
     |                                  v
     |                               JSON-RPC
     |                                  |
    TEST                                v
     ^                                 CLN
     |                                  |
     |                                  v
  pyln-testing <-grpc2py-- cln-grpc <- json2grpc
```
2022-07-21 14:19:06 +09:30
Christian Decker
bac322ccdb pytest: Move generated grpc bindings to pyln-testing
These may eventually end up in pyln-client, as they allow talking to
the GRPC interface exposed by cln-grpc, however for now they are used
for testing only. Once we have sufficient API and test coverage we can
move them and leave imports in their place.
2022-07-21 14:19:06 +09:30
Christian Decker
2d35c9a929 msggen: Do not override method names when loading Schema
We were overriding the name right when loading, which is bad since in
some languages we use the method name as tag in the requests, thus
renaming causes us to call something that isn't defined.

Changelog-Fixed: cln-rpc: Fixed a naming mismatch for `ConnectPeer` causing `connectpeer` to be called on the JSON-RPC
2022-07-21 14:19:06 +09:30
William Casarin
98185dfc2b startup_regtest: add connect helper
`connect 1 2` to connect from l1 to l2, etc
2022-07-18 20:37:32 -05:00
William Casarin
a3f5d31b09 startup_regtest: add experimental-offers 2022-07-18 20:37:32 -05:00
Swapnil
6204d70a37 docs: fix contrib/ docs 2022-07-17 21:40:01 +09:30
Simon Vrouwe
cc40243399 pyln-testing: print content of errlog file when _some_ node failed unexpected 2022-07-16 14:19:05 +09:30