Commit Graph

1251 Commits

Author SHA1 Message Date
daywalker90
0eb65e4bb5 schemas: use sat and sat_or_all again where appropiate
Changelog-None
2024-05-28 09:49:17 -05:00
ShahanaFarooqui
7f5282aa65 docker: Add missing git dependency for Fedora Dockerfile
Fedora image building by build-release.sh currently throws error ` git: command not found`.

Fixed it by adding git in the Dockerfile and loading the image in the script.

Changelog-None.
2024-05-24 15:07:39 -05:00
Alex Myers
c9f7b78473 meta: update python versions to 24.05
Changelog-None
2024-05-23 18:09:03 -05:00
Se7enZ
e5a1d9331f msggen: Include AddPsbtOutput to RPC list and meta field map; regenerate libraries 2024-05-23 15:55:57 -05:00
ShahanaFarooqui
8a0a9a48f1 doc: listpeerchannel typo and description fix
Suggested by: @evansmj

In Draft PRs:
7155: doc: Clarify listpeerchannels documentation.
7156: doc: Fix two typos in documentation and schema descriptions
2024-05-21 17:43:09 -05:00
daywalker90
5430b65b4c msggen: add checkrune method 2024-05-17 16:30:43 +02:00
daywalker90
b4c061798c msggen: add blacklistrune method 2024-05-17 16:30:43 +02:00
daywalker90
473694c39a msggen: add createrune method 2024-05-17 16:30:43 +02:00
daywalker90
19646f93c6 msggen: add showrunes method
Changelog-None
2024-05-17 16:30:43 +02:00
daywalker90
21446c3401 msggen: add multiwithdraw method
Changelog-None
2024-05-17 16:30:43 +02:00
daywalker90
f83732f990 msggen: add makesecret method
Changelog-None
2024-05-17 16:30:43 +02:00
daywalker90
1dd16f72fb msggen: add listconfigs method
Changelog-None
2024-05-17 16:30:43 +02:00
daywalker90
bd575b51e5 msggen: fix edge cases in rust composite field names 2024-05-17 16:30:43 +02:00
daywalker90
6db56180f9 msggen: allow backfilling of already deprecated fields 2024-05-17 16:30:43 +02:00
daywalker90
c9a9d62367 msggen: add listinvoicerequests, disableinvoicerequest, invoicerequest methods
Changelog-None
2024-05-17 16:30:43 +02:00
daywalker90
6acfca7517 msggen: add routes to decode and decodepay
Changelog-None
2024-05-17 16:30:43 +02:00
daywalker90
c8d88c9130 msggen: fix invoice request missing exposeprivatechannels
Changelog-None
2024-05-17 16:30:43 +02:00
daywalker90
bd1123cf32 msggen: add help method
Changelog-None
2024-05-17 16:30:43 +02:00
daywalker90
5510445bbc msggen: add funderupdate method
Changelog-None
2024-05-17 16:30:43 +02:00
daywalker90
5047bc6dd1 msggen: add emergencyrecover, recover, recoverchannel methods
Changelog-None
2024-05-17 16:30:43 +02:00
daywalker90
16c036eed8 msggen: add dev-forget-channel method
Changelog-None
2024-05-17 16:30:43 +02:00
daywalker90
a9ff3cb039 pytests: use reserve_unused_port() everywhere
Changelog-None
2024-05-16 17:31:02 +02:00
Erik De Smedt
934b85a023 Updated poetry.lock file 2024-05-16 14:00:18 +02:00
Erik De Smedt
151975b282 msggen: Allow non_camel_case for notification
In rust enum are expected to be CamelCase.
However, we are generating enum's using CAPITAL_SNAKE_CASE.

This generates a warning and breaks CI.

See `cln_rpc::notifications::ConnectAddressType::LOCAL_SOCKET`
2024-05-16 14:00:18 +02:00
Erik De Smedt
83cc6c44e0 msggen: Schema for ConnectNotification + collisions
The schema in the docs for the `ConnectNotification` was faulty.
I've already fix this in https://github.com/ElementsProject/lightning/pull/7085

The new schema is

```
{
  "connect" :  {
    "address" : {
      "address" : "127.0.0.1",
      "port" : 38012,
      "type" : "ipv4"
  },
  "direction" : "in",
  "id" : "022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59"
  }
}
```

The `address` in the `connect` field will be encoded in protobuf as
`ConnectAddress`. However, this collides with the `ConnectAddress` that
is defined in the `connect` rpc-method.

This commit
- Updates the schema in `doc/schemas/notification/connect.json`
- Changes `msggen` to include an override to `PeerConnect` for any
  notification typename that starts with `Connect`

Both have an `address` field which is a composite type. This results in
naming collisions

schema's: Updated schema for connect-notification

schema for connect notification + overrides

Override ConnectAddress to `PeerConnectAddress` in protobuf
for notifications
2024-05-16 14:00:18 +02:00
Erik De Smedt
521ac6a12a msggen: Implement notification-server 2024-05-16 14:00:18 +02:00
Erik De Smedt
fa43d3ecfc msggen: Add notifications to proto-file 2024-05-16 14:00:18 +02:00
Erik De Smedt
05b9fc1998 msggen: Add conversions for notification pb to json 2024-05-16 14:00:18 +02:00
Erik De Smedt
686dae2b9f msggen : Add notification-messages to proto-file 2024-05-16 14:00:18 +02:00
Erik De Smedt
c5a47c3149 msggen: RequestStructs for notification-streams
I'm working to expose a stream of notifications over grpc.
This requries me to define structs that can be used to request
a stream of notifications.

These structs are all empty.
2024-05-16 14:00:18 +02:00
Erik De Smedt
eea9c8253b msggen: Choose better when a subfield is optional
**Problem Description**

In previous commits I introduced some new fields to `msggen`.
One example is `CustomMsgResponse` in `cln-grpc/src/notification.rs`.

```rust
pub struct CustomMsgResponse {
    #[serde(skip_serializing_if = "Option::is_none")]
    pub peer_id: Option<PublicKey>,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub payload: Option<String>,
}
```

The `peer_id` and `payload` are required parameters.
However, the generated code is still marking them as `Optional`.

This is a choice made by `msggen`. It does this because `payload` and
`peer_id` are recently added fields. By marking the field as optional
the language bindings would also work when used on an older version of
Core-Lightning.

In this scenario. Marking them as optional is overkill.
The `CustomMsgStruct` and `payload` field are created in the same
version of CoreLightning.

This commit solves this behavior.
2024-05-16 14:00:18 +02:00
Erik De Smedt
6e61dde41e msggen: Provide parent in visit-method of patch
In the next commit I'll change the behavior of `OptionalPatch`.
The changes require me to have access to the `parent` of a field.

Splitting it up in a separate commit makes it easier to review.
You can run `msggen` against this version and the previous `version`.

I've tested it. It returns exactly the same output.
2024-05-16 14:00:18 +02:00
Erik De Smedt
15276b7ff5 msggen: Add notification-structs to cln_rpc
In Core Lightning notifications are JSON-messages. This commit
introduces structs that can be used to parse the notification
messages.

Using `msggen` all required tructs are automatically generated
2024-05-16 14:00:18 +02:00
Erik De Smedt
0251b45027 msggen: Move rpc-model generator 2024-05-16 14:00:18 +02:00
Erik De Smedt
d59029b464 msggen: Cut gprc into pieces!!!
No, this isn't my last resort.

The file to generate `gprc`-bindings got quite long.
I've put it in a separate folder and cut it into smaller pieces
2024-05-16 14:00:18 +02:00
Erik De Smedt
4e93bb6d9d msggen: Add notifications to the Model
Included a `notifications`-field in the `Service` and wrote the code
to parse them.

Add block-added to utils.py
2024-05-16 14:00:18 +02:00
daywalker90
c1c67635fc msggen: add upgradewallet method
Changelog-None
2024-05-15 22:11:10 +02:00
daywalker90
8edd3d4a06 msggen: add setpsbtversion method
Changelog-None
2024-05-15 22:11:10 +02:00
daywalker90
ad31a18b1d msggen: add setconfig method
Changelog-None
2024-05-15 22:11:10 +02:00
daywalker90
5ed04c9788 msggen: add sendonionmessage method
Changelog-None
2024-05-15 22:11:10 +02:00
daywalker90
dab9605e1f msggen: add splice_update method 2024-05-15 12:11:18 +02:00
daywalker90
04a2ad6f8b msggen: add splice_signed method 2024-05-15 12:11:18 +02:00
daywalker90
9f9b59d45b msggen: add unreserveinputs method
Changelog-None
2024-05-15 12:11:18 +02:00
daywalker90
15f0beab6b msggen: add reserveinputs method
Changelog-None
2024-05-15 12:11:18 +02:00
daywalker90
047faf1d34 msggen: add sendinvoice method
Changelog-None
2024-05-13 20:06:40 +02:00
daywalker90
54e1c78e46 msggen: add renepaystatus method
Changelog-None
2024-05-13 20:06:40 +02:00
daywalker90
9538ecccad msggen: add renepay method
Changelog-None
2024-05-13 20:06:40 +02:00
daywalker90
5c0f25f916 msggen: add plugin method
Changelog-None
2024-05-13 20:06:40 +02:00
Rusty Russell
cb2c4963f2 bolt12: allow first_node_id in blinded path to be a scid.
We don't actually support it yet, but this threads through the type change,
puts it in "decode" etc.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-12 19:11:43 -05:00
Rusty Russell
cca784c9d0 BOLT: update to include 2016 for max_htlc_cltv.
And deprecate the --max-locktime-blocks which allows them to set it.

Hilariously, the spec misspells CLTV as CTLV at one point, so we work around it for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-09 16:14:23 -05:00
Ken Sedgwick
e3c600b0c8 logging: add TRACE between DEBUG and IO
By moving super-noisy debugging logging to new TRACE level we can
make long-term logging at DEBUG possible w/o removing any logging.

Addresses ([#7279])

Related Issues:
https://github.com/ElementsProject/lightning/issues/6927
https://github.com/chrisguida/smaug/issues/34
https://github.com/ZmnSCPxj/clboss/issues/194
2024-05-08 21:05:49 -05:00
ShahanaFarooqui
e38c828bf0 docker: Fix jq installation error for focal
`jq` is not available in the default repositories of the Ubuntu Focal image. To resolve this, adding the official jq package repository and then installing it.

Changelog-None.
2024-05-06 22:13:54 -05:00
Rusty Russell
168ecb8979 pyln-client: pass through level parameter on command notifications.
Without this, everything came out as level INFO.

Changelog-Fixed: pyln-client: Fix Plugin.notify_message() not to ignore `level` parameter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-04 11:36:23 -05:00
daywalker90
dc099a046c msggen: add disableoffer method 2024-04-25 14:23:30 +02:00
daywalker90
14fea4b72b msggen: add missing bkpr-* methods
Changelog-None
2024-04-25 13:21:25 +02:00
daywalker90
3189ff6a6d msggen: add getlog method
Changelog-None
2024-04-25 13:21:25 +02:00
ShahanaFarooqui
cf5e8822b9 plugin/clnrest: Adding new config param as clnrest-swagger-root
- Updated config params and plugin
- Updated documentation

Changelog-Added: Added a new configuration for clnrest plugin to change the default Swagger UI path from `/` to custom url.
2024-04-25 13:21:25 +02:00
daywalker90
7317bb174f msggen: add delforward method
Changelog-None
2024-04-24 11:47:22 +02:00
daywalker90
448f4c0cfb msggen: add low level fundchannel methods
Changelog-None
2024-04-23 18:43:03 +02:00
daywalker90
c80a70de25 msggen: add methods autoclean-once and autoclean-status, fix bkpr-listincome
Changelog-None
2024-04-23 17:47:56 +02:00
daywalker90
e418d0c08c msggen: rust fix for methods containing '-' 2024-04-23 17:47:56 +02:00
daywalker90
2d0778ec38 msggen: add delpay method 2024-04-22 13:28:58 +02:00
daywalker90
fe180e189c msggen: add low level openchannel methods
Changelog-None
2024-04-22 09:04:43 +09:30
daywalker90
70faab55e4 msggen: fix for methods that contain '_' in their name 2024-04-22 09:04:43 +09:30
Rusty Russell
a55c0d005d spelling: its listinvoices and listinvoicerequests
Not singular (though we used to have a listinvoice, removed in v0.6.1).

Reported-by: "Plant" via email
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-04-20 10:49:50 -04:00
Rusty Russell
901342b50d pyln-testing: require explicit pattern for BROKEN messages.
Rather than `allow_broken_log`, we have `broken_log` which is a regex
indicating what log lines are expected.  This tightens our tests
significantly, as it will catch *unexpected* brokenness.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-04-20 16:36:57 +09:30
daywalker90
ada4de1588 msggen: fix walk through nested json schemas
Changelog-None
2024-04-18 18:16:30 +02:00
daywalker90
507c2de9c1 newaddr: various fixes for msggen and docs
Changelog-None
2024-04-17 14:47:52 +02:00
daywalker90
9ad5c6bc75 msggen: use field numbers from .msggen.json for rust model if available 2024-04-17 14:47:52 +02:00
daywalker90
58b78d2b32 schema: add multifundchannel generation
Changelog-None
2024-04-17 13:06:02 +02:00
daywalker90
4d306e2691 mssgen: preserve existing camel cases
an edge case like MultifundchannelChannel_idsChannel_type
was previously converted to MultifundchannelChannelIdschannelType
instead of the correct MultifundchannelChannelIdsChannelType
2024-04-17 13:06:02 +02:00
ShahanaFarooqui
aff8f0ff5e doc: Updated dev command warning
Changelog-Changed: Documentation: Merged `example_json_request` and `example_json_response` in a single `json_examples` array to maintain the request and its corresponding response together.
2024-04-04 13:16:27 +10:30
ShahanaFarooqui
9e782f491b doc: updated schemas for json_examples 2024-04-04 13:16:27 +10:30
Rusty Russell
3483bff66b doc: document the rune conditions for parsing invoice strings.
This pre-promises an implementation, which we are now going to write!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-27 16:09:23 +10:30
Rusty Russell
4e086f28c3 lightningd: remove channels from listpeers.
Changelog-Removed: JSON-RPC: `listpeers` `channels` (deprecated v23.02, EOL v24.02)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-25 15:02:35 +10:30
Rusty Russell
ba922f9160 lightningd/connectd: remove --experimental-websocket-port
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Config `experimental-websocket-port` (deprecated 23.08, EOL 24.02)
2024-03-25 15:02:35 +10:30
Rusty Russell
7e0e39460b lightningd: remove delexpiredinvoice
Changelog-Removed: JSON-RPC: `delexpiredinvoice` (deprecated v22.11, EOL v24.02)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-25 15:02:35 +10:30
bitkarrot
99aa519457 doc: Add json schema for dev-forget-channel 2024-03-21 20:12:37 -07:00
Rusty Russell
1321d35855 multifundchannel: fix amount argument in schema & docs (it can be "all").
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-21 19:52:56 +10:30
Rusty Russell
4816550b0f lightningd: check rune parameter names with and without punctuation.
Changelog-Changed: runes: named parameters (e.g. `pnameamountmsat`) no longer need to remove underscores (i.e. `pnameamount_msat` now works as expected).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-20 11:18:36 +10:30
Rusty Russell
7af045df4e pyln-testing: use DEBUG_LIGHTNINGD to start gdb on lightningd during test.
We do this for DEBUG_SUBD already, but I wanted to debug the main lightningd.

(We rename --debugger to the more accurate --dev-debug-self)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-20 11:18:36 +10:30
Rusty Russell
4a9b9b8b29 pay: add partial_msat option to make partial payment.
a.k.a. "Pay with a friend!".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `pay` has a new parameter `partial_msat` to only pay part of an invoice (someone else presumably will pay the rest at the same time!)
Suggested-by: Calle
2024-03-20 10:58:28 +10:30
ShahanaFarooqui
9b83b8b967 doc: Delete markdowns as they will be generated by scripts now
This PR will generate doc/lightning-*.md file for rpc commands. It will get the information from consolidated doc/schemas/lightning-*.json file and use it to generate markdown file for RPC commands.

Changelog-Changed: Documentation: great documentation rewrite, all reference pages now generated from the fully-tested JSON schemas and include examples.
2024-03-19 14:58:59 +10:30
ShahanaFarooqui
f72b0fdabd docs: Updates script and fixed test to generate lightning-sql
- Updated doc/Makefile for generating schemas/lightning-sql.json
    - Read the lightning-sql template from schemas/lightning-sql-template.json
    - Generate sql tables data and merge it with json object read from above template
    - Save final merged object in schemas/lightning-sql.json
- Updated doc/Makefile to auto generate lightning-sql.7.md and lightning-sql.7
- Deleted schemas/lightning-sql.json and adding it into .gitignore
- Added lightning-sql specific titles in the fromschema.py script
- Fixed test_sql by changing the sequence for listpeerchannels `reestablished` field
- Ignoring lightning-sql.json for msggen schema.json because it is auto generated by sql plugin and lightning-sql-template.json.
2024-03-19 14:58:59 +10:30
ShahanaFarooqui
2c99f2e100 doc: Remove unused request.json and schema.json files 2024-03-19 14:58:59 +10:30
ShahanaFarooqui
418ecf3e42 script: Update scripts
- Update `make doc-all` script
- `fromschema.py` script
    - Updated to generate whole doc/lightning-*.md via doc/schemas/lightning-*.json file
    - Updated to print request params
    - Added `oneOfMany` condition for request params
    - Added `pairedWith` condition for request params
    - Added `dependentUpon` condition for request params
    - Updated for pre and post_return_value for response
    - Hiding `hidden` fields
    - `descriptions` are array now
    - Unified gaps between titles
    - Added default key-value pair

- script: msggen, sql-schema, listconfig and pyln-testing script updates

This does not add created markdowns for cleaner review. We will add the markdown files in a separate commit.
2024-03-19 14:58:59 +10:30
ShahanaFarooqui
4c370022d0 script: Updated sql-schema_gen and documented it's jq dependency. 2024-03-19 14:58:59 +10:30
ShahanaFarooqui
04884ca664 doc: Merge request and schema jsons in a single file
Merge information from `*.request.json` & `*.schema.json`. Also consolidate remaining details from `*.md` files and create a single file in schemas folder.
2024-03-19 14:58:59 +10:30
ShahanaFarooqui
fdc2dbdbc3 doc: Updated schemas *.schema.json
- Makes emergency schema stricter
- Fixes some formatting
2024-03-19 14:58:59 +10:30
Rusty Russell
d7cc1e4a17 doc: Updated parameter descriptions for *.request.json
Added descriptions for rpc command parameters

This also performs the following fixes:
1. delforward parameters are compulsory (required).
2. disableinvoicerequest request added `added` field.
3. invoice request order fixed (label then description, not vice-versa!).
4. listpeers log levels are a proper enum
5. description parameter documented for sendonion requests.
6. deprecatred amount_msat removed from sendpay request.
7. sendpay request partid type fixed to u64 (was u16!)
8. sendpay request localinvreqid type tightened to hash (was hex)
9. sendpay request payment_metadata and description fields documented.
10. sendpsbt request reserve type fixed to u32 (was boolean)
11. utxopsbt request satoshi type fixed to msat_or_all (was msat)
12. withdraw request parameter satoshi is compulsory (required)
13. fundchannel_start request amount is sat, not msat_or_all.
14. openchannel_init request amount is sat, not msat
15. openchannel_init close_to is a string, not hex.
16: invoice labels can be strings OR numbers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-19 14:58:59 +10:30
ShahanaFarooqui
d9dd3a1840 doc: Adding missing *.schema.json
Adding missing *.schema.json files for rpc commands to create consolidated json files.
Updating and deleting other as per requirement.
2024-03-19 14:58:59 +10:30
ShahanaFarooqui
e8c8c8887b doc: Adding missing *.request.json
Some rpc commands did not have their corresponding *.request.json files to generate the consolidated json
2024-03-19 14:58:59 +10:30
ShahanaFarooqui
8615a87452 Add type currency for offers amount field. Eg.1.50USD. 2024-03-19 14:58:59 +10:30
Rusty Russell
f100b1cdf1 Add type sat.
We actually use this in multiple places for requests.  The key
difference between this and msat is what we do when presented with a
raw number!  I prefer insisting on explicit suffixes, for this reason,
but at least this will document the field correctly!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-19 14:58:59 +10:30
Rusty Russell
ee00f84fed JSON: don't return a raw result string for "stop" and "recover".
In general, results should always be an object, so we can add new fields
later (e.g. warnings!).  But we violated this for "stop", and when "recover"
used the same infrastructure, it started doing the same thing.

I'm assuming nobody cares, so we don't need to do a deprecation cycle.

Changelog-Changed: JSON-RPC: `stop` and `recover` now return a JSON object (not a raw string!) like every other command does.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-19 14:58:59 +10:30
Christian Decker
8418989f9b release: Unbreak the Fedora build
We have installation instructions that tell the user to use `poetry`
and then we ourselves think we're clever and install only a known
subset? It was only a matter of time until we broke this.

Changelog-None
2024-02-28 14:38:10 +01:00
Christian Decker
5e42f4681b release: Bump pyln package versions 2024-02-28 14:38:10 +01:00
Aditya Sharma
9acc1d7db6 peer_control: Add lost_state param inside listpeerchannels rpc, which'd help us identify if we've fall behind or lost some state. 2024-02-16 22:17:46 +01:00
jrakibi
1fdc018b74 Fix incorrect hex value for Signet port 2024-02-16 15:54:29 +01:00
Rusty Russell
6ed17e9e0c pyln-testing: dump gossip store when we complain about bad gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-12 11:43:33 +01:00