Commit graph

547 commits

Author SHA1 Message Date
Alex Myers
a2e458047f reckless: add json output option
Also redirect config creation prompts to stderr in order to not interfere
with json output on stdout.

Changelog-Added: reckless provides json output with option flag -j/--json
2024-08-08 11:33:39 +09:30
Alex Myers
75d8d8b91f reckless: add logger class
This will allow redirection of json output in the following commits.
2024-08-08 11:33:39 +09:30
Alex Myers
40c24065f7 reckless: make options flags position independent
Changelog-Changed: Reckless option flags are now position independent.
2024-08-08 11:33:39 +09:30
Alex Myers
0a6c58133d reckless: add version information to reckless 2024-08-08 11:33:39 +09:30
Alex Myers
9cbaafb525 reckless: add installer for rust plugins
This follows the same structure that enables python virtual environments:
  reckless/
    <plugin_name>/
      <symlink to compiled bin>
      source/
        <clone of original source plugin dir>/

Changelog-Added: Reckless: added the ability to install rust plugins.
2024-08-08 11:33:39 +09:30
Alex Myers
ed2c0ae18f reckless: place source code in source/plugin_name/
rather than just source/.  This is required for cargo install, so let's just
use this paradigm globally.
2024-08-08 11:33:39 +09:30
Alex Myers
c1d46d6ef9 reckless: add type hints for InstInfo 2024-08-08 11:33:39 +09:30
Alex Myers
7387c9cb1f reckless: remove mimetype
The python mimetype package wasn't useful enough in indentifying
filetypes anyhow.
2024-08-08 11:33:39 +09:30
Rusty Russell
a1fdeee76b Makefile: clean up install path handling.
We had some weird code to try to do relative paths.  Instead, use absolute
ones and keep life simple.  Also rename "daemon_dir" to the clearer
"subdaemon_dir" as that's what it's used for.

We now need special magic to do "installcheck", which is a bit awkward,
but at least it's contained.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 18:32:56 +02:00
Rusty Russell
531845971c gossipd: drop zlib support.
This was removed from the spec on Apr 25, 2022.  We stopped ever sending them
in 0.12.0 (2022-08-23).  Now we remove receive support.

Changelog-Protocol: Removed support for zlib-compressed short-channel-ids in query responses.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-10 12:21:19 +09:30
Rusty Russell
fd9975c288 reckless: don't use time-based directories.
This always fails on my test machine, which runs many tests in parallel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
ShahanaFarooqui
b803609247 docs: Removed example usage
Example usage has been either merged with examples or usage keys. This heading was not required separately.

This PR edits & rearranges Examples, Example Notifications and Example Usage data.

Changelog-None.
2024-06-22 17:02:33 +02:00
ShahanaFarooqui
e94b793f05 docs: Example notifications heading
- Added missing Example Notifications Heading
- Renamed `example_json_notifications` to `example_notifications`
- Moved `example_notifications` to the end of the page
2024-06-22 17:02:33 +02:00
ShahanaFarooqui
976c0c90bc docs: Changes examples format
- Changed ALL `doc/schemas/lightning-*.json` file's `json_example` to `examples`
- Change the heading from example to examples
- Bring shell command before the json command
- Move Example to the end of the page
- Remove horizontal line from Example
2024-06-22 17:02:33 +02:00
ShahanaFarooqui
44de5023cb docs: Updated scripts for rpc examples
This update will add lightning-cli examples and make examples heading collapsible.

- The JSON EXAMPLES heading is collapsible now

- Examples in lightning-cli format also

- manpage examples have better formatting

Changelog-None.
2024-06-19 15:56:22 +09:30
Rusty Russell
4a36edf7c1 tools/check-bolt: rewrite to give more accurate diagnostics when we're out of sync.
Just use substrings not regexes, since we have more complex characters now LaTeX
is entering the spec!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 15:54:24 +09:30
Rusty Russell
1de569d38d devtools/gossipwith: use timestamp filter message not obsolete INIT_ROUTING_SYNC.
This means we do have to set the network correctly though,
and also we can get query messages from lightningd which we have to filter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 15:54:24 +09:30
ShahanaFarooqui
c9c1df6710 script: Added caching option for docker publish and shellcheck fixes
Changelog-None.
2024-06-13 11:43:37 +09:30
ShahanaFarooqui
3dc6e6d29e script: Add reproducible build script for Ubuntu v24.04
This PR will help in publishing CLN reproducible binaries for Ubuntu v24.04 (noble).

Please note that I adjusted Dockerfiles for focal and jammy also to keep the base image creation script same for all three images. The step update was required because `noble` only runs with ubuntu:noble setup.

Changelog-None.
2024-06-03 14:23:36 -05:00
ShahanaFarooqui
7f5282aa65 docker: Add missing git dependency for Fedora Dockerfile
Fedora image building by build-release.sh currently throws error ` git: command not found`.

Fixed it by adding git in the Dockerfile and loading the image in the script.

Changelog-None.
2024-05-24 15:07:39 -05:00
nicolas.dorier
369c0c1391 Update Dockerfile and build-release.sh to PR7214 2024-05-23 10:30:15 -05:00
nicolas.dorier
738ddc66cc Use cross compile in dockerfile, fix arm32 support 2024-05-23 10:30:15 -05:00
daywalker90
c98185d944 tools: allow spelling of HtlcTlv 2024-05-17 16:30:43 +02:00
ShahanaFarooqui
9e782f491b doc: updated schemas for json_examples 2024-04-04 13:16:27 +10:30
bstin
6edbc635cc Update hsmtool.c
This line is a cosmetic off-by-one error. There are 7 languages defined, but because counting starts at 0, the display is "Select [0-7]" when in fact only 0-6 are valid choices.
2024-03-20 14:29:47 +10:30
Rusty Russell
9450d46db1 bitcoin/short_channel_id: pass by copy everywhere.
It's a u64, we should pass by copy.  This is a big sweeping change,
but mainly mechanical (change one, compile, fix breakage, repeat).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-20 13:51:48 +10:30
Rusty Russell
e0e879c003 common: remove type_to_string files altogther.
This means including <common/utils.h> where it was indirectly included.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-20 13:51:48 +10:30
Rusty Russell
37d22f9141 global: change all type_to_string to fmt_X.
This has the benefit of being shorter, as well as more reliable (you
will get a link error if we can't print it, not a runtime one!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-20 13:51:48 +10:30
ShahanaFarooqui
f72b0fdabd docs: Updates script and fixed test to generate lightning-sql
- Updated doc/Makefile for generating schemas/lightning-sql.json
    - Read the lightning-sql template from schemas/lightning-sql-template.json
    - Generate sql tables data and merge it with json object read from above template
    - Save final merged object in schemas/lightning-sql.json
- Updated doc/Makefile to auto generate lightning-sql.7.md and lightning-sql.7
- Deleted schemas/lightning-sql.json and adding it into .gitignore
- Added lightning-sql specific titles in the fromschema.py script
- Fixed test_sql by changing the sequence for listpeerchannels `reestablished` field
- Ignoring lightning-sql.json for msggen schema.json because it is auto generated by sql plugin and lightning-sql-template.json.
2024-03-19 14:58:59 +10:30
ShahanaFarooqui
418ecf3e42 script: Update scripts
- Update `make doc-all` script
- `fromschema.py` script
    - Updated to generate whole doc/lightning-*.md via doc/schemas/lightning-*.json file
    - Updated to print request params
    - Added `oneOfMany` condition for request params
    - Added `pairedWith` condition for request params
    - Added `dependentUpon` condition for request params
    - Updated for pre and post_return_value for response
    - Hiding `hidden` fields
    - `descriptions` are array now
    - Unified gaps between titles
    - Added default key-value pair

- script: msggen, sql-schema, listconfig and pyln-testing script updates

This does not add created markdowns for cleaner review. We will add the markdown files in a separate commit.
2024-03-19 14:58:59 +10:30
Rusty Russell
fdfffdc232 common: add routing test using real data which crashes.
The amount is set not to crash by default, but run
"common/test/run-route-infloop 8388607" and you'll see a crash.

Sorry about the 7MB blob, but this testing was quite revealing and
I consider it worth adding.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-07 14:09:14 +01:00
Christian Decker
8418989f9b release: Unbreak the Fedora build
We have installation instructions that tell the user to use `poetry`
and then we ourselves think we're clever and install only a known
subset? It was only a matter of time until we broke this.

Changelog-None
2024-02-28 14:38:10 +01:00
Alex Myers
5c475067b8 reckless: populate local submodules for searching
Git ls-tree does not report submodule contents which was not previously
accounted for.
2024-02-21 14:33:11 +01:00
Alex Myers
538b4c850e reckless: prefer to search a local copy of a remote source
If it has already been cloned, this is less problematic than
using the github rest API.
2024-02-21 14:33:11 +01:00
Alex Myers
bfb29aaef0 reckless: Clone github sources when API access fails
Due to the API ratelimit, this allows cloning a github repo and searching
the result rather than searching via the REST API.  If a source has
already been cloned, it is fetched and the default branch checked out.

Fixes a failure reported by @farscapian

Changelog-Fixed: Reckless no longer fails on github API ratelimit.
2024-02-21 14:33:11 +01:00
Alex Myers
ba9ec412c7 reckless: set relative path of local git repo directories 2024-02-21 14:33:11 +01:00
Alex Myers
47c81995e3 reckless: initialize repos with submodules and other cleanup 2024-02-21 14:33:11 +01:00
Alex Myers
61bbd70bc7 reckless: reduce github API calls
Github API calls are now ratelimited to 60 per hour.  Searching through
the subdirectory contents of lightningd/plugins will hit this limit after
two searches or installs.  The simple answer is to look for the directory
rather than verifying a valid entrypoint in a suitably-named directory
prior to cloning the repository.

Also corrects a bug that was flagging submodules as files while
populating directory contents.
2024-02-21 14:33:11 +01:00
Alex Myers
3879d7b9bd reckless: update timeouts 2024-02-07 13:24:34 +01:00
Alex Myers
90a20ff6e3 reckless: create virtual environments for python plugins
This uses pip to install to a venv when the dependencies are specified
with requirements.txt and poetry when it's present on the system and the
plugin has a pyproject.toml.

The directory structure will be:
reckless/
    source/                (original plugin code here)
        plugin_entrypoint  (original entrypoint)
    plugin_name            (symlink or wrapper to activate venv)
    .metadata              (installation information)
    .venv/                 (python virtual environment)

The wrapper matches the naming of the original plugin entrypoint. The
shebang is modified to use the virtual environment's python binary,
then the original plugin is imported as a module and executed as though
it was run directly.

Changelog-Changed: reckless installs python plugins in virtual environments
2024-02-07 13:24:34 +01:00
Alex Myers
03e78ce6d9 reckless: add option to check out a specific commit
... using the syntax reckless install <plugin_name>@<commit hash>
2024-02-07 13:24:34 +01:00
Alex Myers
d5ed7b7b5c reckless: add installation metadata
The metadata includes an original retrieval source, timestamp, and commit
hash.  This will allow a future update command to more easily evaluate the
status of the existing installation.
2024-02-07 13:24:34 +01:00
Alex Myers
2b502ebcbc reckless: add staging directory and symlink
This creates a separate staging directory from the one used to clone
the source.  A symlink is then added to the plugin's entrypoint which is
now in the source directory.
2024-02-07 13:24:34 +01:00
Alex Myers
c043bf2255 reckless: properly remove entry from reckless sources
Fixes a bug introduced in 6163138420 which
avoided gratuitous rewrites of the lightningd config
2024-02-07 13:24:34 +01:00
Rusty Russell
7f5fe52320 gossipd: remove online gossip_store compaction.
It was an obscure dev command, as it never worked reliably.
It would be much easier to re-implement once this is done.

This turned out to reveal a tiny leak on
tests/test_gossip.py::test_gossip_store_load_amount_truncated where we
didn't immedately free chan_ann if it was dangling.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-04 09:24:44 +10:30
Lagrang3
28c28e4ea9 add --show-secrets option for hsmtool dumponchain 2024-01-31 13:02:51 +01:00
Lagrang3
a1c02e8098 hsmtool: add feature to dump private keys 2024-01-31 13:02:51 +01:00
Rusty Russell
dadbdf488c schemas: deprecated is now a range.
Don't assume removal is +6 months, but have a start deprecation/end support range.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Christian Decker
44b7fe7cc6 meta: Fix the tools/build-release.sh script to match repro instructions 2023-12-29 00:35:45 +01:00
ShahanaFarooqui
78518344f4 docker: Creating docker latest image
- Create docker multiarch image with/without push
- Create docker latest image and publish if asked

Changelog-None
2023-12-12 16:31:44 +01:00