Commit Graph

567 Commits

Author SHA1 Message Date
Matt Whitlock
67d667db90 tools/headerversions.c: fix build without SQLite
Commit 531845971c broke the build without
SQLite because this code:

	new = tal_fmt(NULL, template,
	              IF_SQLITE3(sqlite3_libversion_number()));

preprocesses into:

	new = tal_fmt(NULL, template,
	              );

which has a syntax error. Fix it by moving the comma into the macro
argument.

Fixes: 531845971c
Changelog-None
2024-11-19 09:48:31 +10:30
Rusty Russell
38d229df3d tools/generate-wire.py: make sure TLV array fields are allocated off TLV.
Otherwise the whole thing cannot be tal_steal() onto a different parent.

Here's the difference in generated files:

   --- ./wire/onion_wiregen.c.pre	2024-10-23 12:26:09.023176933 +1030
   +++ ./wire/onion_wiregen.c	2024-10-23 12:26:52.434828303 +1030
   @@ -128,7 +128,7 @@
     	blinded_path->path = num_hops ? tal_arr(blinded_path, struct blinded_path_hop *, 0) : NULL;
    	for (size_t i = 0; i < num_hops; i++) {
    		struct blinded_path_hop * tmp;
   -		tmp = fromwire_blinded_path_hop(blinded_path, cursor, plen);
   +		tmp = fromwire_blinded_path_hop(blinded_path->path, cursor, plen);
    		tal_arr_expand(&blinded_path->path, tmp);
    	}
    
   --- ./wire/bolt12_wiregen.c.pre	2024-10-23 12:26:09.079176474 +1030
   +++ ./wire/bolt12_wiregen.c	2024-10-23 12:26:52.612826902 +1030
   @@ -316,7 +316,7 @@
    	r->offer_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->offer_paths, cursor, plen);
    		tal_arr_expand(&r->offer_paths, tmp);
    	}
    }
   @@ -729,7 +729,7 @@
    	r->offer_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->offer_paths, cursor, plen);
    		tal_arr_expand(&r->offer_paths, tmp);
    	}
    }
   @@ -1052,7 +1052,7 @@
    	r->invreq_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->invreq_paths, cursor, plen);
    		tal_arr_expand(&r->invreq_paths, tmp);
    	}
    }
   @@ -1385,7 +1385,7 @@
    	r->offer_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->offer_paths, cursor, plen);
    		tal_arr_expand(&r->offer_paths, tmp);
    	}
    }
   @@ -1708,7 +1708,7 @@
    	r->invreq_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->invreq_paths, cursor, plen);
    		tal_arr_expand(&r->invreq_paths, tmp);
    	}
    }
   @@ -1781,7 +1781,7 @@
    	r->invoice_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->invoice_paths, cursor, plen);
    		tal_arr_expand(&r->invoice_paths, tmp);
    	}
    }
   @@ -1808,7 +1808,7 @@
    	r->invoice_blindedpay = *plen ? tal_arr(r, struct blinded_payinfo *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_payinfo * tmp;
   -		tmp = fromwire_blinded_payinfo(r, cursor, plen);
   +		tmp = fromwire_blinded_payinfo(r->invoice_blindedpay, cursor, plen);
    		tal_arr_expand(&r->invoice_blindedpay, tmp);
    	}
    }
   @@ -1927,7 +1927,7 @@
    	r->invoice_fallbacks = *plen ? tal_arr(r, struct fallback_address *, 0) : NULL;
    	while (*plen != 0) {
    		struct fallback_address * tmp;
   -		tmp = fromwire_fallback_address(r, cursor, plen);
   +		tmp = fromwire_fallback_address(r->invoice_fallbacks, cursor, plen);
    		tal_arr_expand(&r->invoice_fallbacks, tmp);
    	}
    }
   
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
Rusty Russell
9af01b062c pytest: fix hsmtool which reports leak under address sanitizer.
Couldn't figure out why hsmtool.proc.wait(WAIT_TIMEOUT) returns 1?
hsmtool doesn't ever seem to exit status 1!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
niftynei
e66653fa1d hsmtool: fixup show usage parens + return 2024-11-15 12:48:44 +10:30
Rusty Russell
b7d77b49df hsmtool.c: extend generatehsm to allow command-line args.
Based on the patch by bstin <barry.github@capsmx.com>, which added a separate command,
this simply extends "generatehsm" to allow more options.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: hsmtool: generatehsm can run non-interactive, taking options on the cmdline.
2024-11-15 12:48:44 +10:30
Rusty Russell
ab6f405ae7 common/hsm_encryption: use const char * for errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-13 08:49:55 +10:30
Rusty Russell
dca3a8c72c hsmtool: fix NULL secp256k1_ctx when calling dumpcommitments
secp256k1_ctx is used by pubkey_from_node_id.  Don't try to pick and
choose where to initialize secp256k1_ctx, just always do it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-06 21:17:36 +10:30
Chris Guida
c7189f644a make: Change mv to mv -f when replacing tools/headerversions file
Whenever we build without `make clean` first, this file gets
overwritten. It asks the user for input because it is not run with
`-f`. This causes the build to hang until the user inputs `y`.

Here is an example build:

```
$ make -j7 && sudo make install
CC: cc -DCLN_NEXT_VERSION="v24.08" -DPKGLIBEXECDIR="/usr/local/libexec/c-lightning" -DBINDIR="/usr/local/bin" -DPLUGINDIR="/usr/local/libexec/c-lightning/plugins" -DCCAN_TAL_NEVER_RETURN_NULL=1 -Wall -Wundef -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wold-style-definition -Werror -Wno-maybe-uninitialized -Wshadow=local -std=gnu11 -g -fstack-protector-strong -Og -I ccan -I external/libwally-core/include/ -I external/libwally-core/src/secp256k1/include/ -I external/jsmn/ -I external/libbacktrace/ -I external/gheap/ -I external/build-x86_64-linux-gnu/libbacktrace-build -I external/libsodium/src/libsodium/include -I external/libsodium/src/libsodium/include/sodium -I external/build-x86_64-linux-gnu/libsodium-build/src/libsodium/include -I . -I/usr/local/include  -I/usr/include/postgresql    -DSHACHAIN_BITS=48 -DJSMN_PARENT_LINKS  -DCOMPAT_V052=1 -DCOMPAT_V060=1 -DCOMPAT_V061=1 -DCOMPAT_V062=1 -DCOMPAT_V070=1 -DCOMPAT_V072=1 -DCOMPAT_V073=1 -DCOMPAT_V080=1 -DCOMPAT_V081=1 -DCOMPAT_V082=1 -DCOMPAT_V090=1 -DCOMPAT_V0100=1 -DCOMPAT_V0121=1  -c -o
LD: cc   -Og  config.vars  -Lexternal/build-x86_64-linux-gnu -lwallycore -lsecp256k1 -ljsmn -lbacktrace -lsodium -L/usr/local/include -lm -lsqlite3   -L/usr/lib/x86_64-linux-gnu -lpq -o
mv: replace 'tools/headerversions', overriding mode 0755 (rwxr-xr-x)? cc lightningd/test/run-check_node_announcement.c
cc lightningd/test/run-find_my_abspath.c
cc lightningd/test/run-invoice-select-inchan.c
cc lightningd/test/run-jsonrpc.c
cc lightningd/test/run-log_filter.c
cc lightningd/test/run-log-pruning.c
cc lightningd/test/run-shuffle_fds.c
ld lightningd/test/run-find_my_abspath
ld lightningd/test/run-log-pruning
ld lightningd/test/run-check_node_announcement
ld lightningd/test/run-log_filter
ld lightningd/test/run-jsonrpc
ld lightningd/test/run-shuffle_fds
ld lightningd/test/run-invoice-select-inchan
^Cmake: *** [tools/Makefile:16: tools/headerversions] Interrupt
```

One workaround is to just know that you need to enter `y` here.
But the best solution is probably to update the Makefile like so.

Changelog-None
2024-10-14 15:00:36 +10:30
Rusty Russell
00e0b99d3a hsmtool: provide nodeid from hsm secret.
This allows tools to validate that it is accessing the correct hsm_secret for this node!

This is extremely important for backups: if they are using VLS, they need to back *that*
up instead, for example.

Changelog-Added: `hsmtool`: `getnodeid` command derives the node id from the hsm_secret, to verify it's the correct secret.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-25 12:07:40 -07:00
ShahanaFarooqui
1a71e15cf3 github: Read tag on HEAD in Github Actions
Running `git describe` in local setup correctly identifies the tag available on the tag. But `docker buildx` via actions does not identify it without `--tags` param.

Reference Issue: https://github.com/ElementsProject/lightning/issues/7626

Changelog-Fixed: Docker image created via github actions correctly reads the tag available on the HEAD.
2024-09-05 12:31:49 -07:00
ShahanaFarooqui
48fa438b15 script: Updating SHASUMS for updated zlib1g-dev in Ubuntu Noble
Changelog-Fixed: core lightning's reproducible build for Ubuntu v24.04 (noble).
2024-08-26 17:10:22 -07:00
ShahanaFarooqui
50faa7c122 script: Auto generate rpc examples for documentation
I will commit doc/schemas/lightning-*.json files separately to keep this commit easier to review.

- test-autogenerate-rpc-examples updates all example request & responses in doc/schemas/lightning-*.json files.

- Updated tools/fromschema.py to accommodate the sql JSON example requirement where it does not accept -o in the query but shell does (for queries containing the = sign).

Changelog-None.
2024-08-09 23:56:45 -07:00
Alex Myers
48258ecb31 reckless: correct logging levels
Some output was hidden under normal operation which should
not have been.
2024-08-08 11:33:39 +09:30
Alex Myers
fe58c731ed reckless: escape strings in log 2024-08-08 11:33:39 +09:30
Alex Myers
9a7481b532 reckless: handle failure to find entrypoint cleanly 2024-08-08 11:33:39 +09:30
Alex Myers
dff7b50040 reckless: handle other --json cases without crashing 2024-08-08 11:33:39 +09:30
Alex Myers
efd67b36f0 reckless: handle unresolvable situations in a --json friendly way 2024-08-08 11:33:39 +09:30
Alex Myers
4aef72648c reckless: accept json array arguments as input
Changelog-Added: Reckless: accepts json array input for command targets
2024-08-08 11:33:39 +09:30
Alex Myers
1ac6e25ffd reckless: don't polute stdout with python install
status if --json was requested.
2024-08-08 11:33:39 +09:30
Alex Myers
bb47bc1d4a reckless: all command functions return objects to enable json out
This more easily allows list output for commands accepting list
input, i.e., installing 3 plugins produces 3 outputs.
2024-08-08 11:33:39 +09:30
Alex Myers
a2e458047f reckless: add json output option
Also redirect config creation prompts to stderr in order to not interfere
with json output on stdout.

Changelog-Added: reckless provides json output with option flag -j/--json
2024-08-08 11:33:39 +09:30
Alex Myers
75d8d8b91f reckless: add logger class
This will allow redirection of json output in the following commits.
2024-08-08 11:33:39 +09:30
Alex Myers
40c24065f7 reckless: make options flags position independent
Changelog-Changed: Reckless option flags are now position independent.
2024-08-08 11:33:39 +09:30
Alex Myers
0a6c58133d reckless: add version information to reckless 2024-08-08 11:33:39 +09:30
Alex Myers
9cbaafb525 reckless: add installer for rust plugins
This follows the same structure that enables python virtual environments:
  reckless/
    <plugin_name>/
      <symlink to compiled bin>
      source/
        <clone of original source plugin dir>/

Changelog-Added: Reckless: added the ability to install rust plugins.
2024-08-08 11:33:39 +09:30
Alex Myers
ed2c0ae18f reckless: place source code in source/plugin_name/
rather than just source/.  This is required for cargo install, so let's just
use this paradigm globally.
2024-08-08 11:33:39 +09:30
Alex Myers
c1d46d6ef9 reckless: add type hints for InstInfo 2024-08-08 11:33:39 +09:30
Alex Myers
7387c9cb1f reckless: remove mimetype
The python mimetype package wasn't useful enough in indentifying
filetypes anyhow.
2024-08-08 11:33:39 +09:30
Rusty Russell
a1fdeee76b Makefile: clean up install path handling.
We had some weird code to try to do relative paths.  Instead, use absolute
ones and keep life simple.  Also rename "daemon_dir" to the clearer
"subdaemon_dir" as that's what it's used for.

We now need special magic to do "installcheck", which is a bit awkward,
but at least it's contained.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 18:32:56 +02:00
Rusty Russell
531845971c gossipd: drop zlib support.
This was removed from the spec on Apr 25, 2022.  We stopped ever sending them
in 0.12.0 (2022-08-23).  Now we remove receive support.

Changelog-Protocol: Removed support for zlib-compressed short-channel-ids in query responses.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-10 12:21:19 +09:30
Rusty Russell
fd9975c288 reckless: don't use time-based directories.
This always fails on my test machine, which runs many tests in parallel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
ShahanaFarooqui
b803609247 docs: Removed example usage
Example usage has been either merged with examples or usage keys. This heading was not required separately.

This PR edits & rearranges Examples, Example Notifications and Example Usage data.

Changelog-None.
2024-06-22 17:02:33 +02:00
ShahanaFarooqui
e94b793f05 docs: Example notifications heading
- Added missing Example Notifications Heading
- Renamed `example_json_notifications` to `example_notifications`
- Moved `example_notifications` to the end of the page
2024-06-22 17:02:33 +02:00
ShahanaFarooqui
976c0c90bc docs: Changes examples format
- Changed ALL `doc/schemas/lightning-*.json` file's `json_example` to `examples`
- Change the heading from example to examples
- Bring shell command before the json command
- Move Example to the end of the page
- Remove horizontal line from Example
2024-06-22 17:02:33 +02:00
ShahanaFarooqui
44de5023cb docs: Updated scripts for rpc examples
This update will add lightning-cli examples and make examples heading collapsible.

- The JSON EXAMPLES heading is collapsible now

- Examples in lightning-cli format also

- manpage examples have better formatting

Changelog-None.
2024-06-19 15:56:22 +09:30
Rusty Russell
4a36edf7c1 tools/check-bolt: rewrite to give more accurate diagnostics when we're out of sync.
Just use substrings not regexes, since we have more complex characters now LaTeX
is entering the spec!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 15:54:24 +09:30
Rusty Russell
1de569d38d devtools/gossipwith: use timestamp filter message not obsolete INIT_ROUTING_SYNC.
This means we do have to set the network correctly though,
and also we can get query messages from lightningd which we have to filter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 15:54:24 +09:30
ShahanaFarooqui
c9c1df6710 script: Added caching option for docker publish and shellcheck fixes
Changelog-None.
2024-06-13 11:43:37 +09:30
ShahanaFarooqui
3dc6e6d29e script: Add reproducible build script for Ubuntu v24.04
This PR will help in publishing CLN reproducible binaries for Ubuntu v24.04 (noble).

Please note that I adjusted Dockerfiles for focal and jammy also to keep the base image creation script same for all three images. The step update was required because `noble` only runs with ubuntu:noble setup.

Changelog-None.
2024-06-03 14:23:36 -05:00
ShahanaFarooqui
7f5282aa65 docker: Add missing git dependency for Fedora Dockerfile
Fedora image building by build-release.sh currently throws error ` git: command not found`.

Fixed it by adding git in the Dockerfile and loading the image in the script.

Changelog-None.
2024-05-24 15:07:39 -05:00
nicolas.dorier
369c0c1391 Update Dockerfile and build-release.sh to PR7214 2024-05-23 10:30:15 -05:00
nicolas.dorier
738ddc66cc Use cross compile in dockerfile, fix arm32 support 2024-05-23 10:30:15 -05:00
daywalker90
c98185d944 tools: allow spelling of HtlcTlv 2024-05-17 16:30:43 +02:00
ShahanaFarooqui
9e782f491b doc: updated schemas for json_examples 2024-04-04 13:16:27 +10:30
bstin
6edbc635cc Update hsmtool.c
This line is a cosmetic off-by-one error. There are 7 languages defined, but because counting starts at 0, the display is "Select [0-7]" when in fact only 0-6 are valid choices.
2024-03-20 14:29:47 +10:30
Rusty Russell
9450d46db1 bitcoin/short_channel_id: pass by copy everywhere.
It's a u64, we should pass by copy.  This is a big sweeping change,
but mainly mechanical (change one, compile, fix breakage, repeat).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-20 13:51:48 +10:30
Rusty Russell
e0e879c003 common: remove type_to_string files altogther.
This means including <common/utils.h> where it was indirectly included.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-20 13:51:48 +10:30
Rusty Russell
37d22f9141 global: change all type_to_string to fmt_X.
This has the benefit of being shorter, as well as more reliable (you
will get a link error if we can't print it, not a runtime one!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-20 13:51:48 +10:30
ShahanaFarooqui
f72b0fdabd docs: Updates script and fixed test to generate lightning-sql
- Updated doc/Makefile for generating schemas/lightning-sql.json
    - Read the lightning-sql template from schemas/lightning-sql-template.json
    - Generate sql tables data and merge it with json object read from above template
    - Save final merged object in schemas/lightning-sql.json
- Updated doc/Makefile to auto generate lightning-sql.7.md and lightning-sql.7
- Deleted schemas/lightning-sql.json and adding it into .gitignore
- Added lightning-sql specific titles in the fromschema.py script
- Fixed test_sql by changing the sequence for listpeerchannels `reestablished` field
- Ignoring lightning-sql.json for msggen schema.json because it is auto generated by sql plugin and lightning-sql-template.json.
2024-03-19 14:58:59 +10:30
ShahanaFarooqui
418ecf3e42 script: Update scripts
- Update `make doc-all` script
- `fromschema.py` script
    - Updated to generate whole doc/lightning-*.md via doc/schemas/lightning-*.json file
    - Updated to print request params
    - Added `oneOfMany` condition for request params
    - Added `pairedWith` condition for request params
    - Added `dependentUpon` condition for request params
    - Updated for pre and post_return_value for response
    - Hiding `hidden` fields
    - `descriptions` are array now
    - Unified gaps between titles
    - Added default key-value pair

- script: msggen, sql-schema, listconfig and pyln-testing script updates

This does not add created markdowns for cleaner review. We will add the markdown files in a separate commit.
2024-03-19 14:58:59 +10:30