Commit Graph

13002 Commits

Author SHA1 Message Date
Matt Morehouse
4f30857401 lightningd: close plugin dir on return
Memory leak detected by ASan:

==880002==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 32816 byte(s) in 1 object(s) allocated from:
    #0 0x5039e7 in malloc (lightningd/lightningd+0x5039e7)
    #1 0x7f2e8c203884 in __alloc_dir (/lib64/libc.so.6+0xd2884)
2023-06-05 16:16:21 +02:00
Matt Morehouse
e81bf8b89f devtools: call common_setup in main
Fixes nullability errors detected by UBSan:

wire/fromwire.c:173:46: runtime error: null pointer passed as argument 1, which is declared to never be null
external/libwally-core/src/secp256k1/include/secp256k1.h:432:3: note: nonnull attribute specified here
    #0 0x65214a in fromwire_secp256k1_ecdsa_signature wire/fromwire.c:173:6
    #1 0x659500 in printwire_secp256k1_ecdsa_signature devtools/print_wire.c:331:1
    #2 0x646ba2 in printwire_channel_update wire/peer_printgen.c:1900:7
    #3 0x637182 in printpeer_wire_message wire/peer_printgen.c:128:11
    #4 0x65a097 in main devtools/decodemsg.c:85:10
2023-06-05 16:16:21 +02:00
Matt Morehouse
3edb092bd8 plugins: initialize destination_reachable
Otherwise we later copy the uninitialized memory to descendants,
triggering undefined behavior:

plugins/libplugin-pay.c:2882:34: runtime error: load of value 190, which is not a valid value for type 'bool'
2023-06-05 16:16:21 +02:00
Matt Morehouse
45cb2b149c db: avoid calling memcpy on NULL
It is possible for db_column_bytes() to return 0 and for
db_column_blob() to return NULL even when db_column_is_null() returns
false. We need to short circuit in this case.

Detected by UBSan:

  db/bindings.c:479:12: runtime error: null pointer passed as argument 2, which is declared to never be null
  /usr/include/string.h:44:28: note: nonnull attribute specified here

  #0 0x95f117 in db_col_arr_ db/bindings.c:479:2
  #1 0x95ef85 in db_col_channel_type db/bindings.c:459:32
  #2 0x852c03 in wallet_stmt2channel wallet/wallet.c:1483:9
  #3 0x81f396 in wallet_channels_load_active wallet/wallet.c:1749:23
  #4 0x81f03d in wallet_init_channels wallet/wallet.c:1765:9
  #5 0x72f1f9 in load_channels_from_wallet lightningd/peer_control.c:2257:7
  #6 0x672856 in main lightningd/lightningd.c:1121:25
2023-06-05 16:16:21 +02:00
Matt Morehouse
a5afb4f811 common: remove json_stream_log_suppress
The function is tiny and was only used in one location. And that one
location was leaking memory.

Detected by ASan:

==2637667==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 7 byte(s) in 1 object(s) allocated from:
    #0 0x4cd758 in __interceptor_strdup
    #1 0x64c70c in json_stream_log_suppress_for_cmd lightning/lightningd/jsonrpc.c:597:31
    #2 0x68a630 in json_getlog lightning/lightningd/log.c:974:2
    ...

SUMMARY: AddressSanitizer: 7 byte(s) leaked in 1 allocation(s).
2023-06-05 16:16:21 +02:00
Matt Morehouse
c05d7e4f78 bitcoin: avoid adding to NULL
Detected by UBSan:

$ UBSAN_OPTIONS=print_stacktrace=1 ./wallet/test/run-psbt_fixup

bitcoin/psbt.c:733:2: runtime error: applying zero offset to null pointer
    #0 0x53c829 in psbt_from_bytes lightning/bitcoin/psbt.c:733:2
    #1 0x5adcb0 in main lightning/wallet/test/run-psbt_fixup.c:174:10

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior bitcoin/psbt.c:733:2
2023-06-05 16:16:21 +02:00
Shahana Farooqui
73b73483bf doc: making release document update 2023-06-05 14:54:32 +02:00
Rusty Russell
50a7681171 doc/Makefile: restore CI check for schema changes.
We now know the base reference, and we've rebased, so we can do a
simple diff.  Also, this means we can use a magic commit message
`No-schema-diff-check` to suppress false positives.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-05 09:25:57 +09:30
Rusty Russell
e0d2c39565 CI: use rebasing instead of merge.
This reflects what we actually do when we apply the commit, and also
means we can easily iterate the commits.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-05 09:25:57 +09:30
Rusty Russell
3ac949d4c3 listconfigs: add plugin field if config is for a plugin.
I chose the full path name, not just the basename.

Suggested-by: @SimonVrouwe
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
6275dd384c lightningd: deprecate listconfigs direct fields.
Changelog-Deprecated: JSON-RPC: `listconfigs` direct fields, use `configs` sub-object and `set`, `value_bool`, `value_str`, `value_int`, or `value_msat` fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
8d02f33d28 reckless: use modern listconfigs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
ea928bfca1 pytest: use modern listconfigs.
Use the configs object, as the others are about to be deprecated.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
9cb2b2f13a listconfigs: show plugin options in 'configs' with normal options.
This integrates them with configvars properly: they almost "just work"
in listconfigs now, and we don't put them in a special sub-object
under their plugin.

Unfortunately, this means `listconfigs` now has a loose schema: any
plugin can add something to it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: reloaded plugins get passed any vars from configuration files.
Changelog-Deprecated: Config: boolean plugin options set to `1` or `0` (use `true` and `false` like non-plugin options).
2023-06-03 10:50:29 +09:30
Rusty Russell
1b252f349f config: replace accept-htlc-tlv-types with accept-htlc-tlv-type
We use multi-specifiable options elsewhere, this is just another.
Otherwise you can't add, you can only set them all.

Changelog-Added: Config: `accept-htlc-tlv-type` (replaces awkward-to-use `accept-htlc-tlv-types`)
Changelog-Deprecated: Config: `accept-htlc-tlv-types` (use `accept-htlc-tlv-type` multiple times)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
0df97547dd lightningd: don't simply ignore defaults on flags, deprecate.
Changelog-Deprecated: Plugins: `default` no longer accepted on `flag` type parameters (it was silently ignored, so just don't set it).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
b9270c564a lightningd: listconfigs update, using configvars code.
listconfigs is convenient, but it doesn't handle multi-options well: it
outputs an object with duplicate fields in this case (e.g. log-file), nor
is it extensible to show more than raw values.

However, listconfigs doesn't do what other list commands do (use a
sub-object "configs") so we can put the new values under that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listconfigs` now has `configs` subobject with more information about each config option.
2023-06-03 10:50:29 +09:30
Rusty Russell
f1e994a095 lightningd: fix listconfigs rpc-file-mode
It literally contained \" to avoid it being interpreted as a literal;
now we have OPT_SHOWINT we no longer need this hack.

It's obscure, so I'm not bothering with a deprecation cycle.

Changelog-Fixed: JSON-RPC: `listconfigs` `rpc-file-mode` no longer has gratuitous quotes (e.g. "0600" not "\"0600\"").
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
8281008b08 lightningd: annotate configuration settings which contain JSON literals.
We have hacky code to show some listconfigs values as literals; instead
explicitly encode the types.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
86abb4c4bd lightningd: use OPT_EXITS for options which exit.
Clearly, listconfigs shouldn't list these.

Also, hoist the opt_hidden check since it's independent of whether
there's an arg or not.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
de148febb1 lightningd: make --clear-plugins override prior plugin configvars.
It's an obscure command, but this we won't see old plugin commands in
listconfigs (once it uses configvars).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
5a0994c2d0 lightningd: allow --regtest.
Developers, rejoice (we already have --testnet, --signet and --mainnet!).

Changelog-Added: Config: `--regtest` option as alias for `--network=regtest`
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
edbaa944da lightningd: switch parsing to common/configvar
Now we wire in the code which gathers configvars and parses from there;
lightningd keeps the array of configuration variables for future use.

Note that lightning-cli also needs to read the config, but it has its
own options (including short ones!) and doesn't want to use this
configvar mechanism, so we have a different API for that now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
f54e4f3084 lightningd: explicitly mark developer-only options with OPT_DEV.
This is a nod towards moving to runtime-developer-mode, and also
quite clear; we currently have all these options prefixed with dev,
but this flags makes handling explicit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
ec8bba6b8a dev-allowdustreserve: make this a DEVELOPER option.
Otherwise, we should rename it?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
ea665d1851 lightningd: explicitly note what parameters can be specified multiple times.
This will be used for listconfig, which knows these should be presented
as arrays, not single values.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
36200a6593 common/configvar: routines to manage config settings.
These are gathered from the config files and the commandline, but the
process is rather complex!  We want to remember where the options came
from in future (for a `setconfig` command), and also generalize
and simplify handling.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
f7df7eeb42 lightningd: implement more show commands to reduce listconfigs special casing.
Note that this actually changes listconfigs output for three msat
fields, which were not changed with the great msat merge.  Since
listconfigs isn't actually used by grpc, and the values are always a
little vague, I simply changed this.

Changelog-Fixed: JSON-RPC: `listconfigs` `htlc-minimum-msat`, `htlc-maximum-msat` and `max-dust-htlc-exposure-msat` fields are now numbers, not strings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
1bb462759d lightningd: simplify listconfigs loop.
Now we can look up the name using opt_find_long, we can do that first,
simplifying this code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
f45c17450b ccan: update to latest ccan/opt
This adds:
1. ability to search for an option by name.
2. allowance to set our own bits when registering options.
3. show callbacks which can say "don't show", and variable length.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
9b98f9789e bcli: fix type of rpcport.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
d348554ff4 pyln-client: don't try to convert objects to Millisatoshi.
The new listconfigs fields will be objects, and this messes us up!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
45e16180bc lightningd: remove deprecated null for missing plugin options.
We leave the code in contrib/pyln-client/pyln/client/lightning.py to handle
msat null fields for now, though, for a bit more compatibility.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
2bbd9b8a72 common: allow optional fields in json_scan().
Currently it fails if a field is missing, but sometimes that's OK.  So
allow a fieldname ending in `?` to mean "skip over if it's missing".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Christian Decker
1e457bd48d rs: Bump version of cln-plugin 2023-06-02 14:47:13 +02:00
Alex Myers
40a50c1ada gossipd: don't fail on gossip deletion
Reported in #6270, there was an attempt to delete gossip overrunning
the end of the gossip_store. This logs the gossip type that was attempted to be deleted and avoids an immediate crash (tombstones would be fine to
skip over at least.)

Changelog-None
2023-06-02 12:02:39 +09:30
Christian Decker
23088911f7 grpc: The plugin needs to use the server feature on cln-grpc 2023-06-01 13:41:21 +09:30
Christian Decker
00ffb39ed6 rs: Guard the cln-rpc dependency behind "server" feature in cln-grpc
The `cln-grpc` crate really has a dual purpose: server and
client. Having the server feature be included by default means that we
are pulling in `cln-rpc` which is a Unix only crate, because of the
use of UDS to talk to CLN. We want to use `cln-grpc` on clients too,
and those might not be Unix variants, hence they'd fail when compiling
`cln-rpc`. This PR guards the Unix-related parts behind the `server`
feature flag.
2023-06-01 13:41:21 +09:30
Rusty Russell
ca34931a29 doc/Makefile: suppress stupid check.
I wasted far too much time on this, disable and reenable later.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell
fafeb65012 lightningd: deprecated experimental-websocket-port now we can use bind=ws:
Changelog-Deprecated: `experimental-websocket-port`: use `--bind=ws::<portnum>`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell
3d5367278b lightningd: allow --bind=ws:
Changelog-Added: Config: `bind=ws:...` to explicitly listen on a websocket.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell
a6772e9dec common: add new internal type for websockets.
Now it's not a public type, we need a way to refer to it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell
7b843e8e58 lightningd: deprecate weird listening options.
These are only likely to confuse users, by silently changing behavior.

Changelog-Deprecated: Config: bind-addr=xxx.onion and addr=xxx.onion, use announce=xxx.onion (which was always equivalent).
Changelog-Deprecated: Config: addr=/socketpath, use listen=/socketpath (which was always equivalent).
2023-06-01 09:28:39 +09:30
Rusty Russell
21958879cf lightningd: deprecated --announce-addr-dns.
This obsoletes the use of --announce-addr-dns which I know Michael
didn't really like either.

Changelog-Deprecated: Config: `announce-addr-dns`; use `--bind-addr=dns:ADDR` for finer control.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell
6d0b46c9b1 wireaddr: clean up option parsing, support dns: prefix directly.
This is a major cleanup to how we parse addresses.

1. parse_wireaddr now supports the "dns:" prefix to support dns records (type 5).
2. We are less reliant on separate_address_and_port() which gets confused by
   that colon.
3. We explicitly test every possible address type we can get back from
   parsing, and handle them appropriately.

We update the documentation to use the clearer HOSTNAME vs DNS prefixes now
we also have `dns:` as a prefix.

Changelog-Added: Config: `bind` can now take `dns:` prefix to advertize DNS records.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell
3f35d48fe4 common: remove websocket type from wireaddr.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell
e66cf46a71 connectd: don't advertise websocket addresses.
I never really liked this hack: websockets are useful, advertizing
them not so much.

Note that we never actually documented that we would advertize these!

Changelog-EXPERIMENTAL: Protocol: Removed support for advertizing websocket addresses in gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell
d40379885d common/wireaddr.h: simplify parse_wireaddr API.
1. Make it the standard "return the error" pattern.
2. Rather than flags to indicate what types are allowed, have the callers
   check the return explicitly.
3. Document the APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell
ddb79162ab connectd: clean up add_gossip loops.
This contained cut & paste code, and it wasn't clear to me that
the first loop included DNS entries with IPv6 entries.

Instead, allow the iterator to take multiple types, and use
a switch statement so compile will break as new types are added.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell
1b6ff0b2fc staticbackup: don't use wireaddr_internal.
This is an internal type: it has no API guarantees (indeed, I'm about
to change it, which is how I discovered scb was using it).

Fortunately for every case we care about, it is actually a wireaddr
(in theory the peer can connect locally using a local socket, but this
is mostly for testing and is a very strange setup, and so simply don't
do scb for those).

In this case, the wire encoding is a single byte followed by the
wireaddr, so open-code that in scb_wire.csv for compatibility.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30