Commit Graph

1147 Commits

Author SHA1 Message Date
Rusty Russell
7157a92ea6 channeld: import updated channel_upgrade spec.
It now uses raw bitfields instead of a subtype, and only allows a single
option for any upgrade.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 16:07:21 +02:00
Rusty Russell
55dbe82162 features: EXPERIMENTAL_FEATURES: advertize option_quiesce
The latest draft has a feature bit here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 16:07:21 +02:00
Rusty Russell
c92ce59892 BOLT 12: switch invoice_request/invoice to singular chain field.
We keep the now-removed chains field, and in deprecated mode, we set it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: bolt12: `chains` in invoice_request and invoice is deprecated, `chain` is used instead.
2021-10-08 13:47:30 +02:00
Rusty Russell
45bf7a3974 bolt12: update to latest spec.
Main changes are:
1. Uses point32 instead of pubkey32.
2. Uses issuer instead of vendor.
3. Uses byte instead of u8.
4. blinded_path num_hops is now a byte, not u16 (we don't use that yet!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: bolt12: `vendor` is deprecated: the field is now called `issuer`.
2021-10-08 13:47:30 +02:00
Rusty Russell
8f582e770c BOLT12: use point32 instead of pubkey32.
That's the modern BOLT12 term.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 13:47:30 +02:00
Rusty Russell
51ed7557a1 bolt12: upgrade to handle latest test vectors.
The latest ones use lno, not lni (this unit tests loads from
../lightning-rfc, silently exiting if it doesn't have the test
vector).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 13:47:30 +02:00
ZmnSCPxj jxPCSnmZ
20fe5c6ab1 common/json.c: Have json_add_literal *use* len argument.
The current `json_add_literal` does not use its `len` argument, at all:

09c2fef4a4/common/json.c (L1076-L1083)

Which is something of a WTF, why even require the `len` argument if it
is not even used in the function?
This would have been acceptable if it were a callback and the
callback-requiring function passed it in always, but this function is
not intended to be a callback but instead to be called directly.

Existing callers always pass in `strlen(literal)`, so this bug was not
noticed before:

09c2fef4a4/lightningd/jsonrpc.c (L511)
09c2fef4a4/lightningd/jsonrpc.c (L583)
09c2fef4a4/lightningd/options.c (L1408-L1409)
2021-10-06 16:09:20 +02:00
W. J. van der Laan
dd93f5dd29 Add missing headers for compiling on FreeBSD 13
- `netinet/in.h`: struct sockaddr_in, struct sockaddr_in6
- `unistd.h`: close()
2021-09-24 12:40:09 +09:30
Rusty Russell
09e0794b06 common/features: try to add all proposed feature names.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-23 15:02:05 +02:00
Rusty Russell
7ef2f4d7fb devtools/features: tool to convert feature bitmap to names.
For example:

```
$ ./devtools/features 80008008226aa2
option_data_loss_protect/odd (optional)
option_upfront_shutdown_script/odd (optional)
option_gossip_queries/odd (optional)
option_var_onion_optin/odd (optional)
option_gossip_queries_ex/odd (optional)
option_static_remotekey/odd (optional)
option_payment_secret/even (compulsory)
option_basic_mpp/odd (optional)
option_anchor_outputs/odd (optional)
option_shutdown_anysegwit/odd (optional)
option_onion_messages/odd (optional)
option_unknown_54/odd (optional)
```


Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-23 15:02:05 +02:00
Rusty Russell
79e09b92ef Makefile: remove generated files.
By popular merge-hell demand.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: Python is now required to build, as generated files are no longer checked into the repository.
2021-09-22 15:25:14 +02:00
Rusty Russell
eaec226f99 common: json_to_reply_path to helper to handle blindedpath/onion_message hook
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell
3a966191b8 common/blindedpath: enctlv unwrapping primitives.
And we check them on our test vectors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell
b98e8a099d common/blindedpath: enctlv creation primitives.
And test vector generation!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell
f31f7b1eec common/sphinx: add helper to prepend length to payload.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell
1f8564ffae type_to_string: detect type failure in DEVELOPER mode.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
Rusty Russell
24536c5561 common/autodata: use instead of ccan/autodata
This means it needs to be linked ~everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
Rusty Russell
b594c53771 common/autodata: autodata replacement which uses __attribute__((constructor))
This is a GCC extension, but avoids much other messiness.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
Jan Sarenik
284ad2bade Fix for Alpine Linux ond OpenBSD
After recent header files clean-up it was not possible to
build c-lightning 7401b2682. This patch fixes it both for
Alpine Linux and OpenBSD.

Proposed-by: nathanael <nathanael@dalliard.ch>

Changelog-None
2021-09-20 14:44:27 +02:00
Rusty Russell
7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
ea30c34d82 cleanup: remove unneeded includes in header files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
00a0d09340 tools/check-includes.sh: test that c files include their .h files.
This is best-practice (to ensure prototypes match up), but there were a
few places we didn't (at least, directly).  Make it a requirement,
either of form "foo.h" or <dir/foo.h>.

The noise is the change to our print templates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
acef45d02e common/channel_type: fix up bolt quotes now channel-types is merged.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
183fe107e8 lightningd: use channel_type, pass to-and-from channeld.
Instead of explicit option_static_remotekey and option_anchor_outputs flags.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
740afb822c common/initial_channel: use channel_type instead of individual option bools.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
cb22015b2a common/channel_type: wrapper for generated 'struct channel_type'.
We make it a first-class citizen internally, even though we won't use
it over the wire (at least, non-experimental builds).  This scheme
follows the latest draft, in which features are flagged compulsory.

We also add several helper functions.

Since uses the *even* bits (as per latest spec), not the *odd* bits,
we have some other fixups.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
1b8551923d tools: handle generating print templates for CSV without tlvs, messages
We want to use this to handle the simple description for channel_type.

It also needs to handle variable-size types (just like subtypes).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
fb4edc2938 Makefile: update bolt version to include option_anchors_zero_fee_htlc_tx.
This touches a lot of text, mainly to change "if `option_anchor_outputs`"
to "if `option_anchors`"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-09 12:04:48 +09:30
Rusty Russell
1b2c6964fd Makefile: update spec version
This includes the new bolt11 test vectors, and also removes the
requirement that HTLCs be less than 2^32 msat.  We keep that for now
because Electrum enforced it on receive: in two releases we will stop
that too.

So no longer warn about needing mpp in that case either.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Protocol: No longer restrict HTLCs to
2021-09-09 12:04:48 +09:30
Rusty Russell
88d55441c5 channeld: allow large HTLCs if peer offers option_support_large_channel
This check is going away anyway (only Electrum enforced it), but we
know that all wumbo peers expect large HTLCs to work today.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: Allow sending large HTLCs if peer offers option_support_large_channel (> 4294967295msat)
2021-09-09 12:04:48 +09:30
Rusty Russell
1f7fb33563 common/memleak: hoist strmap helper from out of lightningd/memdump.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell
a5fee67b91 common/memleak: take over dump_memleak(), allow print pointer.
This will let plugins use it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
niftynei
0f05238e97 psbt: set the witness script to null, also
Otherwise libwally pushes the psbt-key for 'witness script' onto the
serialized version and we fail the 'is this identical' check.

Relevant line from libwally, where if bytes, we push a psbt_key.

```
static void push_typed_varbuff(unsigned char **cursor, size_t *max,
                               uint64_t type,
                               const unsigned char *bytes, size_t bytes_len)
{
    if (bytes) {
        push_psbt_key(cursor, max, type, NULL, 0);
        push_varbuff(cursor, max, bytes, bytes_len);
    }
}
```

Reported-By: @grubles
Changelog-Fixed: openchannel_signed would fail on PSBT comparison of materially identical PSBTs
2021-09-05 14:24:14 +02:00
niftynei
1782ae8590 dual-fund psbt: hoist up single use method so we can test it
Also remove all the unused code (checking for differences for a side
etc)
2021-09-05 14:24:14 +02:00
Rusty Russell
372fffac69 Anchor outputs are enabled by dual-funding, so this was reported by
users there (and reliably detected now).

Reported-by: @larsschenk
Reported-by: @nickfarrow
Fixes: #4728
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: EXPERIMENTAL: crash for some users while requesting dual funding leases.
2021-09-02 15:06:18 +02:00
Rusty Russell
fe86c117d9 datastore: turn keys into arrays
After some discussion with @shesek, and my own usage, we agreed that
a more comprehensive interface, which explicitly supports grouping,
is desirable.

Thus keys are now arrays, with the semantic that a key is either a
parent or has a value, never both.

For convenience in the JSON schema, we always return them as arrays,
though we accept simple strings as arguments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell
533571a655 datastore: add generation, simple atomicity.
We add a generation counter, and allow update or del conditional
on a given generation.

Formalizes error codes, too, since we have more now.

Suggested-by: @shesek
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell
cf1dd779d1 gossmap: don't spew to stderr, include counter for callers.
Fixes: #4722
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: don't drop complaints about silly channels to stderr.
2021-08-24 06:42:32 +09:30
Antoine Poinsot
5d9be5ab97 common/setup: check libwally initialization
We already assume they were succesful, so we'd better be checking them.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-08-14 12:44:33 +09:30
Rusty Russell
17489120e6 common/test/run-bolt12_period.c: fix warnings at -O3.
assertions are useful here anyway, inside unit tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-14 12:07:57 +09:30
Rusty Russell
55e167115b common/tests: these test are no longer experimental-only.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-14 12:07:57 +09:30
Rusty Russell
c448170d5f common/bolt12_merkle.c: fix comments.
I spent an hour thinking this code had a bug (see test vector fix);
we *do* overallocate the tree, but that's deliberate: we fill with NULLs
and ignore on recursion.

The Merkle recurse comment had an out-by-one, and the NULL-pad
technique used was uncommented.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-30 15:45:30 +09:30
Rusty Russell
207f1def93 common/test: test the test vector JSON from the spec (if present).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-30 15:45:30 +09:30
Rusty Russell
83d9a0005d common/test: Fix test vector generation.
We were printing out the final merkle root before calculating it,
resulting in the final one being the same as the previous.

Reported-by: Aditya Sharma
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-30 15:45:30 +09:30
Rusty Russell
60edf14fc7 common: fix up missing file which can cause parallel build issues.
And add checks for any others!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-23 13:26:31 -04:00
Rusty Russell
ceb40dea38 lightningd: don't turn zero-length tlv fields into NULL.
Fixes: #4667
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-21 13:27:27 -04:00
Rusty Russell
7769903f1a bolt12: import latest spec (timestamp -> created_at).
@shesek points out that we called this field created_at in bolt11 decode,
which makes more sense anyway.

Changelog-EXPERIMENTAL: bolt12 decode `timestamp` field deprecated in favor of new name `created_at`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-21 13:27:27 -04:00
Rusty Russell
1aa2ba0991 common/lease_rates: more things const.
And fix up lease_rates_fromhex to be the obvious calling convention.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-20 13:28:38 -04:00
Rusty Russell
6da915c96a lease_rates: lease_rates_eq.
We don't actually use the NULL-is-equal in the one caller, so ignore that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-20 13:28:38 -04:00
Rusty Russell
30c57f9d35 lease_rates, funder: use overflow helpers
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-20 13:28:38 -04:00