mirrors/core-lightning
1
0
Fork 0
mirror of https://github.com/ElementsProject/lightning.git synced 2024-11-19 09:54:16 +01:00
Code Issues Projects Releases Wiki Activity
15,612 Commits 30 Branches 190 Tags 191 MiB
67d667db90
Commit Graph

44 Commits

Author SHA1 Message Date
Rusty Russell
38d229df3d tools/generate-wire.py: make sure TLV array fields are allocated off TLV. 
Otherwise the whole thing cannot be tal_steal() onto a different parent.

Here's the difference in generated files:

   --- ./wire/onion_wiregen.c.pre	2024-10-23 12:26:09.023176933 +1030
   +++ ./wire/onion_wiregen.c	2024-10-23 12:26:52.434828303 +1030
   @@ -128,7 +128,7 @@
     	blinded_path->path = num_hops ? tal_arr(blinded_path, struct blinded_path_hop *, 0) : NULL;
    	for (size_t i = 0; i < num_hops; i++) {
    		struct blinded_path_hop * tmp;
   -		tmp = fromwire_blinded_path_hop(blinded_path, cursor, plen);
   +		tmp = fromwire_blinded_path_hop(blinded_path->path, cursor, plen);
    		tal_arr_expand(&blinded_path->path, tmp);
    	}
    
   --- ./wire/bolt12_wiregen.c.pre	2024-10-23 12:26:09.079176474 +1030
   +++ ./wire/bolt12_wiregen.c	2024-10-23 12:26:52.612826902 +1030
   @@ -316,7 +316,7 @@
    	r->offer_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->offer_paths, cursor, plen);
    		tal_arr_expand(&r->offer_paths, tmp);
    	}
    }
   @@ -729,7 +729,7 @@
    	r->offer_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->offer_paths, cursor, plen);
    		tal_arr_expand(&r->offer_paths, tmp);
    	}
    }
   @@ -1052,7 +1052,7 @@
    	r->invreq_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->invreq_paths, cursor, plen);
    		tal_arr_expand(&r->invreq_paths, tmp);
    	}
    }
   @@ -1385,7 +1385,7 @@
    	r->offer_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->offer_paths, cursor, plen);
    		tal_arr_expand(&r->offer_paths, tmp);
    	}
    }
   @@ -1708,7 +1708,7 @@
    	r->invreq_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->invreq_paths, cursor, plen);
    		tal_arr_expand(&r->invreq_paths, tmp);
    	}
    }
   @@ -1781,7 +1781,7 @@
    	r->invoice_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->invoice_paths, cursor, plen);
    		tal_arr_expand(&r->invoice_paths, tmp);
    	}
    }
   @@ -1808,7 +1808,7 @@
    	r->invoice_blindedpay = *plen ? tal_arr(r, struct blinded_payinfo *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_payinfo * tmp;
   -		tmp = fromwire_blinded_payinfo(r, cursor, plen);
   +		tmp = fromwire_blinded_payinfo(r->invoice_blindedpay, cursor, plen);
    		tal_arr_expand(&r->invoice_blindedpay, tmp);
    	}
    }
   @@ -1927,7 +1927,7 @@
    	r->invoice_fallbacks = *plen ? tal_arr(r, struct fallback_address *, 0) : NULL;
    	while (*plen != 0) {
    		struct fallback_address * tmp;
   -		tmp = fromwire_fallback_address(r, cursor, plen);
   +		tmp = fromwire_fallback_address(r->invoice_fallbacks, cursor, plen);
    		tal_arr_expand(&r->invoice_fallbacks, tmp);
    	}
    }
   
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2024-11-18 11:03:26 +10:30
Rusty Russell
5e76c74622 tools/generate_wire.py: don't declare unused for variable. 
Ubuntu clang 15.0.2-1 complains:

```
wire/peer_exp_wiregen.c:257:14: error: variable 'i' set but not used [-Werror,-Wunused-but-set-variable]
        for (size_t i = 0; *plen != 0; i++) {
                    ^
wire/peer_exp_wiregen.c:1373:14: error: variable 'i' set but not used [-Werror,-Wunused-but-set-variable]
        for (size_t i = 0; *plen != 0; i++) {
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2022-11-18 12:26:59 +01:00
adi2011
286d6c3165 tools/gen: Always return bool! 2022-07-14 12:24:48 -05:00
Rusty Russell
83ee68ab06 common/tlvstream: put TLV checking back in the generic function. 
Callers were supposed to call "tlv_fields_valid" after fromwire_tlv,
but few did.  Make this the default, and call the underlying function
directly where we want to be more flexible (one place).

This loses the ability to allow misordered fields, or to pass through
*any* even fields.  We restore that for special cases in the next
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2022-03-25 13:55:44 +10:30
Rusty Russell
a770f51d0e tools/generate_wire.py: make functions allocate the TLV. 
Requiring the caller to allocate them is ugly, and differs from
other types.

This means we need a context arg if we don't have one already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2022-03-25 13:55:44 +10:30
Rusty Russell
12c7b156c5 tools/generate_wire.py: allow generated C files to have includes added. 
We previously ignored --include= for these, but onion is about to start
needing bolt12.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2022-03-25 13:55:44 +10:30
Rusty Russell
ceb40dea38 lightningd: don't turn zero-length tlv fields into NULL. 
Fixes: #4667
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2021-07-21 13:27:27 -04:00
Christian Decker
b4ead97517 tlv: Allow passing some extra types to accept when parsing the stream 2021-06-26 10:55:13 +09:30
Rusty Russell
2fea448498 gen/impl_template: fix generation of singleton varsize elements. 
And as Lisa requested, add testcases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2021-06-04 16:13:08 +09:30
Rusty Russell
5c167d16ab tools/generate-wire.py: use helpers. 
This was terrible cut & paste.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2020-12-04 20:16:54 -06:00
Rusty Russell
dc83e64003 tools/generate-wire: don't use void * pointers for tlv fromwire. 
And fix up the one place which got it wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2020-12-04 20:16:54 -06:00
Rusty Russell
0e805427dc tools/generate-wire.py: strip trailing whitespace on lines, fix bolt quotes. 
There's a lot of it, and it means we can't `make check-source` on
these files.

Also bring bolt quotes up-to-date.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2020-09-09 16:23:58 +09:30
Rusty Russell
897c53ce1c tools/generate-wire.py: fix loop logic for towire_xxx_array 
has_len_fields() doesn't cover our blacklist of variable types, so if
we have an array of them, this logic is wrong.  This happens in the
the bolt13 patch:

```C
struct tlv_offer_tlvs_blindedpath {
        struct pubkey blinding;
        struct onionmsg_path **path;
};
```

Before:
wire/gen_bolt13_tlv.c:
```C
		for (size_t i = 0; i < tal_count(r->blindedpath->path); i++)
		towire_onionmsg_path(&ptr, r->blindedpath->path + i);
```
After:
```C
		for (size_t i = 0; i < tal_count(r->blindedpath->path); i++)
		towire_onionmsg_path(&ptr, r->blindedpath->path[i]);
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2020-09-02 09:46:37 +09:30
Rusty Russell
76813d6f90 tools/generate-wire.py: fix varsize assignment. 
Code like this is suspicious:

	subtype_varsize_struct->field_0 = *fromwire_test_features(subtype_varsize_struct, cursor, plen);

In fact, it is a memory leak since we copy and don't free the fromwire
result.  Really, field_0 should be a pointer.

We don't hit this case (yet!) in spec-generated code, but I did for
bolt13.

Here's the difference in gen_test output:

```patch
  diff -ur /tmp/before/test/gen_test.c /tmp/after/test/gen_test.c
  --- /tmp/before/test/gen_test.c	2020-05-07 16:23:31.651611235 +0930
  +++ /tmp/after/test/gen_test.c	2020-05-07 16:20:54.232574482 +0930
  @@ -214,12 +214,12 @@
   static void towire_subtype_varsize_struct(u8 **p, const struct subtype_varsize_struct *subtype_varsize_struct)
   {
   
  -	towire_test_features(p, &subtype_varsize_struct->field_0);
  +	towire_test_features(p, subtype_varsize_struct->field_0);
   }
   static void fromwire_subtype_varsize_struct(const u8 **cursor, size_t *plen, struct subtype_varsize_struct *subtype_varsize_struct)
   {
   
  - 	subtype_varsize_struct->field_0 = *fromwire_test_features(subtype_varsize_struct, cursor, plen);
  + 	subtype_varsize_struct->field_0 = fromwire_test_features(subtype_varsize_struct, cursor, plen);
   }
   
   /* SUBTYPE: SUBTYPE_VAR_LEN */
  @@ -373,7 +373,7 @@
   
   	ptr = tal_arr(ctx, u8, 0);
   
  -	towire_test_features(&ptr, &r->tlv3->features);
  +	towire_test_features(&ptr, r->tlv3->features);
   
   	towire_amount_msat(&ptr, r->tlv3->amount_msat_1);
   
  @@ -385,7 +385,7 @@
   	struct tlv_test_n1 *r = vrecord;
   
   	r->tlv3 = tal(r, struct tlv_test_n1_tlv3);
  -	r->tlv3->features = *fromwire_test_features(r->tlv3, cursor, plen);
  +	r->tlv3->features = fromwire_test_features(r->tlv3, cursor, plen);
   	r->tlv3->amount_msat_1 = fromwire_amount_msat(cursor, plen);
   	r->tlv3->amount_msat_2 = fromwire_amount_msat(cursor, plen);
   }
  @@ -824,11 +824,11 @@
   
   	towire_test_short_id(&ptr, &r->tlv3->subtype);
   
  -	towire_subtype_var_len(&ptr, &r->tlv3->varlen_subtype);
  +	towire_subtype_var_len(&ptr, r->tlv3->varlen_subtype);
   
  -	towire_subtype_var_assign(&ptr, &r->tlv3->varlen_assigned);
  +	towire_subtype_var_assign(&ptr, r->tlv3->varlen_assigned);
   
  -	towire_subtype_varlen_varsize(&ptr, &r->tlv3->test_sbt_varlen_varsize);
  +	towire_subtype_varlen_varsize(&ptr, r->tlv3->test_sbt_varlen_varsize);
   
   		for (size_t i = 0; i < 2; i++)
   		towire_u32(&ptr, r->tlv3->arr_assign[i]);
  @@ -868,9 +868,9 @@
   
   	r->tlv3 = tal(r, struct tlv_test_n3_tlv3);
   	fromwire_test_short_id(cursor, plen, &r->tlv3->subtype);
  -	r->tlv3->varlen_subtype = *fromwire_subtype_var_len(r->tlv3, cursor, plen);
  -	r->tlv3->varlen_assigned = *fromwire_subtype_var_assign(r->tlv3, cursor, plen);
  -	r->tlv3->test_sbt_varlen_varsize = *fromwire_subtype_varlen_varsize(r->tlv3, cursor, plen);
  +	r->tlv3->varlen_subtype = fromwire_subtype_var_len(r->tlv3, cursor, plen);
  +	r->tlv3->varlen_assigned = fromwire_subtype_var_assign(r->tlv3, cursor, plen);
  +	r->tlv3->test_sbt_varlen_varsize = fromwire_subtype_varlen_varsize(r->tlv3, cursor, plen);
   		for (size_t i = 0; i < 2; i++) {
   		u32 tmp;
   		tmp = fromwire_u32(cursor, plen);
  diff -ur /tmp/before/test/gen_test.h /tmp/after/test/gen_test.h
  --- /tmp/before/test/gen_test.h	2020-05-07 16:23:30.399617108 +0930
  +++ /tmp/after/test/gen_test.h	2020-05-07 16:20:52.912584680 +0930
  @@ -45,7 +45,7 @@
           struct test_short_id field_1;
   };
   struct subtype_varsize_struct {
  -        struct test_features field_0;
  +        struct test_features *field_0;
   };
   struct subtype_var_len {
           struct test_short_id *field_2;
  @@ -60,15 +60,15 @@
           struct test_short_id field3[2];
   };
   struct tlv_test_n1_tlv3 {
  -        struct test_features features;
  +        struct test_features *features;
           struct amount_msat amount_msat_1;
           struct amount_msat amount_msat_2;
   };
   struct tlv_test_n3_tlv3 {
           struct test_short_id subtype;
  -        struct subtype_var_len varlen_subtype;
  -        struct subtype_var_assign varlen_assigned;
  -        struct subtype_varlen_varsize test_sbt_varlen_varsize;
  +        struct subtype_var_len *varlen_subtype;
  +        struct subtype_var_assign *varlen_assigned;
  +        struct subtype_varlen_varsize *test_sbt_varlen_varsize;
           /*  array assigtest_nable */
           u32 arr_assign[2];
           /*  array structs */
  Binary files /tmp/before/test/gen_test.o and /tmp/after/test/gen_test.o differ
  Binary files /tmp/before/test/run-test-wire and /tmp/after/test/run-test-wire differ
  Binary files /tmp/before/test/run-test-wire.o and /tmp/after/test/run-test-wire.o differ
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2020-09-02 09:46:37 +09:30
Rusty Russell
dc8458d1e2 tools: fix comment in template. 
It's (usually) a .csv not an _csv file.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2020-08-25 12:53:13 +09:30
Rusty Russell
197d1bcef2 wire: move towire/fromwire_short_channel_id out to bitcoin/short_channel_id.c 
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2020-05-18 14:51:12 +02:00
Rusty Russell
b0c9059602 tools/generate-wire: no more lonely messages! 
When we have only a single member in a TLV (e.g. an optional u64),
wrapping it in a struct is awkward.  This changes it to directly
access those fields.

This is not only more elegant (60 fewer lines), it would also be
more cache friendly.  That's right: cache hot singles!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2020-05-06 14:56:09 -05:00
Christian Decker
5325ff6352 json-rpc: Don't let users send messages that are handled internally 
We cannot let users use `sendcustommsg` to inject messages that are handled
internally since it could result in our internal state tracking being borked.
 2020-01-28 23:50:52 +01:00
Rusty Russell
eb6a768741 tools: don't use bytelength as array length for arrays in TLVs. 
This matters now we have an array in tlv_init_tlvs!  We were overallocating
in fromwire by 32x!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2019-12-13 16:36:40 +01:00
Christian Decker
626675c83c tlv: Migrate tlv serialization to typesafe function 2019-12-03 00:37:15 +00:00
Christian Decker
e12b5c3824 tlv: Add a typesafe serialization function for tlv namespaces 
This is the counterpart to the typesafe deserialization function implemented
in an earlier commit.
 2019-12-03 00:37:15 +00:00
Christian Decker
5a78671d9f wire: Remove unused fromwire_tlvs 
We are now using the typesafe variant everywhere.
 2019-12-03 00:37:15 +00:00
Christian Decker
69c17d2d31 wire: Let the TLV _is_valid function actually return validity 
I got this one wrong myself, since the function name implied a boolean
result. So I changed it to take the optional err_index as argument.
 2019-12-03 00:37:15 +00:00
darosior
670f92002d tools/gen/impl_template: correct one-line for-loops indentation 2019-11-26 21:30:25 +01:00
darosior
5fb8e0aade tools/gen/impl_template: correct tlvs fromwire's for-loop 2019-11-26 21:30:25 +01:00
Christian Decker
2519f934aa tlv: Add validity check codegen for the tlv namespaces 
Since the parser itself just parses and doesn't include validation anymore we
need to put that functionality somewhere. The validation consists of enforcing
that the types are in monotonically increasing order without duplicates and
that for the even types we know how to handle it.
 2019-11-22 04:40:25 +00:00
Christian Decker
5794c83b4d tlv: Add typesafe fromwire codegen for TLV namespaces 
We were weaving in and out of generic code through `fromwire_tlvs` with custom
parameters defining the types in that namespace. This hard-wires the parser
with the namespace's types. Slowly trying to deprecate `fromwire_tlvs` in
favor of this typesafe variant.
 2019-11-22 04:40:25 +00:00
Christian Decker
2255024ead tlv: Add raw fields so we can store unknown fields as well 2019-11-22 04:40:25 +00:00
lisa neigut
3f1f075421 tools: add ability to wrap wire messages with ifs 
Makes it possible to hide wire messages behind EXPERIMENTAL_FEATURES
flag.
 2019-10-10 05:57:45 +00:00
lisa neigut
e99720344e wire-gen: ensure that tlv messages are correctly ordered 
Our TLV serializer relies on TLV outputs to be ordered by type
number. Prior to this commit we relied on 1) the ordering in the
RFC to be correct and 2) users to be using a version of Python that
respects stable ordering of dicts (i.e. Python 3.7+)

Instead of relying on these implicitly, we now explicitly sort messages
by type number when the TLV sets.

Resolves #2956.

Thanks-To: @ScottTre for the sort function
Reported-By: @ZmnSCPxj
 2019-08-21 03:53:54 +00:00
Rusty Russell
b10e0e08bb tools/generate-wire.py: add option to expose tlv_record_type 
Next update adds TLV test vectors: without this, we get a warning
about them being unused.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2019-08-02 17:32:48 +02:00
Rusty Russell
3f8600e9c0 tools/generate-wire.py: handle implicit tlv length fields. 
TLVs have an implicit `len` field, so allow expressions containing
that (eg. `len-1`), but assume it means "the remainder of the
message".

This means in most places, f.size() needs an fallback for the
implicit-length case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
 2019-07-27 21:19:57 -05:00
lisa neigut
42251cc5b9 bolt-gen: fixup subtype nested varsize definition 
variable length varsized subtypes were being allocated as
structs at not pointers; this fixes that.
 2019-07-27 05:18:27 +00:00
lisa neigut
068496298c bolt-gen: rm unused bolt-generator; rename new bolt generator 
delete now unused wire-generator and replace it with the newer
version.
 2019-07-24 06:31:46 +00:00
lisa neigut
236d26308f bolt-gen: make optional 'assignable' fields work 
make assignable 'optional' fields work, add test for them
 2019-07-24 06:31:46 +00:00
lisa neigut
009e9dc945 bolt-gen: neaten up generated comments 
remove extra space from the lead-in for inline comments

so that a provided comment like this

    # This is a comment

will appear like this

    /* This is a comment */
 2019-07-24 06:31:46 +00:00
lisa neigut
281b4c241e bolt-gen: fixup the devtool/decodemsg printing facility 
Fixup TLV handling in the bolt printing utility, `devtools/decodemsg`
 2019-07-24 02:52:53 +00:00
lisa neigut
4261e508a9 bolt-gen: add TLV support 
Add in support for buiding TLV's (minus the printing capability)
 2019-07-24 02:52:53 +00:00
lisa neigut
12125ec8dc bolt-gen: small cleanups 
we were incorrectly printing the references for a subtype case, plus
some errant styling,readability changes
 2019-07-24 02:52:53 +00:00
lisa neigut
f2819ba7d8 bolt-gen: remove 'is-optional' qualifier from msg field 
we now handle optional fields, so we should include them in
the message parsing signatures
 2019-07-24 02:52:53 +00:00
lisa neigut
96bf7aead5 bolt-gen: handle variable-sized and optionals 
actually do the right thing for variable-sized and optional field
types
 2019-07-24 02:52:53 +00:00
lisa neigut
ade594e941 bolt-gen: add 'top-line' file comments to output 
if there are any comments that aren't "attached" to a message,
print them at the top of the generated file. we need this for
the fancy auto-gen'd dependencies in the tool-wiregen tests.
 2019-07-24 02:52:53 +00:00
lisa neigut
fe3f4f52a0 bolt-gen: handle csv inline comments 
The bolts don't have in-line comments, but the internal wire
message CSVs do. This adds the 'comments' back in to the
generated docs.
 2019-07-24 02:52:53 +00:00
lisa neigut
6c240ab589 bolts: new parsing script and templates for new bolt format 
the RFC's extract-format.py is switching to a new format.
this script can correctly parse them.

mostly moves logic over from generate-wire.py, uses a
Python formatting libarary called mako, which needs to be
installed prior to running this script.

you can add it to your system with

    sudo apt-get install python3-mako
 2019-07-16 06:10:58 +00:00