Commit Graph

23 Commits

Author SHA1 Message Date
Christian Decker
5794c83b4d tlv: Add typesafe fromwire codegen for TLV namespaces
We were weaving in and out of generic code through `fromwire_tlvs` with custom
parameters defining the types in that namespace. This hard-wires the parser
with the namespace's types. Slowly trying to deprecate `fromwire_tlvs` in
favor of this typesafe variant.
2019-11-22 04:40:25 +00:00
Christian Decker
2255024ead tlv: Add raw fields so we can store unknown fields as well 2019-11-22 04:40:25 +00:00
lisa neigut
3f1f075421 tools: add ability to wrap wire messages with ifs
Makes it possible to hide wire messages behind EXPERIMENTAL_FEATURES
flag.
2019-10-10 05:57:45 +00:00
lisa neigut
e99720344e wire-gen: ensure that tlv messages are correctly ordered
Our TLV serializer relies on TLV outputs to be ordered by type
number. Prior to this commit we relied on 1) the ordering in the
RFC to be correct and 2) users to be using a version of Python that
respects stable ordering of dicts (i.e. Python 3.7+)

Instead of relying on these implicitly, we now explicitly sort messages
by type number when the TLV sets.

Resolves #2956.

Thanks-To: @ScottTre for the sort function
Reported-By: @ZmnSCPxj
2019-08-21 03:53:54 +00:00
Rusty Russell
b10e0e08bb tools/generate-wire.py: add option to expose tlv_record_type
Next update adds TLV test vectors: without this, we get a warning
about them being unused.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell
3f8600e9c0 tools/generate-wire.py: handle implicit tlv length fields.
TLVs have an implicit `len` field, so allow expressions containing
that (eg. `len-1`), but assume it means "the remainder of the
message".

This means in most places, f.size() needs an fallback for the
implicit-length case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-27 21:19:57 -05:00
Rusty Russell
2e68b88be8 tools: fix tlv generation
We need to hand -s to both header and body generation, or neither:

wire/gen_peer_wire.c:53:13: error: static declaration of ‘towire_channel_update_timestamps’ follows non-static declaration
In file included from wire/gen_peer_wire.c:5:
./wire/gen_peer_wire.h:78:6: note: previous declaration of ‘towire_channel_update_timestamps’ was here

We also need it for printwire, otherwise we get static unused functions for subtypes:

devtools/gen_print_wire.c:155:13: error: ‘printwire_channel_update_checksums’ defined but not used [-Werror=unused-function]
 static void printwire_channel_update_checksums(const char *fieldname, const u8 **cursor, size_t *plen)
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
devtools/gen_print_wire.c:133:13: error: ‘printwire_channel_update_timestamps’ defined but not used [-Werror=unused-function]
 static void printwire_channel_update_timestamps(const char *fieldname, const u8 **cursor, size_t *plen)
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-27 21:19:57 -05:00
lisa neigut
42251cc5b9 bolt-gen: fixup subtype nested varsize definition
variable length varsized subtypes were being allocated as
structs at not pointers; this fixes that.
2019-07-27 05:18:27 +00:00
lisa neigut
b6d4c372bb bolt-gen: fix broken 'exposed subtype' generation
borked in 96bf7aea; fixed here.
2019-07-24 06:31:46 +00:00
lisa neigut
068496298c bolt-gen: rm unused bolt-generator; rename new bolt generator
delete now unused wire-generator and replace it with the newer
version.
2019-07-24 06:31:46 +00:00
lisa neigut
236d26308f bolt-gen: make optional 'assignable' fields work
make assignable 'optional' fields work, add test for them
2019-07-24 06:31:46 +00:00
lisa neigut
ff82c918a5 bolt-gen: formatting cleanups
add brackets for clarity when printing subtypes; also remove
unnecessary exception (`find_` will raise an exception if
the key is not present)
2019-07-24 06:31:46 +00:00
lisa neigut
316edb39a4 bolt-gen: for wire messages, print out optional fields (if present)
optional fields should be printed, if they exist. so let's print them!
2019-07-24 06:31:46 +00:00
lisa neigut
009e9dc945 bolt-gen: neaten up generated comments
remove extra space from the lead-in for inline comments

so that a provided comment like this

    # This is a comment

will appear like this

    /* This is a comment */
2019-07-24 06:31:46 +00:00
lisa neigut
281b4c241e bolt-gen: fixup the devtool/decodemsg printing facility
Fixup TLV handling in the bolt printing utility, `devtools/decodemsg`
2019-07-24 02:52:53 +00:00
lisa neigut
4261e508a9 bolt-gen: add TLV support
Add in support for buiding TLV's (minus the printing capability)
2019-07-24 02:52:53 +00:00
lisa neigut
12125ec8dc bolt-gen: small cleanups
we were incorrectly printing the references for a subtype case, plus
some errant styling,readability changes
2019-07-24 02:52:53 +00:00
lisa neigut
f2819ba7d8 bolt-gen: remove 'is-optional' qualifier from msg field
we now handle optional fields, so we should include them in
the message parsing signatures
2019-07-24 02:52:53 +00:00
lisa neigut
96bf7aead5 bolt-gen: handle variable-sized and optionals
actually do the right thing for variable-sized and optional field
types
2019-07-24 02:52:53 +00:00
lisa neigut
ade594e941 bolt-gen: add 'top-line' file comments to output
if there are any comments that aren't "attached" to a message,
print them at the top of the generated file. we need this for
the fancy auto-gen'd dependencies in the tool-wiregen tests.
2019-07-24 02:52:53 +00:00
lisa neigut
fe3f4f52a0 bolt-gen: handle csv inline comments
The bolts don't have in-line comments, but the internal wire
message CSVs do. This adds the 'comments' back in to the
generated docs.
2019-07-24 02:52:53 +00:00
lisa neigut
d7a68b75f1 bolt-gen: fixup broken subtype parsing 2019-07-24 02:52:53 +00:00
lisa neigut
6c240ab589 bolts: new parsing script and templates for new bolt format
the RFC's extract-format.py is switching to a new format.
this script can correctly parse them.

mostly moves logic over from generate-wire.py, uses a
Python formatting libarary called mako, which needs to be
installed prior to running this script.

you can add it to your system with

    sudo apt-get install python3-mako
2019-07-16 06:10:58 +00:00