Commit Graph

36 Commits

Author SHA1 Message Date
Rusty Russell
a770f51d0e tools/generate_wire.py: make functions allocate the TLV.
Requiring the caller to allocate them is ugly, and differs from
other types.

This means we need a context arg if we don't have one already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell
8f39002715 tools/generate-wire.py: have printwire routines do demarshal.
This works better in general: let printwire_x do the work of figuring
out how to demarshal x.  This is particularly important for TLVs, which
require a call to tlv_x_new() first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell
88de64a580 tools/generate-wire.py: simplify printwire routines, fix ... handling.
We make them return bool, and always use names `cursor` and `plen` in
callers, for simplicity.

Also, `...` means "loop until finished" not "loop this many bytes".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell
4ffda340d3 check: make sure all files outside contrib/ include "config.h" first.
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).

config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell
7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
225fec065a Build race fixes: create ccan/config.h atomically, make tools/test/test_gen.c depend on it
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-14 12:43:54 +09:30
Rusty Russell
2fea448498 gen/impl_template: fix generation of singleton varsize elements.
And as Lisa requested, add testcases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-04 16:13:08 +09:30
Rusty Russell
25b5e1e099 update-mocks: make sure we cover all test programs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
a8177e9013 Makefile: make check-includes check all the non-generated files.
Note that check-whitespace and check-bolt already do this, so we
can eliminate redundant lines in common/Makefile and bitcoin/Makefile.

We also include the plugin headers in ALL_C_HEADERS so they get
checked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Rusty Russell
5e865ce42b Makefile: unify generated files definition.
We change gen_ to _gen, because filtering on gen_% doesn't work if they're
in subdirectories :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Rusty Russell
f7de4ac919 Makefile: fix missing dependencies for tools tests.
This means they are included in `make check-source`, which they now
pass (since previous commit).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-09 16:23:58 +09:30
Rusty Russell
c34c055d82 Makefile: use completely separate spec-derived files for EXPERIMENTAL_FEATURES
This avoids overwriting the ones in git, and generally makes things neater.

We have convenience headers wire/peer_wire.h and wire/onion_wire.h to
avoid most #ifdefs: simply include those.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Rusty Russell
390c6470ad configure: we don't need mako any more.
And guard the mako-specific tests with a check instead.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell
8150d28575 Makefile: use generic rules to make spec-derived sources.
Now we use the same Makefile rules for all CSV->C generation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
niftynei
3640befc10 tools-make: also be quiet if --quiet is flagged
Missed a update-mocks.sh call
2020-05-30 15:36:56 +02:00
Rusty Russell
cfb320c972 wire: move remaining bitcoin functions out to bitcoin/ files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
lisa neigut
3f1f075421 tools: add ability to wrap wire messages with ifs
Makes it possible to hide wire messages behind EXPERIMENTAL_FEATURES
flag.
2019-10-10 05:57:45 +00:00
Rusty Russell
278b12982d tools/test/Makefile: disable unless we're in DEVELOPER mode.
Because it required update-mocks, which is a hack which relies on the
format of linker errors (!) I'd prefer to make this --enable-developer
only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-21 05:07:15 +00:00
Rusty Russell
c74c1e56cc tools/test/Makefile: fix always-on generation of tools/test/gen_test.c
We need to tell it that the tmp file is an intermediate, so doesn't need
remaking if it doesn't exist.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-21 05:07:15 +00:00
Rusty Russell
303c221641 tools/test/Makefile: we depend on tools/test/gen_test.h
This is because update-mocks actually tries to build it to get link
errors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-19 03:43:13 +00:00
Rusty Russell
c0ba5879df tools/test: can't generate mocks until submodules checked out.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-17 14:08:43 +09:30
Rusty Russell
dbb15b0045 Makefile: fix occasional race where we fail to compile tools/tests.
This simplifies the dependencies:
1. Objs depend on headers, not other objs.
2. Programs depend on objs.
3. A .o file will generally implicitly depend on the .c file it's built from.
4. If a file has a build line, it's often better to list all deps there.
5. I spotted some missing 'make clean' files.

The particular problem in this case seems to be that make would use
tools/test/gen_test.c before it was ready.  It's probably confused by
the use of recursive make via update-mocks, so explicitly split that
into two stages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-17 04:20:05 +00:00
Rusty Russell
9e5b9a31c9 FreeBSD: fix build.
This makes it build for me on FreeBSD 11:
1. $(MAKE) has to passed through into update-mocks.
2. FreeBSD sed doesn't turn \n into a newline on RHS.
3. Bash and mako dependencies were missing from INSTALL.md

Fixes: #2850
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-29 14:24:29 +02:00
Rusty Russell
3c3a89c483 tools/generate-wire.py: add test for implicit lengths.
Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-27 21:19:57 -05:00
Rusty Russell
b1738c5b89 tools: fix 32 bit compile error
```
tools/test/enum.c: In function ‘fromwire_test_enum’:
tools/test/enum.c:11:34: error: format ‘%ld’ expects argument of type ‘long int’, but argument 2 has type ‘size_t {aka unsigned int}’ [-Werror=format=]
  printf("fromwire_test_enum at %ld\n", *max);
```

and:

```
devtools/print_wire.c: In function ‘printwire_tlvs’:
devtools/print_wire.c:201:22: error: format ‘%ld’ expects argument of type ‘long int’, but argument 2 has type ‘u64 {aka long long unsigned int}’ [-Werror=format=]
    printf("**TYPE #%ld UNKNOWN for TLV %s**\n", type, fieldname);
                      ^
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-25 11:41:48 +08:00
lisa neigut
46cbca97ea bolt-gen: use 'enum' instead of 'e:' prefix for enums
use 'enum ' in wirespec instead of 'e:' as an indicator
that a field is an enum.
2019-07-24 06:31:46 +00:00
lisa neigut
068496298c bolt-gen: rm unused bolt-generator; rename new bolt generator
delete now unused wire-generator and replace it with the newer
version.
2019-07-24 06:31:46 +00:00
lisa neigut
236d26308f bolt-gen: make optional 'assignable' fields work
make assignable 'optional' fields work, add test for them
2019-07-24 06:31:46 +00:00
lisa neigut
316edb39a4 bolt-gen: for wire messages, print out optional fields (if present)
optional fields should be printed, if they exist. so let's print them!
2019-07-24 06:31:46 +00:00
lisa neigut
b30d7d26ea bolt-gen: handle enums
Add parsing know-how for enum fields. This is necessary for
internally defined wire generators. Enums are denoted by prefixing
the field with an `e:`.

Ex:

   msgdata,msg_name,field_name,e:enum_type,
2019-07-24 06:31:46 +00:00
lisa neigut
cfd56d86ee bolt-gen: keep single instance of type_obj per type
we rely, perhaps a bit hackily, on there only being one copy of
each type object floating about. using `deepcopy` on a Message
for message extensions destroys this paradigm, which breaks
things in the case where it's a later defined subtype that contains
variable-length members.

to fix this, we modify `__deepcopy__` on the Field class, such
that it preserves the reference to the original type_obj instance.
2019-07-24 06:31:46 +00:00
lisa neigut
7f5ba35e95 bolt-gen-tests: namespace test tlv's under test_
for posterity.
2019-07-24 02:52:53 +00:00
lisa neigut
281b4c241e bolt-gen: fixup the devtool/decodemsg printing facility
Fixup TLV handling in the bolt printing utility, `devtools/decodemsg`
2019-07-24 02:52:53 +00:00
lisa neigut
cedebfd2d9 bolt-gen: add truncated uint support for TLV fields
TLV's have truncated values, add support for them. plus
some 'compilation' tests.
2019-07-24 02:52:53 +00:00
lisa neigut
4261e508a9 bolt-gen: add TLV support
Add in support for buiding TLV's (minus the printing capability)
2019-07-24 02:52:53 +00:00
lisa neigut
79f13fa429 bolt-gen: add compilation tests
Add a test for checking that the bolt-gens do the right thing
for a fairly exhaustive test case set (and that it compiles).

Note that this doesn't check that we've got the memory assignment
pieces worked out.

It's got a kind of exotic reliance on the update-mocks in that in
order to depend on as little of the wire/ code as possible (we
only import wire/wire.h), we include an AUTOGENERATE comment
in the test_cases CSV file, and then run update-mocks as part of
the build for that file.
2019-07-24 02:52:53 +00:00