Commit Graph

164 Commits

Author SHA1 Message Date
Rusty Russell
ab28972aee tools/generate-wire.py: style cleanups.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell
e217bc1220 per-commit-secret is a struct secret, not a sha256.
Well, it's generated by shachain, so technically it is a sha256, but
that's an internal detail.  It's a secret.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell
ac4c6b1a82 tools/generate-wire.py: support for optional fields, with ? before typename.
We already work around this by using an array with a 0/1 length convention,
but better to be explicit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell
c02ff11506 print_wire: hand field names to print routines.
This lets us override how we print them.

Also, add dependency on header for devtools/Makefile.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-01 14:55:29 +02:00
arowser
85e02127a9 align generate wire template 2018-06-20 06:56:42 +00:00
practicalswift
160f9b06a5 Fix variable typo in generate-wire.py 2018-02-24 11:44:32 +01:00
practicalswift
0a20bd44f0 Fix flake8 warnings: W191 indentation contains tabs + E101 indentation contains mixed spaces and tabs 2018-02-22 22:21:31 +01:00
practicalswift
e18948a6a9 Remove unused variables. Fix flake8 warnings. 2018-02-22 22:21:31 +01:00
practicalswift
ae7d857c44 Make Python code conform to the Python PEP 8 style guide 2018-02-22 10:47:19 +01:00
Rusty Russell
e92b710406 tools/generate-wire.py: remove length argument from fromwire_ routines.
We always hand in "NULL" (which means use tal_len on the msg), except
for two places which do that manually for no good reason.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell
b7db06b577 tools/generate-wire.py: wirestring type for handing strings.
A convenient alias for char *, though we don't allow control characters
so our logs can't be fooled with embedded \n.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
Rusty Russell
674a2c7554 tools/generate_wire.py: make bitcoin_tx a varlen structs
Now it allocates naturally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
Rusty Russell
526d3a232e tools/generate_wire.py: generate varlen arrays properly.
These are now logically arrays of pointers.  This is much more natural,
and gets rid of the horrible utxo array converters.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
Rusty Russell
ad8dfaca1c tools/generate_wire.py: make varlen structs self-allocate.
If we tell it a struct is variable length, make fromwire() allocate
and return it off ctx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
Rusty Russell
32abb4d928 tools/generate-wire.py: --printwire option to create print routines.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-02 00:57:10 +00:00
ZmnSCPxj
fdbf2f41b4 generate-wire.py: Add scare comments to output.
Discourage developers from modifying files generated by generate-wire.py
2018-01-10 23:53:12 +00:00
Christian Decker
59128a86ee hsm: Make sure to pass close_info along with the UTXO
The close_info is needed to re-derive the secret key that is supposed
to be used to sign the input spending the output.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
practicalswift
5123c4e059 Fix access to undefined variable 2017-12-28 16:05:15 +01:00
practicalswift
aee3661b35 Fix typos (repeated words) 2017-12-28 16:04:38 +01:00
Rusty Russell
810abb6b21 bitcoin: create new wrapper type bitcoin_blkid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_blkid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Rusty Russell
0237e0b28c bitcoin: create new wrapper type bitcoin_txid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_txid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
practicalswift
0353ec0983 Remove trailing whitespace 2017-12-11 03:35:59 +00:00
practicalswift
61c47c09d0 Fix typos 2017-12-08 13:07:20 +01:00
Rusty Russell
44e45348f2 option-data-loss-protect: fix generate-wire.py and update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 15:26:41 +01:00
Rusty Russell
5320cab2de generate-wire.py: create structures for optional fields.
If a structure foo has a optional fields opt1 and opt2, this creates
towire_foo, towire_foo_opt1 and towire_foo_opt2 (since opt2 implies opt1),
similarly for fromwire_*.

This requires the callers to be updated to call the correct routines (eg.
try fromwire_foo_opt2, then fromwire_foo_opt1, then finally fromwire_foo),
but this is a minimal change to the generation code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-09 13:55:24 +01:00
William Casarin
52f741de97 tools/generate-wire.py: generalize python3 path
Generalize generate-wire.py python3 path

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:12:37 +02:00
Rusty Russell
32631b4278 generate-wire.py: add --bolt arg, use size->type hacks only when that's specified.
For our own internal comms CSVs, we should always name explicit types.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell
b3514d3430 generate-wire.py: generate chain_hash fields as sha256_double.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-22 14:56:02 +02:00
Rusty Russell
aa23a2a93f generate-wire.py: handle unmarshalling of a single varlen_struct
We previously only handled it in arrays.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-20 13:06:41 +09:30
Rusty Russell
7884343c15 tools/generate-wire.py: enums are assignable.
Simplfies their marshalling/unmarshalling.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-07-12 10:21:16 +09:30
Rusty Russell
be6347b814 tools/generate-wire.py: unmarshall zero-length arrays as NULL.
This is symmetrical with marshalling.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell
da18645901 tools/generate-wire.py: don't avoice tal_count() on NULL.
It actually works fine with recent ccan/tal (returning 0).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell
9868e896d9 tools/generate-wire.py: handle marshal/unmarshal of enum types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-23 09:29:42 +09:30
Rusty Russell
ad3ab53fa4 lightningd/htlc_wire: wire types for sending HTLCs to/from channel daemon.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-23 09:29:42 +09:30
Rusty Russell
a2ffccd740 tools/generate-wire.py: fix generation of array of assignable types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-07 09:19:04 +09:30
Rusty Russell
cb5ad91788 tools/generate-wire.py: allow consecutive fields to use same len var.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-07 09:19:04 +09:30
Rusty Russell
6e0e1c7067 Update to latest BOLT (hyphens changed to underscores).
Now in sync with 8ee57b97738b1e9467a1342ca8373d40f0c4aca5.

Our tool doesn't need to convert them any more, but we actually had a
mis-typed field in the HSM which needed fixing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-05-12 12:59:09 +02:00
Rusty Russell
9b6753173c tools/generate-wire: allow constant multiplier in fields.
Not just len*u8, but 1254*u8 for explicit fixed-length arrays. 

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-04-01 23:59:46 +10:30
Rusty Russell
d072f2ec06 wire: understand struct preimage, don't treat it as sha256.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-04-01 23:59:46 +10:30
Christian Decker
b2ea4cfd66 wiregen: Passing ctx to array helpers that require it
Some of the struct array helpers need to allocate data when
deserializing their fields. The `getnodes` reply is one such example
that allocates the hostname. Since the change to calling array helpers
the getnodes call was broken because it was attempting to allocate off
of the entry, which did not have a tal header, thus failing.
2017-03-20 11:18:00 +10:30
Rusty Russell
e042198cf8 tools/generate-wire.py: allow typename instead of type sizes.
We use the fourth value (size) to determine the type, unless the fifth
value is suppled.  That's silly: allow the fourth value to be a typename,
since that's the only reason we care about the size at all!

Unfortunately there are places in the spec where we use a raw fieldname
without '*1' for a length, so we have to distingish this from the
typename case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-03-16 14:35:25 +10:30
Rusty Russell
9768e4c907 tools/generate-wire.py: open-code array handling for non-u8.
Except for the trivial case of u8 arrays, have the generator create
the loop code for the array iteration.

This removes some trivial helpers, and avoids us having to write more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-03-16 14:35:23 +10:30
Christian Decker
4a995a42de gen-wire: Cleanup of the wire generator tool
The wiregen tool was a bit hard to maintain since it was printing all
over the place, mixing template and processing logic. This commit
tears the two apart, externalizes everything that is not a single code
line, and repackages it into templates. Specifically functions are now
their own template and header/implementation files are a template.

Furthermore this simplifies some of the boilerplate of mapping types
to sizes and back again, by extracting them into dicts.

All changes have been verified to produce identical results on the
current wire definitions, except a bit of whitespace changes.
2017-03-06 10:22:02 +10:30
Rusty Russell
7419fde9a0 Update to new spec: differentiate channel_id and short_channel_id.
The spec 4af8e1841151f0c6e8151979d6c89d11839b2f65 uses a 32-byte 'channel-id'
field, not to be confused with the 8-byte short ID used by gossip.  Rename
appropriately, and update to the new handshake protocol.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-03-02 22:51:49 +10:30
Rusty Russell
9fd40da38c generate-wire: don't hand unknown structures specially.
It's awkward to handle them differently.  But this change means we
need to expose them to the generated code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-02-21 15:15:19 +10:30
Rusty Russell
fd09345628 generate-wire: handle variable-size arrays of unknown types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-02-21 14:59:14 +10:30
Rusty Russell
1edce4878d bitcoin/signature.h: remove struct signature.
It's a wrapper around secp256k1_ecdsa_signature, so use that directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-25 11:03:55 +10:30
Rusty Russell
2b8c7cc840 tools/generate-wire.py: don't allocate on unknown names.
This introduces a potential leak; use a static buffer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:20:20 +10:30
Rusty Russell
42f474af18 wire: make lengths of variable fields implied by tal_count()
This is a much nicer interface, and works better in practice too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:19:25 +10:30
Rusty Russell
b6a55a68ea tools/generate-wire.py: handle non-integer enum values.
BOLT 4 uses values like "PERM|1", so allow that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-06 13:40:06 +10:30
Rusty Russell
2057de6d4f tools/generate-wire.py: Apparently OptionParser is deprecated, use argparse.
It is a bit nicer though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-06 13:24:45 +10:30
Rusty Russell
8b72604d4d wire: allow #include directives in csv comments, and unknown structure types.
This lets us marshal and unmarshal undefined complex types; the fromwire
function will allocate it for us, so we don't even need to know the size.

This turns out to be really nice for marshalling 'struct crypto_state'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-06 13:24:45 +10:30
Rusty Russell
f50af430e1 wire: add bool routines.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell
937a62100f generate-wire.py: allow optional typename in csv file.
For our internal CSV files, we can specify the type explicitly rather
than trying to guess (eg. bool).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell
8ad1298f88 generate-wire.py: add enum->name function.
Pretty!  Takes an int instead of the enum directly, because in the
main daemon we call it via a function pointer, so want them all the
same type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell
54cfbfeba9 generate-wire.py: generate enum in order provided in CSV
This means the comments make more sense.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell
ba9e4f9377 generate-wire.py: allow hex values in csv files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell
bdaa22e247 generate-wire.py: allow NULL len parameter from fromwire_*
It implies tal_count() gives the length. Great for almost all callers which
don't care if there are extra bytes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell
e076d56709 generate-wire.py: include type bytes in towire/fromwire routines.
This removes some redundancy in creating messages, but also allows
a lazy form or parsing without explicitly checking the type.

A helper fromwire_peektype() is added to look up the type and handle
the too-short-for-type problem.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell
3e044fdd62 generate-wire.py: allow comments.
We emit them into the generated code at appropriate points, but it
would be better if we simply preserved the order they were given in.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell
bf7fefdc32 generate-wire.py: add memcheck() to towire functions.
Sanity check that we're not streaming uninitialized bytes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell
c864b28068 generate-wire.py: don't generate structures, hand in all values.
This is a bit more awkward for large structures, but avoids
indirection for the simpler ones (I copied the structures for the test
code, however).  We also remove explicit padding.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell
326a9c9477 tools/generate-wire: node_announcement.alias is not an sha256
It's 32 bytes long, but it's just a string.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell
a08a2105ea generate-wire.py: generalize, move to tools.
We're going to want to use this for inter-daemon comms, so generalize it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30