Commit Graph

121 Commits

Author SHA1 Message Date
Christian Decker
640e64cb73 db: Switch statement lookup to use the original query instead
Using a generated identifier with filename and line proved to be brittle since
compilers assign the __LINE__ macro differently on multi-line macro
invocations.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Christian Decker
bf613fa48a postgres: Add postgres statement rewriting support
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Rusty Russell
895e552475 BOLT: update to master with gossip_queries_ex.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell
f4e7ed179f devtools/mkencoded: tool to encode short_channel_id / other for testing
In particular, it does zlib.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-12 05:11:56 +00:00
Rusty Russell
8ef996d56e devtools/mkgossip: tool to create gossip msgs for testing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-12 05:11:56 +00:00
Rusty Russell
5592cfb174 devtools/dump-gossipstore: add flag to show deleted entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-12 05:11:56 +00:00
Rusty Russell
87f0ee6351 channeld: set option_static_remotekey when negotiated.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell
c99906a9a9 per-peer-daemons: tie in gossip filter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Rusty Russell
aca2e4f722 common/memleak: add dynamic hooks for assisting memleak.
Rather than reaching into data structures, let them register their own
callbacks.  This avoids us having to expose "memleak_remove_xxx"
functions, and call them manually.

Under the hood, this is done by having a specially-named tal child of
the thing we want to assist, containing the callback.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Rusty Russell
acf3952acc JSON: remove handling of pre-Adelaide (B:T:N) short_channel_ids.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:19:14 +02:00
Rusty Russell
a134062f98 bolt11: handle 9 fields for new features.
This implements https://github.com/lightningnetwork/lightning-rfc/pull/656

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-05 23:39:05 -05:00
Christian Decker
70e8da4fbd wallet: Add read-only flag to extracted queries
This gets rid of the two parallel execution paths of read-only and write
queries, by explicitly stating with each query whether it is a read-only
query, we only need to remember the ones marked as write queries.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker
e4ab98459c wallet: Add tooling to extract SQL queries and generate driver info
This is the counterpart of the annotations we did in the last few commits. It
extracts queries, passes them through a driver-specific query rewriter and
dumps them into a driver-specific query-list, along with some metadata to
facilitate processing later on. The generated query list is then registered as
a `db_config` and will be loaded by the driver upon instantiation.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Rusty Russell
2600a6ed2e channeld: get current block height when an HTLC fails.
We need it to put in the error code for
WIRE_INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
9dd314226d devtools/mkclose: make a mutual close transaction.
Guess who's been writing gossip protocol tests?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-26 08:44:22 +00:00
Rusty Russell
39b34a35c8 bitcoin/tx.c: don't free witness implicitly.
This causes a crash in mkfunding, which didn't expect it:

    $ devtools/mkfunding 16835ac8c154b616baac524163f41fb0c4f82c7b972ad35d4d6f18d854f6856b 1 0.01btc 253 76edf0c303b9e692da9cb491abedef46ca5b81d32f102eb4648461b239cb0f99 0000000000000000000000000000000000000000000000000000000000000010 0000000000000000000000000000000000000000000000000000000000000020
    # funding sig: 798d96d5a057b5b7797988a855217f41af05ece3ba8278366e2f69763c72e78565d5dd7eeddc0766ddf65557c92b9c52c301f23f94d2cf681860d32153e6ae1e
    # funding witnesses: [
    Aborted (core dumped)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-26 08:44:22 +00:00
Rusty Russell
e40f07803c devtools/gossipwith: allow --max-messages=0 to avoid reading msgs at all.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-20 00:07:38 +00:00
Rusty Russell
f1e84b3d99 devtools: make clean should remove devtools/gen_print_onion_wire.[c,h,o]
Reported-by: @JavierRSobrino
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-13 05:12:10 +00:00
lisa neigut
0acdeeec06 fixup mkcommit's reverse ordered chainparams 2019-08-09 05:07:18 +00:00
lisa neigut
c7f3fa34b2 funding tx: include segwit marker + flag in fee calculation
Noticed an off by one error when running tests for dual-funding;
we're not including the two 'header' segwit bytes in our weight
calculations.
2019-08-03 05:19:24 +00:00
lisa neigut
b0b6ddb66f devtools: rm repetitive field (it's included in remotesecrets) 2019-08-03 05:19:24 +00:00
lisa neigut
abcde461df devtools: print witnesses along with signatures for mkfunding
we need the witnesses for dual-funding transactions
2019-08-03 05:19:24 +00:00
Rusty Russell
3fa375881a bigsize: make it a proper first-class type.
It doesn't belong in bitcoin, and should not be confused with varint_t.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Christian Decker
9288a7906b tx: Add chainparams to struct bitcoin_tx as context
The way we build transactions, serialize them, and compute fees depends on the
chain we are working on, so let's add some context to the transactions.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
Rusty Russell
201b531eb6 devtools: fix credit script.
Lisa's git name is lower-case, whereas CHANGELOG.md uses upper case,
so it doesn't realize she's named a commit already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 17:38:11 +08:00
Christian Decker
581694fdda devtools: Minor cleanup of the onion command line tool
Simplifying some operations, erroring in some cases and moving to global
defines for constants.

Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
c752c3318d sphinx: Cleanup sphinx onion construction, remove realm
The realm has lost significance, so let's unify this into the type.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
660921a9dd sphinx: Introduce a runtest command to the onion tool
The `runtest` command takes a JSON onion spec, creates the onion and decodes
it with the provided private keys. It is fully configurable and can be used
for the test-vectors in the spec.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
6831db62f7 sphinx: Clean up after migrating to the sphinx_path struct
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
ca0dc01bee sphinx: Fix the onion cli tool to take public keys
This was a mismatch between the go tool and this test tool so far. Just
aligning the tools to allows for easier testing.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
a0a1a1f752 sphinx: Add function to add a new v0 hop to a sphinx_path
This is just taking the existing serialization code and repackaging it in a
more useful form.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Rusty Russell
c580225c38 wire: handle bigsize/varint fields.
They're currently called varint, but there's a proposal to call them all
bigsize.  Allow both for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-27 21:19:57 -05:00
Rusty Russell
2e68b88be8 tools: fix tlv generation
We need to hand -s to both header and body generation, or neither:

wire/gen_peer_wire.c:53:13: error: static declaration of ‘towire_channel_update_timestamps’ follows non-static declaration
In file included from wire/gen_peer_wire.c:5:
./wire/gen_peer_wire.h:78:6: note: previous declaration of ‘towire_channel_update_timestamps’ was here

We also need it for printwire, otherwise we get static unused functions for subtypes:

devtools/gen_print_wire.c:155:13: error: ‘printwire_channel_update_checksums’ defined but not used [-Werror=unused-function]
 static void printwire_channel_update_checksums(const char *fieldname, const u8 **cursor, size_t *plen)
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
devtools/gen_print_wire.c:133:13: error: ‘printwire_channel_update_timestamps’ defined but not used [-Werror=unused-function]
 static void printwire_channel_update_timestamps(const char *fieldname, const u8 **cursor, size_t *plen)
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-27 21:19:57 -05:00
Rusty Russell
b1738c5b89 tools: fix 32 bit compile error
```
tools/test/enum.c: In function ‘fromwire_test_enum’:
tools/test/enum.c:11:34: error: format ‘%ld’ expects argument of type ‘long int’, but argument 2 has type ‘size_t {aka unsigned int}’ [-Werror=format=]
  printf("fromwire_test_enum at %ld\n", *max);
```

and:

```
devtools/print_wire.c: In function ‘printwire_tlvs’:
devtools/print_wire.c:201:22: error: format ‘%ld’ expects argument of type ‘long int’, but argument 2 has type ‘u64 {aka long long unsigned int}’ [-Werror=format=]
    printf("**TYPE #%ld UNKNOWN for TLV %s**\n", type, fieldname);
                      ^
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-25 11:41:48 +08:00
ZmnSCPxj
559e3f35a6 devtools/: Ignore mkcommit and mkfunding executables. 2019-07-24 16:15:38 +02:00
lisa neigut
281b4c241e bolt-gen: fixup the devtool/decodemsg printing facility
Fixup TLV handling in the bolt printing utility, `devtools/decodemsg`
2019-07-24 02:52:53 +00:00
Rusty Russell
0eda6a5ce7 devtools/mkcommit: cleanups.
Also, change the values to match the spec values (I made the
to_self_delays different to catch more bugs).

Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
Rusty Russell
63d5c7fc1f devtools/onion: allow setting the hop_data.
This is also required for actually creating usable onions.  For the moment,
due to API limitations, we only let them set realm 0.

Note that the privkey parsing was broken, requiring an additional two
hex digits, overflowing the buffer, and were ignored.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
Rusty Russell
b9ea1165c4 devtools/onion: allow setting the associated data.
This is required for actually creating usable onions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
Rusty Russell
1b8adabd4f devtools/onion: fix --decode
Add odd-length string can never be valid hex!

In addition, don't try to print the next hop if there isn't one, but
always print the (raw) payload.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
Rusty Russell
12d8be4fdc devtools: have mkcommit support HTLCs.
This allows for complete channel simulation, including HTLC
transactions, but means we use higher-level primitives to
make the easy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
Rusty Russell
8b09a9ad06 devtools: add mkfunding and mkcommit.
These utilities allow us to create valid test txs and information given both
sides' complete set of secrets.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
Rusty Russell
5879f6b4e0 devtools/gossipwith: expose the INIT messages.
Usually we send a canned INIT message and ignore theirs.  This adds
an option to not do that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
Rusty Russell
1036f98613 devtools/gossipwith: allow setting the secret key.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
Rusty Russell
af535bf5ff devtools/gossipwith: allow interaction.
This lets us use it as an interactive driver of conversation, rather
than writing all packets then reading all packets.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-17 12:55:37 -05:00
lisa neigut
5c07afac7d bolt: update to BOLT spec changes (extract format + type specifications)
updates the bolt version to 6639cef095a2ecc7b8f0c48c6e7f2f906fbfbc58.

this requires us to use the new bolt parser at generate-bolt.py
and updates to all of the type specifications (ie. from u8 -> byte)
2019-07-16 06:10:58 +00:00
Rusty Russell
0d2a4830ed ccan: update to faster and correct crc32c implementation.
I decided to try a faster implementation, only to find our crc32c was
not correct!  Ouch.

I removed the crc32c functions from ccan/crc, and added a new crc32c
module which has the Mark Adler x86-64-optimized variants.

We bump gossip_store version again, since csums have changed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-11 23:40:10 +00:00
Rusty Russell
0e7b26d7c7 devtools/credit: script to highlight contributions for this commit.
And update MAKING-RELEASES.md to refer to it

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-07 13:37:58 +02:00
Rusty Russell
2abae05daa devtools/create-gossipstore: write timestamps to the gossip_store.
We need the timestamp for channel_announcement, but this is simplified
because MCP always follows the channel_announcement by a channel_update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-04 01:29:39 +00:00
Rusty Russell
948490ec58 gossipd: add timestamp in gossip store header.
(We don't increment the gossip_store version, since there are only a
few commits since the last time we did this).

This lets the reader simply filter messages; this is especially nice since
the channel_announcement timestamp is *derived*, not in the actual message.

This also creates a 'struct gossip_hdr' which makes the code a bit
clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-04 01:29:39 +00:00