Add a test for checking that the bolt-gens do the right thing
for a fairly exhaustive test case set (and that it compiles).
Note that this doesn't check that we've got the memory assignment
pieces worked out.
It's got a kind of exotic reliance on the update-mocks in that in
order to depend on as little of the wire/ code as possible (we
only import wire/wire.h), we include an AUTOGENERATE comment
in the test_cases CSV file, and then run update-mocks as part of
the build for that file.
updates the bolt version to 6639cef095a2ecc7b8f0c48c6e7f2f906fbfbc58.
this requires us to use the new bolt parser at generate-bolt.py
and updates to all of the type specifications (ie. from u8 -> byte)
These are generalized from our internal implementations.
The main difference is that 'struct json_escaped' is now 'struct
json_escape', so we replace that immediately.
The difference between lightningd's json-writing ringbuffer and the
more generic ccan/json_out is that the latter has a better API and
handles escaping transparently if something slips through (though
it does offer direct accessors so you can mess things up yourself!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I decided to try a faster implementation, only to find our crc32c was
not correct! Ouch.
I removed the crc32c functions from ccan/crc, and added a new crc32c
module which has the Mark Adler x86-64-optimized variants.
We bump gossip_store version again, since csums have changed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Unfortuntely we get spurious uninitialized variable warnings with
anything but -O3 or no optimization, so set default CWARNFLAGS
appropriately.
MCP bench results without optimization:
store_load_msec:28509-31001(29206.6+/-9.4e+02)
vsz_kb:580004-580016(580006+/-4.8)
store_rewrite_sec:11.640000-12.730000(11.908+/-0.41)
listnodes_sec:1.790000-1.880000(1.83+/-0.032)
listchannels_sec:21.180000-21.950000(21.476+/-0.27)
routing_sec:2.210000-11.160000(7.126+/-3.1)
peer_write_all_sec:36.270000-41.200000(38.168+/-1.9)
MCP bench with -Og: 22% speedup vs no optimization
store_load_msec:21963-23645(22841+/-6.6e+02)
vsz_kb:579916
store_rewrite_sec:10.080000-10.960000(10.456+/-0.3)
listnodes_sec:1.280000-1.390000(1.338+/-0.047)
listchannels_sec:14.770000-16.080000(15.518+/-0.46)
routing_sec:0.990000-6.660000(3.958+/-2.2)
peer_write_all_sec:29.950000-32.950000(31.138+/-1)
MCP bench with -O2: 31% speedup vs no optimization
store_load_msec:20713-22088(21505.6+/-4.8e+02)
vsz_kb:579928
store_rewrite_sec:9.570000-11.200000(10.192+/-0.54)
listnodes_sec:0.960000-1.090000(1.028+/-0.045)
listchannels_sec:10.400000-11.770000(11.012+/-0.48)
routing_sec:0.300000-3.140000(1.978+/-1.1)
peer_write_all_sec:28.980000-30.310000(29.572+/-0.44)
MCP bench with -O3 -flto: 36% speedup vs no optimization
store_load_msec:19616-20191(19862.6+/-1.9e+02)
vsz_kb:578452
store_rewrite_sec:8.980000-9.960000(9.55+/-0.32)
listnodes_sec:0.920000-1.910000(1.18+/-0.38)
listchannels_sec:8.960000-9.450000(9.206+/-0.16)
routing_sec:0.730000-1.850000(1.438+/-0.42)
peer_write_all_sec:28.090000-29.410000(28.772+/-0.42)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In particular, `make unittest/bitcoin/test/run-secret_eq_consttime`
didn't set VALGRIND if it was set via config.vars, so runs for a *long*
time.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
These weren't checked by CI yet, and they are really short so I just added
them to the check-python target.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
We need to do it in various places, but we shouldn't do it lightly:
the primitives are there to help us get overflow handling correct.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Otherwise a straight "make install" gives:
install: cannot stat 'plugins/pay': No such file or directory
make: *** [Makefile:482: install-program] Error 1
Fixes: #2288
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is mainly just copying over the copy-editing from the
lightning-rfc repository.
[ Split to just perform changes after the UNKNOWN_PAYMENT_HASH change --RR ]
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Rusty Russell <@rustyrussell>
This is based on Christian's change, but removes all trace of the old codes.
I've proposed another spec change which removes this code altogether:
https://github.com/lightningnetwork/lightning-rfc/pull/544
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Rusty Russell <@rustyrussell>
This is mainly just copying over the copy-editing from the
lightning-rfc repository.
[ Split to just perform changes prior to the UNKNOWN_PAYMENT_HASH change --RR ]
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Rusty Russell <@rustyrussell>
git is needed to generate version information
Add a sanity check so that the build don't continue with an empty VERSION. This
is useful for sandboxed build where we might have forgot to include git.
Signed-off-by: William Casarin <jb55@jb55.com>
There were a few reports that upgrading Ubuntu recently caused issues
because we assert that the sqlite3 library version matches the one we
were built with. 'make' doesn't fix this, because it doesn't know the
external libraries have changed.
Fix this harder, with a helper which updates a file every binary depends
on, which gets relinked every time so we detect link changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that this changes the order of arguments to pipecmd to match the
documentation, so we fix all the callers!
Also make configure re-run when configurator changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This will kill a test that was running for 550 seconds, so that we
get a traceback before the travis inactivity timeout of 600 seconds
kicks in. The traceback should show us which test got stuck and where
it got stuck.
Signed-off-by: Christian Decker <@cdecker>
The only change is that the final_incorrect_htlc_amount field is now 64
bit. Since no implementation yet parses that field, we just updated it
quietly in the spec.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
After Ubuntu 18.10 upgrade, lots of new flake8 warnings.
$ flake8 --version:
3.5.0 (mccabe: 0.6.1, pycodestyle: 2.4.0, pyflakes: 1.6.0) CPython 3.6.7rc1 on Linux
Note it seems that W503 warned about line breaks before binary
operators, and W504 complains about them after. I prefer W504, so
disable W503.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was from a different series, so I just cherry-picked it.
It adds ccan/membuf as a depenency of ccan/rbuf, though we don't use
it directly yet.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If another channel has set the optional `htlc_maximum_msat` field,
we should correctly parse that field and respect it when drawing up
routes for payments.
The code to regenerate the local BOLT copy was causing eternal rebuild.
So only build if it's wrong.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
