I decided to try a faster implementation, only to find our crc32c was
not correct! Ouch.
I removed the crc32c functions from ccan/crc, and added a new crc32c
module which has the Mark Adler x86-64-optimized variants.
We bump gossip_store version again, since csums have changed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need the timestamp for channel_announcement, but this is simplified
because MCP always follows the channel_announcement by a channel_update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Initialize infd to STDIN_FILENO if the input file argument is missing.
Caught with gcc version: 7.4.0
devtools/create-gossipstore.c: In function ‘main’:
devtools/create-gossipstore.c:130:9: error: ‘infd’ may be used
uninitialized in this function [-Werror=maybe-uninitialized]
while (read_all(infd, &be_inlen, sizeof(be_inlen))) {
Suggested-by: @ZmnSCPxj <https://github.com/ElementsProject/lightning/pull/2674#issuecomment-495617253>
Signed-off-by: William Casarin <jb55@jb55.com>
Save some overhead, plus gets us ready for giving subdaemons direct
store access. This is the first time we *upgrade* the gossip_store,
rather than just discarding.
The downside is that we need to add an extra message after each
channel_announcement, containing the channel capacity.
After:
store_load_msec:28337-30288(28975+/-7.4e+02)
vsz_kb:582304-582316(582306+/-4.8)
store_rewrite_sec:11.240000-11.800000(11.55+/-0.21)
listnodes_sec:1.800000-1.880000(1.84+/-0.028)
listchannels_sec:22.690000-26.260000(23.878+/-1.3)
routing_sec:2.280000-9.570000(6.842+/-2.8)
peer_write_all_sec:48.160000-51.480000(49.608+/-1.1)
Differences:
-vsz_kb:582320
+vsz_kb:582316
-listnodes_sec:2.100000-2.170000(2.118+/-0.026)
+listnodes_sec:1.800000-1.880000(1.84+/-0.028)
-peer_write_all_sec:51.600000-52.550000(52.188+/-0.34)
+peer_write_all_sec:48.160000-51.480000(49.608+/-1.1)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
[devtools/create-gossipstore.c:153]: (error) Uninitialized variable: scidsats
scidsats access is gated by csvfile, which means this warning is a false
positive. However, it's cleaner to gate scidsts on itself, rather than
the cmdline option which caused it to be populated, so do that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
added sanity check to make sure scid of csv is the same as scid in gossip.
Revised style, mem allocation, and error checks
[ Minor fixups, and updated benchmark script -- RR ]
With data.tar.gz: 456609740 Apr 2 12:33
store_load_msec:35300-42354(37118.2+/-2.7e+03)
vsz_kb:582832
store_rewrite_sec:12.700000-13.430000(12.988+/-0.27)
listnodes_sec:3.000000-3.160000(3.076+/-0.057)
listchannels_sec:30.790000-31.690000(31.03+/-0.34)
routing_sec:0.00
peer_write_all_sec:63.640000-67.860000(66.294+/-1.4)
on gcc (GCC) 7.4.0
devtools/create-gossipstore.c: In function ‘load_scid_file’:
devtools/create-gossipstore.c:22:9: error: ignoring return value of ‘fscanf’ ...
fscanf(scidfd, "%d\n", &n);
^~~~~~~~~~~~~~~~~~~~~~~~~~
devtools/create-gossipstore.c:24:9: error: ignoring return value of ‘fscanf’ ...
fscanf(scidfd, "%s\n", title);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make: *** [<builtin>: devtools/create-gossipstore.o] Error 1
Signed-off-by: William Casarin <jb55@jb55.com>
on gcc (GCC) 7.4.0
devtools/create-gossipstore.c: In function ‘main’:
devtools/create-gossipstore.c:107:9: error: ‘infd’ may be used uninitialized ..
while (read_all(infd, &be_inlen, sizeof(be_inlen))) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Signed-off-by: William Casarin <jb55@jb55.com>
