380 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Roman Zeyde	fd16082925	Don't start bulk loader if already done	2018-06-28 13:41:46 +03:00
Roman Zeyde	4537fd9636	Use higher RocksDB parallelism	2018-06-28 13:02:55 +03:00
Roman Zeyde	88fb3dc17e	Don't buffer blobs/blocks between threads	2018-06-28 11:25:36 +03:00
Roman Zeyde	8e2d4b6652	Add rayon TODO	2018-06-28 11:21:32 +03:00
Roman Zeyde	1a2bd66454	Fix formatting	2018-06-28 11:15:59 +03:00
Roman Zeyde	70743c9a10	Integrate bulk parser into main server binary	2018-06-28 11:15:30 +03:00
Roman Zeyde	13b773f6e2	Improve logging at daemon and mempool	2018-06-28 09:57:14 +03:00
Roman Zeyde	0cceb790b9	Rename parse into bulk	2018-06-28 00:20:24 +03:00
Roman Zeyde	67b79079b8	Measure indexing duration over a signle blk*.dat file	2018-06-27 12:27:57 +03:00
Roman Zeyde	c2c8e42712	Use readahead during bulk loads	2018-06-27 10:58:02 +03:00
Roman Zeyde	fe03b121f1	Update multiple use formatting	2018-06-27 00:28:50 +03:00
Roman Zeyde	35c8c1d000	Improve logging during block headers' download	2018-06-27 00:28:50 +03:00
Roman Zeyde	7832be013d	Increase intermediate and final SSTable sizes ... Otherwise, there are more than 1K files, causing full compaction to fail (due to `ulimit -n` being 1024 by default).	2018-06-26 22:53:21 +03:00
Roman Zeyde	e7e9a4bd34	Add mempool limiting TODO	2018-06-26 18:52:06 +03:00
Roman Zeyde	4b008809ff	Settubg max_open_files doesn't seem to help with full compactions	2018-06-26 15:32:59 +03:00
Roman Zeyde	1c70addede	Fix travis	2018-06-26 09:32:35 +03:00
Roman Zeyde	f631bf26af	Dedup FakeStore definition	2018-06-25 23:06:24 +03:00
Roman Zeyde	c197a51bc4	Support testnet network magic constant	2018-06-25 22:52:31 +03:00
Roman Zeyde	f003bc5afc	Stop parsing if not enough bytes for magic	2018-06-25 22:17:34 +03:00
Roman Zeyde	0de580a300	Perform compaction after blk*.dat parsing is over	2018-06-25 22:16:46 +03:00
Roman Zeyde	8e0a719cee	Replace eprintln!() by error!()	2018-06-25 22:16:22 +03:00
Roman Zeyde	4a56884da2	Use iterator adapter to find last indexed blockhash	2018-06-25 14:14:05 +03:00
Roman Zeyde	9a5f746cc0	Refactor blk*.dat parser	2018-06-25 14:06:07 +03:00
Roman Zeyde	e505eeabd3	Monitor parsed blocks indexing progress	2018-06-25 13:22:56 +03:00
Roman Zeyde	ec844fe876	Support vectorized counter metric	2018-06-25 13:22:30 +03:00
Roman Zeyde	2cf765ba5c	Make sure no block is skipped during parsing	2018-06-25 13:11:46 +03:00
Roman Zeyde	5891133528	Improve parsing logging	2018-06-25 13:11:12 +03:00
Roman Zeyde	841a800505	Support simple histogram metric	2018-06-25 13:07:07 +03:00
Roman Zeyde	712c86d405	Fix small typo in bench_index	2018-06-25 10:52:44 +03:00
Roman Zeyde	4200648c8d	Refactor blk*.dat parser	2018-06-25 10:46:30 +03:00
Roman Zeyde	2625070cef	Allow loading indexed block hashes	2018-06-25 10:46:05 +03:00
Roman Zeyde	f3d0384caa	Allow iterating on HeaderList	2018-06-25 10:44:51 +03:00
Roman Zeyde	a1ee8ee3b8	Don't collect rows into a single list before writing to DB	2018-06-24 14:24:47 +03:00
Roman Zeyde	456e38fa20	Add fake Store impl (for tests)	2018-06-24 14:15:54 +03:00
Roman Zeyde	19c63cf06f	Limit max # of RockDB open files	2018-06-24 13:20:33 +03:00
Roman Zeyde	6f3424f734	Move block indexing into parse module	2018-06-24 13:20:09 +03:00
Roman Zeyde	8b0b4323e0	Store indexed rows into a DB	2018-06-24 09:51:18 +03:00
Roman Zeyde	df3d9cf6e2	Rename data into blob at blk*.dat parser	2018-06-23 23:36:06 +03:00
Roman Zeyde	c3cbf94551	Remove unneeded use	2018-06-23 23:36:06 +03:00
Roman Zeyde	b2b799c4ce	Fix logging during parsing	2018-06-23 23:36:06 +03:00
Roman Zeyde	8785f2747d	Add metrics to parse module	2018-06-23 23:25:32 +03:00
Roman Zeyde	75b3fc969d	Allow closing Channel's internal sender ... Otherwise, we may get stuck in a deadlock (e.g. parse module): - the channel's receiver waits for messages - but the last channel's sender is kept alive (at the channel)	2018-06-23 23:25:32 +03:00
Roman Zeyde	a8e26e6e33	Revert "Update formatting following Rust 1.27 update" ... This reverts commit 7372f8e239.	2018-06-23 19:57:49 +03:00
Roman Zeyde	144bc7ba6a	Improve error message when monitoring fails	2018-06-23 16:18:41 +03:00
Roman Zeyde	11da7d884b	Add benchmark for blk*.dat parser ... Also, create the parser from an existing daemon	2018-06-23 16:17:04 +03:00
Roman Zeyde	f3a559824c	Make index_block() a public function ... So it could be used by blk*.dat parsing	2018-06-22 18:56:21 +03:00
Roman Zeyde	47a1de666a	Parse blk*.dat files concurrently ... Reading and parsing should run in separate threads.	2018-06-22 18:20:29 +03:00
Roman Zeyde	7372f8e239	Update formatting following Rust 1.27 update	2018-06-22 18:18:45 +03:00
Roman Zeyde	656eef336b	Add async parser for blk*.dat files	2018-06-21 23:56:03 +03:00
Roman Zeyde	8925857085	Move logging initialization into config.rs	2018-06-21 22:44:28 +03:00