39 Commits

Author	SHA1	Message	Date
Roman Zeyde	2852020c58	Fix leftovers at examples/load.rs	2018-07-13 22:24:14 +03:00
Roman Zeyde	48cd929a99	Remove StoreOptions and enable manual compactions explicitly	2018-07-13 14:23:10 +03:00
Roman Zeyde	b9a4cb4318	Move bulk reader and indexer to separate module	2018-07-12 16:47:39 +03:00
Roman Zeyde	4c7413db9d	Increase open files limit explicitly ... Otherwise, full compaction may fail.	2018-07-12 14:44:10 +03:00
Roman Zeyde	62a78f18ad	Put "finish" marker after compaction is over	2018-07-12 10:46:33 +03:00
Roman Zeyde	f92dfc808d	Log indexed blk*.dat file path	2018-07-12 09:25:17 +03:00
Roman Zeyde	7b5679db70	Panic if DB exists before running bulk load benchmark	2018-07-11 21:26:12 +03:00
Roman Zeyde	b32e503db2	Rename regular indexing benchmark	2018-07-11 21:26:12 +03:00
Roman Zeyde	dd70a1cc8a	Persist last indexed block marker	2018-07-11 21:26:12 +03:00
Roman Zeyde	10919035bd	Rename bulk load benchmark	2018-07-11 20:44:05 +03:00
Roman Zeyde	4da521b3cd	Refactor bulk load benchmark ... - bulk writes take ~1h (reading blk*.dat files at ~48MB/s) - full compaction takes ~45m (writing 37GB of SSTables)	2018-07-11 20:42:57 +03:00
Roman Zeyde	e69686f988	Use multiple bulk index workers	2018-07-10 23:52:26 +03:00
Roman Zeyde	fb372b2915	Split reading from indexing blk*.dat files ... Also, use a single Vec<Row> for write operation	2018-07-10 15:37:56 +03:00
Roman Zeyde	4c182a0650	Fix formatting	2018-07-09 17:10:24 +03:00
Roman Zeyde	da570a9182	Fix benchmarks	2018-07-08 23:23:36 +03:00
Roman Zeyde	e4a3581da1	Use Path::join() for DB directory	2018-07-06 11:49:25 +03:00
Roman Zeyde	130ced9ae0	Format using nightly rustfmt	2018-07-03 11:46:36 +03:00
Roman Zeyde	82c2caa3af	Fix Daemon usage by benchmarks	2018-07-03 10:52:09 +03:00
Roman Zeyde	7581fd7d1a	Move bulk load helper back to electrs	2018-06-28 23:08:34 +03:00
Roman Zeyde	fd16082925	Don't start bulk loader if already done	2018-06-28 13:41:46 +03:00
Roman Zeyde	70743c9a10	Integrate bulk parser into main server binary	2018-06-28 11:15:30 +03:00
Roman Zeyde	0cceb790b9	Rename parse into bulk	2018-06-28 00:20:24 +03:00
Roman Zeyde	fe03b121f1	Update multiple use formatting	2018-06-27 00:28:50 +03:00
Roman Zeyde	1c70addede	Fix travis	2018-06-26 09:32:35 +03:00
Roman Zeyde	f631bf26af	Dedup FakeStore definition	2018-06-25 23:06:24 +03:00
Roman Zeyde	0de580a300	Perform compaction after blk*.dat parsing is over	2018-06-25 22:16:46 +03:00
Roman Zeyde	8e0a719cee	Replace eprintln!() by error!()	2018-06-25 22:16:22 +03:00
Roman Zeyde	9a5f746cc0	Refactor blk*.dat parser	2018-06-25 14:06:07 +03:00
Roman Zeyde	712c86d405	Fix small typo in bench_index	2018-06-25 10:52:44 +03:00
Roman Zeyde	4200648c8d	Refactor blk*.dat parser	2018-06-25 10:46:30 +03:00
Roman Zeyde	6f3424f734	Move block indexing into parse module	2018-06-24 13:20:09 +03:00
Roman Zeyde	8b0b4323e0	Store indexed rows into a DB	2018-06-24 09:51:18 +03:00
Roman Zeyde	b2b799c4ce	Fix logging during parsing	2018-06-23 23:36:06 +03:00
Roman Zeyde	8785f2747d	Add metrics to parse module	2018-06-23 23:25:32 +03:00
Roman Zeyde	75b3fc969d	Allow closing Channel's internal sender ... Otherwise, we may get stuck in a deadlock (e.g. parse module): - the channel's receiver waits for messages - but the last channel's sender is kept alive (at the channel)	2018-06-23 23:25:32 +03:00
Roman Zeyde	a8e26e6e33	Revert "Update formatting following Rust 1.27 update" ... This reverts commit 7372f8e239.	2018-06-23 19:57:49 +03:00
Roman Zeyde	11da7d884b	Add benchmark for blk*.dat parser ... Also, create the parser from an existing daemon	2018-06-23 16:17:04 +03:00
Roman Zeyde	7372f8e239	Update formatting following Rust 1.27 update	2018-06-22 18:18:45 +03:00
Roman Zeyde	0afde683c1	Move benchmark tool to examples/	2018-06-14 15:22:47 +03:00