440 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Roman Zeyde	7b5679db70	Panic if DB exists before running bulk load benchmark	2018-07-11 21:26:12 +03:00
Roman Zeyde	b32e503db2	Rename regular indexing benchmark	2018-07-11 21:26:12 +03:00
Roman Zeyde	dd70a1cc8a	Persist last indexed block marker	2018-07-11 21:26:12 +03:00
Roman Zeyde	6609d6e008	Store indexed blocks' hashes	2018-07-11 21:01:54 +03:00
Roman Zeyde	2158fa1961	Sort blk*.dat files	2018-07-11 21:00:57 +03:00
Roman Zeyde	10919035bd	Rename bulk load benchmark	2018-07-11 20:44:05 +03:00
Roman Zeyde	4da521b3cd	Refactor bulk load benchmark ... - bulk writes take ~1h (reading blk*.dat files at ~48MB/s) - full compaction takes ~45m (writing 37GB of SSTables)	2018-07-11 20:42:57 +03:00
Roman Zeyde	1f979baf8f	Increase compaction readahead to 2MB ... This should improve compaction throughput on HDDs.	2018-07-11 20:33:09 +03:00
Roman Zeyde	219a962145	Use sort_unstable_by() instead of implementing Ord for Row	2018-07-11 20:31:44 +03:00
Roman Zeyde	ef7a7eac8a	Sort rows before writing to DB ... Can happen in parallel, before sending the rows to the DB writer thread.	2018-07-11 16:26:06 +03:00
Roman Zeyde	49ec4d1c38	Support SSTable ingestion into DB	2018-07-11 13:48:57 +03:00
Roman Zeyde	e69686f988	Use multiple bulk index workers	2018-07-10 23:52:26 +03:00
Roman Zeyde	4b83c6536f	Add SSTableWriter to store module	2018-07-10 16:09:31 +03:00
Roman Zeyde	fb372b2915	Split reading from indexing blk*.dat files ... Also, use a single Vec<Row> for write operation	2018-07-10 15:37:56 +03:00
Roman Zeyde	56229409a0	Simplify SIGINT polling	2018-07-10 15:29:56 +03:00
Roman Zeyde	314978f30a	Simplify bulk indexing flow ... Use separate thread per blk*.dat file.	2018-07-10 15:28:04 +03:00
Roman Zeyde	45f3c7d178	Make store::Row sortable (by key) and clonable	2018-07-09 19:07:33 +03:00
Roman Zeyde	baf2d8624d	Rename bulk_load() -> bulk_index()	2018-07-09 18:02:49 +03:00
Roman Zeyde	e88f075c50	Use 1MB block for RocksDB	2018-07-09 17:10:49 +03:00
Roman Zeyde	4c182a0650	Fix formatting	2018-07-09 17:10:24 +03:00
Roman Zeyde	f4ac913c39	Pass blk*.dat file path during bulk indexing process	2018-07-09 16:53:40 +03:00
Roman Zeyde	89551f3e5d	Don't keep too much LOG.old.* files	2018-07-09 12:59:10 +03:00
Roman Zeyde	be20b5bd05	Update store to use pingcap RocksDB API	2018-07-08 23:24:21 +03:00
Roman Zeyde	586d6132d6	Use 'pingcap' RocksDB fork	2018-07-08 23:24:21 +03:00
Roman Zeyde	da570a9182	Fix benchmarks	2018-07-08 23:23:36 +03:00
Roman Zeyde	c82ceb6ad7	Update usage documentation for JSONRPC cookie	2018-07-08 22:22:32 +03:00
Roman Zeyde	57c67163dd	Fail connection in case of HTTP error	2018-07-08 22:17:35 +03:00
Roman Zeyde	1762523e1a	Allow setting JSONRPC cookie via command-line	2018-07-08 22:17:35 +03:00
Roman Zeyde	d1e03f33a9	Add Gitter badge	2018-07-08 14:11:48 +03:00
Roman Zeyde	a797a3864e	Move usage instructions to doc/ subdirectory	2018-07-07 12:20:04 +03:00
Roman Zeyde	6035bd164b	Read cookie as a trimmed String	2018-07-07 11:47:19 +03:00
Roman Zeyde	1475fc949d	Refactor bitcoind cookie handling	2018-07-07 11:25:25 +03:00
Roman Zeyde	93f1b85cda	Use ms precision for logging	2018-07-07 11:02:29 +03:00
Roman Zeyde	53c0c02732	Use explicit integer types for BlockchainInfo	2018-07-07 10:41:24 +03:00
Roman Zeyde	e4a3581da1	Use Path::join() for DB directory	2018-07-06 11:49:25 +03:00
Roman Zeyde	dbad1f80d0	Update TODOs	2018-07-05 12:37:25 +03:00
Roman Zeyde	f4a22fe57e	Update mempool and index before starting RPC server	2018-07-04 08:05:21 +03:00
Roman Zeyde	51a579cebe	Connection failures return Result<T> instead of crashing	2018-07-03 21:06:59 +03:00
Roman Zeyde	07bd5401d4	Remove fixed TODOs ... Deprecated by bulk indexing from blk*.dat files	2018-07-03 21:05:28 +03:00
Roman Zeyde	5326cde4d3	Update TODOs	2018-07-03 21:02:12 +03:00
Roman Zeyde	a0030fa8ed	Remove short flag for --db-dir	2018-07-03 20:54:59 +03:00
Roman Zeyde	6d17bd38fa	Add crates.io badge	2018-07-03 15:41:56 +03:00
Roman Zeyde	23b2e4ea9c	Show # of mempool transactions	2018-07-03 15:38:30 +03:00
Roman Zeyde	91f9a90ca0	Add reference to bitcoincore-indexd	2018-07-03 15:34:24 +03:00
Roman Zeyde	ee11923b66	Add "PRs welcome" badge	2018-07-03 15:32:25 +03:00
Roman Zeyde	637e34c303	Visualize bitcoind mempool fee rates	2018-07-03 15:19:49 +03:00
Roman Zeyde	130ced9ae0	Format using nightly rustfmt	2018-07-03 11:46:36 +03:00
Roman Zeyde	82c2caa3af	Fix Daemon usage by benchmarks	2018-07-03 10:52:09 +03:00
Roman Zeyde	e637ac848f	Use ~/.bitcoin as default datadir	2018-07-03 10:51:44 +03:00
Roman Zeyde	1eb92f7372	Allow specifying custom bitcoind data directory	2018-07-03 10:09:16 +03:00