Duplicate entries are currently possible in the following scenario:
1: Add entries to the mapslice.
2: 1st map is full. Move onto the 2nd map.
3: Delete any entry in the first map.
4: Attempt to add an entry in the 2nd map.
When attempting (4), the entry should just be overwritten but a
duplicate gets added.
Pruning stale blocks will make the block validation fail for the block
that the prune was triggered on as BlockHeightByHash will not return the
height for blocks that are not in the main chain.
We add a test case to ensure that the test fails in the above case.
If the prune will delete block past the last flush hash of the
utxocache, the cache will need to be flushed first to avoid a case
where the utxocache is irrecoverable. The newly added code adds this
flush logic to connectBlock.
flushNeededAfterPrune returns true if the utxocache needs to be flushed
after the pruning of the given slice of block hashes. For the utxo
cache to be recoverable while pruning is enabled, we need to make sure
that there exists blocks since the last utxo cache flush. If there are
blocks that are deleted after the last utxo cache flush, the utxo set is
irrecoverable. The added method provides a way to tell if a flush is
needed.
This change is part of the effort to add utxocache support to btcd.
utxo cache is now used by the BlockChain struct. By default it's used
and the minimum cache is set to 250MiB. The change made helps speed up
block/tx validation as the cache allows for much faster lookup of utxos.
The initial block download in particular is improved as the db i/o
bottleneck is remedied by the cache.
The implemented utxocache implements connectTransactions just like
utxoviewpoint and can be used as a drop in replacement for
connectTransactions.
One thing to note is that unlike the utxoViewpoint, the utxocache
immediately deletes the spent entry from the cache. This means that the
utxocache is unfit for functions like checkConnectBlock where you expect
the entry to still exist but be marked as spent.
disconnectTransactions is purposely not implemented as using the cache
during reorganizations may leave the utxo state inconsistent if there is
an unexpected shutdown. The utxoViewpoint will still have to be used
for reorganizations.
This change is part of the effort to add utxocache support to btcd.
mapslice allows the caller to allocate a fixed amount of memory for the
utxo cache maps without the mapslice going over that fixed amount of
memory. This is useful as we can have variable sizes (1GB, 1.1GB, 2.3GB,
etc) while guaranteeing a memory limit.
