The block files may be open when deleteFile is called. This resulted in
files not being deleted and erroring out on windows. Properly closing
the files closing the files avoids this error.
This change lets us test that we don't attempt to delete open files with
unit tests. On Windows this behavior is not allowed which results in an
error but on linux and osx it's allowed. To better test Windows
compatibilty adding this explicit check in is useful.
The testing function in export_test.go is changed to just export.go so
that callers outside the ffldb package will be able to call the
function.
The main use for this is so that the prune code can be triggered from
the blockchain package. This allows testing code to have less than
1.5GB worth of blocks to trigger the prune.
PruneBlocks used to delete files immediately before the database
transaction finished. By making the prune atomic, we can guarantee that
the database flush will happen before the utxo cache is flushed,
ensuring that the utxo cache is never in an irrecoverable state.
This change is part of the effort to add pruning support to btcd.
BeenPruned will return true if the database has ever been pruned. This
allows for accurate prune status to be reported as well as ux
improvements by disallowing the user to accidently remove or enable
indexes.
This change is part of the effort to add pruning support to btcd.
PruneBlocks will prune the earliest block files until it reaches the
given target size. The returned hashes are the hashes of the blocks
that were pruned.
This change is part of the effort to add pruning support to btcd.
scanBlockFiles nows supports scanning files from an arbitrary block
number. When blocks are pruned, the file number may not start from 0.
To account for this, scanBlockFiles now scans and retreives the start
and the end block file numbers and scans those files.
The doc formatting changes introduced in the recent go version is
increasing the diff for all of the new commits. Formatting it all in
this commit will help the readability of future PRs by reducing the
diff.
In this commit, we update all the btcutil imports to point to the new
sub-module.
In the same commit, we also modify the recently added `btcutil/go.mod`
file as we need to continue pointing to the _old_ version of btcd, until
we merge this PR and push a new tag.
Now that all headers are stored in the in-memory index, the database
bucket managed by blockchain, and in the flat files, it makes sense to
drop the redundant data from the block index bucket in ffldb.
To avoid modifying the database interface, this reimplements
FetchBlockHeader(s) to use header data stored in flat files. This can
be trivially implemented by delegating to FetchBlockRegion.
The btclog package has been changed to defining its own logging
interface (rather than seelog's) and provides a default implementation
for callers to use.
There are two primary advantages to the new logger implementation.
First, all log messages are created before the call returns. Compared
to seelog, this prevents data races when mutable variables are logged.
Second, the new logger does not implement any kind of artifical rate
limiting (what seelog refers to as "adaptive logging"). Log messages
are outputted as soon as possible and the application will appear to
perform much better when watching standard output.
Because log rotation is not a feature of the btclog logging
implementation, it is handled by the main package by importing a file
rotation package that provides an io.Reader interface for creating
output to a rotating file output. The rotator has been configured
with the same defaults that btcd previously used in the seelog config
(10MB file limits with maximum of 3 rolls) but now compresses newly
created roll files. Due to the high compressibility of log text, the
compressed files typically reduce to around 15-30% of the original
10MB file.
The github markdown interpreter has been changed such that it no longer
allows spaces in between the brackets and parenthesis of links and now
requires a newline in between anchors and other formatting. This
updates all of the markdown files accordingly.
While here, it also corrects a couple of inconsistencies in some of the
README.md files.
This is mostly a backport of some of the same modifications made in
Decred along with a few additional things cleaned up. In particular,
this updates the code to make use of the new chainhash package.
Also, since this required API changes anyways and the hash algorithm is
no longer tied specifically to SHA, all other functions throughout the
code base which had "Sha" in their name have been changed to Hash so
they are not incorrectly implying the hash algorithm.
The following is an overview of the changes:
- Remove the wire.ShaHash type
- Update all references to wire.ShaHash to the new chainhash.Hash type
- Rename the following functions and update all references:
- wire.BlockHeader.BlockSha -> BlockHash
- wire.MsgBlock.BlockSha -> BlockHash
- wire.MsgBlock.TxShas -> TxHashes
- wire.MsgTx.TxSha -> TxHash
- blockchain.ShaHashToBig -> HashToBig
- peer.ShaFunc -> peer.HashFunc
- Rename all variables that included sha in their name to include hash
instead
- Update for function name changes in other dependent packages such as
btcutil
- Update copyright dates on all modified files
- Update glide.lock file to use the required version of btcutil
