This modifies the blockNode and BestState structs in the blockchain
package to store hashes directly instead of pointers to them and updates
callers to deal with the API change in the exported BestState struct.
In general, the preferred approach for hashes moving forward is to store
hash values in complex data structures, particularly those that will be
used for cache entries, and accept pointers to hashes in arguments to
functions.
Some of the reasoning behind making this change is:
- It is generally preferred to avoid storing pointers to data in cache
objects since doing so can easily lead to storing interior pointers
into other structs that then can't be GC'd
- Keeping the hash values directly in the block node provides better
cache locality
Since the code base is currently in the process of changing over to
decouple download and connection logic, but not all of the necessary
parts are updated yet, ensure blocks that are in the database, but do
not have an associated main chain block index entry, are treated as if
they do not exist for the purposes of chain connection and selection
logic.
This refactors the block index logic into a separate struct and
introduces an individual lock for it so it can be queried independent of
the chain lock.
This modifies the block node structure to include a couple of extra
fields needed to be able to reconstruct the block header from a node,
and exposes a new function from chain to fetch the block headers which
takes advantage of the new functionality to reconstruct the headers from
memory when possible. Finally, it updates both the p2p and RPC servers
to make use of the new function.
This is useful since many of the block header fields need to be kept in
order to form the block index anyways and storing the extra fields means
the database does not have to be consulted when headers are requested if
the associated node is still in memory.
The following timings show representative performance gains as measured
from one system:
new: Time to fetch 100000 headers: 59ms
old: Time to fetch 100000 headers: 4783ms
This removes the CalcPastMedianTime since it is now exposed much more
efficiently via the MedianTime field of the BestState snapshot returned
from the BestSnapshot function.
This modifies the block nodes used in the blockchain package for keeping
track of the block index to use int64 for the timestamps instead of
time.Time.
This is being done because a time.Time takes 24 bytes while an int64
only takes 8 and the plan is to eventually move the entire block index
into memory instead of the current dynamically-loaded version, so
cutting the number of bytes used for the timestamp by a third is highly
desirable.
Also, the consensus code requires working with unix-style timestamps
anyways, so switching over to them in the block node does not seem
unreasonable.
Finally, this does not go so far as to change all of the time.Time
references, particularly those that are in the public API, so it is
purely an internal change.
This modifies the rpctest harness and its associated memwallet to make
use of the new filter-based notifications since the old notifications
are now deprecated.
It also updates the glide.lock file to require the necessary
btcrpcclient version.
This modifies the blockchain code to store all blocks that have passed
proof-of-work and contextual validity tests in the database even if they
may ultimately fail to connect.
This eliminates the need to store those blocks in memory, allows them to
be available as orphans later even if they were never part of the main
chain, and helps pave the way toward being able to separate the download
logic from the connection logic.
This modifies the .travis.yml file to install a release version of glide
instead of using its latest master branch. This will prevent upstream
changes from inadvertently breaking the CI builds.
This contains a bit of cleanup and additional logic to improve the
recently-added ability to specify additional checkpoints via the
--addcheckpoint option.
In particular:
- Improve error messages in the checkpoint parsing
- Correct the mergeCheckpoints function to weed out duplicate height
checkpoints while using the most-recently provided one as described by
its comment
- Add an assertion to blockchain.New that the provided checkpoints are
sorted as required
- Keep comments to 80 columns and use two spaces after periods in them to
be consistent with the rest of the code base
- Make the entry in doc.go match the actual btcd -h output
The most recent version of glide creates a working directory named
.glide that includes .go files and thus must be excluded from the find
operation that generates merged test coverage of all packages.
A DNS lookup was being attempted on onion addresses causing
connections to fail. This has been fixed by introducing type
onionAddr (which implements a net.Addr interface) and passing
it to btcdDial.
Also, the following onion related fixes have been made:
* getaddednodeinfo - updated to handle onion addrs.
* TorLookupIP - fixed err being shadowed.
* newServer - rename tcpAddr to netAddr
* addrStringToNetAddr - skip if host is already an IP addr.
* addrStringToNetAddr - err if tor is disabled
* getaddednodeinfo - check if host is already an IP addr.
ScriptVerifyNullFail defines that signatures must be empty if a
CHECKSIG or CHECKMULTISIG operation fails.
This commit also enables ScriptVerifyNullFail at the mempool policy
level.
This updates the data driven transaction script tests to use the most
recent format and test data as implemented by Core so the test data can
more easily be updated and help prove cross-compatibility correctness.
In particular, the new format combines the previously separate valid and
invalid test data files into a single file and adds a field for the
expected result. This is a nice improvement since it means tests can
now ensure script failures are due to a specific expected reason as
opposed to only generically detecting failure as the previous format
required.
The btcd script engine typically returns more fine grained errors than
the test data expects, so the test adapter handles this by allowing
expected errors in the test data to be mapped to multiple txscript
errors.
It should also be noted that the tests related to segwit have been
stripped from the data since the segwit PR has not landed in master yet,
however the test adapter does recognize the new ability for optional
segwit data to be supplied, though it will need to properly construct
the transaction using that data when the time comes.
This converts the majority of script errors from generic errors created
via errors.New and fmt.Errorf to use a concrete type that implements the
error interface with an error code and description.
This allows callers to programmatically detect the type of error via
type assertions and an error code while still allowing the errors to
provide more context.
For example, instead of just having an error the reads "disabled opcode"
as would happen prior to these changes when a disabled opcode is
encountered, the error will now read "attempt to execute disabled opcode
OP_FOO".
While it was previously possible to programmatically detect many errors
due to them being exported, they provided no additional context and
there were also various instances that were just returning errors
created on the spot which callers could not reliably detect without
resorting to looking at the actual error message, which is nearly always
bad practice.
Also, while here, export the MaxStackSize and MaxScriptSize constants
since they can be useful for consumers of the package and perform some
minor cleanup of some of the tests.
If a host is down and doesn't send a TCP RST, the net.Dial function
blocks until the OS times out the connection. Convert to using
DialTimeout with a 30 second default timeout.
Fail fast if glide or gometalinter isn't available.
And also fail if running glide or gometalinter fails.
Before goclean.sh could pass even if glide or gometalinter wasn't
available at all. It seems that even though we use `set -ex`, their
failure was hidden either inside the $() or behind the pipe.
This adds tests which exercise the BIP0009 state transitions using the
newly available soft fork status information in the getblockchaininfo
RPC.
The following is an overview of the tests added:
- Assert the chain height is 0 and the state is ThresholdDefined
- Generate 1 fewer blocks than needed to reach the first state
transition
- Assert chain height is expected and state is still ThresholdDefined
- Generate 1 more block to reach the first state transition
- Assert chain height is expected and state moved to ThresholdStarted
- Generate enough blocks to reach the next state transition window, but
only signal support in 1 fewer than the required number to achieve
ThresholdLockedIn
- Assert chain height is expected and state is still ThresholdStarted
- Generate enough blocks to reach the next state transition window with
only the exact number of blocks required to achieve locked in status
signalling support
- Assert chain height is expected and state moved to ThresholdLockedIn
- Generate 1 fewer blocks than needed to reach the next state transition
- Assert chain height is expected and state is still ThresholdLockedIn
- Generate 1 more block to reach the next state transition
- Assert chain height is expected and state moved to ThresholdActive
In addition, it updates the existing BIP0009 mining tests to include
extra assertions that the chain height is at the expected height in
addition to checking that the bits are correctly set according to the
expected state.
This commit updates the fields of GetBlockChainInfoResult to reflect
the current state of the RPC as widely implemented by other full-node
implementations.
Replace assignments to individual fields of wire.NetAddress with
creating the entire object at once, as one would do if the type was
immutable.
In some places this replaces the creation of a NetAddress with a
high-precision timestamp with a call to a 'constructor' that converts
the timestamp to single second precision. For consistency, the tests
have also been changed to use single-precision timestamps.
Lastly, the number of allocations in readNetAddress have been reduced by
reading the services directly into the NetAddress instead of first into
a temporary variable.
The thresholdState and deploymentState functions expect the block node
for the block prior to which the threshold state is calculated, however
the startup code which checked the threshold states was using the
current best node instead of its parent.
While here, also update the comments and rename a couple of variables to
help make this fact more clear.
The CSV consensus rules dictate that the opcode fails when the
transaction version is not at least version 2, however that only applies
if the disable flag is not set in the sequence.
This is not an issue at the current time because we do not yet enforce
CSV at a consensus level, however, I noticed this discrepancy when doing
a thorough audit of the CSV paths due to the ongoing work to add full
consensus-enforced CSV support.
As a result, this must be merged prior to enabling consensus enforcement
for CSV or it would open up the potential for a hard fork.
This modifies the code to only enforce the fairly expensive BIP0030
(duplicate transcactions) checks when the chain has not yet reached the
BIP0034 activation height (and is not one of the 2 special historical
blocks that break the rule and prompted BIP0034 to being with) since
that BIP made it impossible to create duplicate coinbases and thus
removed the possibility of creating transactions that overwrite older
ones.
This is a rather large optimization because the check is expensive due
to involving a ton of cache misses in the utxoset. For example, the
following are times it took to perform the BIP0030 check on blocks
425490 - 425502 and a system with a relatively old Hitachi spinner HDD:
block 425490: 674.5857ms
block 425491: 726.5923ms
block 425492: 827.6051ms
block 425493: 680.0863ms
block 425494: 722.0917ms
block 425495: 700.0889ms
block 425496: 647.5823ms
block 425497: 445.0565ms
block 425498: 602.5765ms
block 425499: 375.0476ms
block 425500: 771.0979ms
block 425501: 461.5586ms
block 425502: 603.0766ms
As can be seen from these numbers, this reduces the block validation
time by an average of just over half a second for the given
representative data set and hardware.
Signed-off-by: Dave Collins <davec@conformal.com>
