`rust-bitcoin v0.30.0` made some changes in this area that no longer
allow us to work with the previously exposed `U256` type. While `Work`
and `Target` (they're inverses of each other) essentially represent the
same concept, it makes more sense from their API's perspective to only
expose difficulty transitions and adjustments on `Target`s.
In general, only one request will be in flight at a time in
`lightning-block-sync`. Ideally we'd only have one connection, but
without using the `futures` mutex type.
Here we solve this narrowly for the one-request-at-a-time case by
caching the connection and takeing the connection out of the cache
while we work on it.
Some how I'd understood that `futures` had reasonable MSRV
guarantees (e.g. at least Debian stable), but apparently that isn't
actually the case, as they bumped it to upgrade to syn (with
apparently no actual features or bugfixes added as a result?) with
no minor version bump or any available alternative (unlike Tokio,
which does LTS releases).
Luckily its relatively easy to just drop the `futures` dependency -
it means a new connection for each request, which is annoying, but
certainly not the end of the world, and its easier than trying to
deal with pinning `futures`.
See https://github.com/rust-lang/futures-rs/pull/2733
`serde` doesn't bother with MSRVs, so its expected to break
frequently. Yesterday, the `derive` feature had its MSRV broken in
a patch version without care.
Luckily its trivial for us to remove the `serde` dependency in
`lightning-block-sync`, using only `serde_json` for the JSON
deserialization part. It even ends up net-negative on LoC.
Users of the RpcClient had no way to access the error code
returned by bitcoind's rpc. We embed a new RpcError struct
as the inner error for the returned io::Error. Users can access
both the code and the message using this inner struct.
This adds testing of the `futures` feature to CI. In order to avoid
introducing a dependency on `std`, we switch to using the `futures-util`
crate directly enabling only a subset of features. To this end, we also
switch to using the `select_biased!` macro, which is equivalent to
`select!` except that it doesn't choose ready futures pseudo-randomly
at runtime.
Expand the BlockSource trait to allow filtered blocks now that
chain::Listen supports them (d629a7edb7).
This makes it possible to use BIP 157/158 compact block filters with
lightning-block-sync.
The `chain::Listen` interface provides a block-connection-based
alternative to the `chain::Confirm` interface, which supports
providing transaction data at a time separate from the block
connection time.
For users who are downloading the full headers tree (e.g. from a
node over the Bitcoin P2P protocol) but who are not downloading
full blocks (e.g. because they're using BIP 157/158 filtering)
there is no API that matches exactly their event stream -
`chain::Listen` requries full blocks for each block,
`chain::Confirm` requires breaking each connection event into two
calls.
Given its incredibly trivial to take a `TransactionData` in
addition to a `Block` in `chain::Listen` we do so here, adding a
default-implementation `block_connected` which simply creates the
`TransactionData`, which ultimately all of the `chain::Listen`
implementations currently do anyway.
Closes#1128.
Querying a BlockSource is a logically immutable operation. Use non-mut
references in its interface to reflect this, which allows for users to
hold multiple references if desired.
Users who want to use lightning-block-sync's init module would
be reasonable in wanting to use it in a multithreaded environment,
however because it takes a list of listeners as dyn chain::Listen
without any Send or Sync bound they fail in doing so.
Here we make the type bounds on `chain::Listen` generic across
`chain::Listen + ?Sized`, which the existing bound of `&dyn
chain::Listen` satisfies. Thus, this is strictly less restrictive
and allows for the use of `&dyn chain::Listen + Send + Sync`.
std::io::ErrorKind is a `#[non_exhaustive]` enum as more specific
error types are to be added in the future. It was unclear in the
docs until very recently, however, that this is to be done by
re-defining `ErrorKind::Other` errors to new enum variants. Thus,
our tests which check explicitly for `ErrorKind::Other` as a
result of trying to access a directory as a file were incorrect.
Sadly, these generated no meaningful feedback from rustc at all,
except that they're suddenly failing in rustc beta!
After some back-and-forth, it seems rustc is moving forward
breaking existing code in future versions, so we move to the
"correct" check here, which is to check the raw IO error.
See rust-lang/rust#86442 and rust-lang/rust#85746 for more info.
Previously, poll::Validate was pub(crate) to force external implementors
of Poll to define their implementation in terms of ChainPoller. This was
because ChainPoller::look_up_previous_header checks for consistency
between headers and any errors would be checked at that level rather
than by the caller. Otherwise, if performed by the caller, a Poll
implementation would not be aware if the underlying BlockSource was
providing bad data and therefore couldn't react accordingly.
Widening the visibility to pub relaxes this requirement, although it's
still encourage to use ChainPoller to implement Poll. This permits
either copying or moving lightning-block-sync's test utilities to a
separate crate since they use poll::Validate.
If the HttpClient attempts to reconnect to bitcoind that is no longer
running, the client fails to get the address from the stream. Cache the
address when connecting to prevent this.
Bitcoin Core's JSON RPC server returns errors as HTTP error responses
with JSON content in the body. Parse this content as JSON to give a more
meaningful error. Otherwise, "binary" is given because the content
contains ASCII control characters.
Return an HTTP error response as a status code and contents. This allows
clients to interpret the response as desired (e.g., the contents as a
JSON-formatted error).
Early sample testing showed multiple users hitting
EWOULDBLOCK/EAGAIN waiting for an initial response from Bitcoin
Core while it was doing some long operation (eg UTXO cache
flushing). Instead of only waiting 5 seconds for each attempt, we
now wait a full two minutes, but only for the first header
response, not each byte.
