This commit fixes an edge case that the sweeper's best known block
height is behind arbitrator's, which may cause an issue when creating
sweeping tx, as we may end up using an old block height from
arbitrator's view.
This commit exports and renames the following variable names:
- `PendingInput` is now `PendingInputResponse` as it's responding to a
request.
- `pendingInput` is now renamed and exported as `SweeperInput`.
- `pendingInputs` is now renamed and exported as `InputsMap`.
This commit is first made from running:
```
gofmt -d -w -r 'PendingInput -> PendingInputResponse' .
gofmt -d -w -r 'pendingInput -> SweeperInput' .
gofmt -d -w -r 'pendingInputs -> InputsMap' .
```
And followed by some docs and variable names fixes.
There's no need use the prefix `pending` as the inputs in the sweeper
can only be pending, so it's renamed, also to avoid the confusion with
the type `pendingInputs`.
This commit adds a new interface, `Bumper`, to handle RBF for a given
input set. It's responsible for creating the sweeping tx using the input
set, and monitors its confirmation status to decide whether a RBF should
be attempted or not.
We leave implementation details to future commits, and focus on mounting
this `Bumper` interface to our sweeper in this commit.
This commit changes `markInputsPendingPublish` to take `InputSet` only.
This is needed for the following commits as we won't be able to know the
tx being created beforehand, yet we still want to make sure these inputs
won't be grouped to another input set as it complicates our RBF process.
This commit makes the `ClusterInputs` directly returning the `InputSet`
so the sweeper doesn't know about the existence of `Cluster` interface.
This way we can have a deeper interface as the sweeper only needs to
interact with `Aggregator` only to get the final input sets, leaving the
implementation details being managed by `SimpleAggregator` and future
aggregators.
This commit adds a new interface `Cluster` to manage cluster-level
inputs grouping. This new interface replaces the `inputCluster` and will
be futher refactored so the sweeper can use a much smaller coin
selection lock.
This commit changes the source that drives the state changes in the
sweeper. Previously we used a ticker with default interval of 30s to
trigger sweepings periodically. The assumption is, within this 30s we'd
batch multiple inputs into one transaction to maximize profits. However,
the efficacy of this batch is questionable.
At a high level, we can put our inputs into two categories - one that's
forced, and one that's not. For forced inputs, we should sweep them
immediately as the force flag indicates they are very urgent, eg,
CPFPing the force closing tx. For non-forced inputs, such as anchors
or HTLCs with CLTV that's far away, we can wait to sweep them till a new
block comes in and triggers the sweeping process.
Eventually, all inputs will be deadline-aware, and the sweeper will
consult our fee bumper about the most economical fee rate to be used for
a given deadline. Since the deadlines here are blockstamp, it's also
easier to manage them if the sweeper is also using blockstamp instead of
timestamp.
sweeper
This commit implements a new method, `LookupInputMempoolSpend` to do
lookups in the mempool. This method is useful in the case when we only
want to know whether an input is already been spent in the mempool by
the time we call.
This commit removes the logic where we remove an input when it's been
published more than 10 times. This is needed as in our future fee
bumper, we might start with a low fee and rebroadcast the same input for
hundred of blocks.
This commit changes how a new input sweep request is handled - now we
will query the mempool and see if it's already been spent. If so, we'll
update its state as we may need to RBF this input.
This commit refactors the grouping logic into a new interface
`UtxoAggregator`, which makes it easier to write tests and opens
possibility for future customized clustering strategies.
The old clustering logic is kept as and moved into `SimpleAggregator`.
This commit adds a new interface `FeePreference` which makes it easier
to write unit tests and allows more customized implementation in
following commits.
This commit refactors the sweeper so the method `feeRateForPreference`
is now moved to `FeePreference`, which makes our following refactor
easier to handle.
This commit changes how we create the input sets which are used to
construct the sweeping transactions. Assume the sweeper has two inputs,
one is new and one is retried, we'd end up having two transactions,
- tx1: which spends both the new and old inputs.
- tx2: which spends the new inputs only.
When publishing these txes, depending on which one gets into the mempool
first, the other one will be viewed as an RBF for the first one since
they both spending the same input(the new input).
This is now fixed by only attempt to publish the second tx when there
isn't a first tx - when there is a tx1, it means the new inputs are
already used in this tx along with retried inputs, hence there's no need
to publish tx2 which spends the new inputs only.
This commit attempts to make the polling logic in sweeper more linear.
Previously, the sweep's timer is reset/restarted in multiple places,
such as when a new input comes in, or a new block comes in, or a
previous input being spent, making it difficult to follow. We now remove
the old timer and replaces it with a simple polling logic - we will
schedule sweeps every 5s(default), and if there's no input to be swept,
we'd skip, just like the previous `scheduleSweep` does.
It's also worthy noting that, although `scheduleSweep` triggers the
timer to tick, by the time we do the actual sweep in `sweepCluster`,
conditions may have changed. This is now also fixed because we only have
one place to create the clusters and sweeps.
This commit makes sure an input is only added to the cluster when it has
successfully estimated its fee rate. Previously, when an error is
returned from `feeRateForPreference`, we'd still add this input to the
cluster, resulting a **lower** fee rates being used because when
averaging the fee rates, we'd think this input has zero fee rate
specified.
An unit test is patched to make the method `clusterByLockTime` more
robust.
This commit updates the `fee()` method in `weightEstimator` to make sure
when doing CPFP we are not exceeding the max allowed fee rate. In order
to use the max fee rate, we need to modify several methods to pass the
configured value to the estimator.
We remove the publishing of the last published sweep tx during the
startup of the sweeper. This republishing can lead to situations
where funds of the default wallet might be locked for neutrino
backend clients.
Moreover all related tests are removed as well.
With this change, transactions created via craftSweepTx will be
standard. Previously, p2wsh/p2pkh scripts passed in via SendCoins would
be weighted as p2wpkh scripts. With a feerate of 1 sat/vbyte,
transactions returned would be non-standard. Luckily, the critical
sweeper subsystem only used p2wpkh scripts so this only affected
callers from the rpcserver.
Also added is an integration test that fails if SendCoins manages
to generate a non-standard transaction. All script types are now
accounted for in getWeightEstimate, which now errors if an unknown
script type is passed in.
This commit was previously split into the following parts to ease
review:
- 2d746f68: replace imports
- 4008f0fd: use ecdsa.Signature
- 849e33d1: remove btcec.S256()
- b8f6ebbd: use v2 library correctly
- fa80bca9: bump go modules
This commit updates call-sites to use the proper dust limits for
various script types. This also updates the default dust limit used
in the funding flow to be 354 satoshis instead of 573 satoshis.
