# Sweep `sweep` is a subservice that handles sweeping UTXOs back to `lnd`'s wallet. Its main purpose is to sweep back the outputs resulting from a force close transaction, although users can also call `BumpFee` to feed new unconfirmed inputs to be handled by the sweeper. In order to sweep economically, the sweeper needs to understand the time sensitivity and max fees that can be used when sweeping the inputs. This means each input must come with a deadline and a fee budget, which can be set via the RPC request or the config, otherwise the default values will be used. Once offered to the sweeper, when a new block arrives, inputs with the same deadline will be batched into a single sweeping transaction to minimize the cost. The sweeper will publish this transaction and monitor it for potential fee bumping, a process that won’t exit until the sweeping transaction is confirmed, or the specified budget has been used up. ## Understanding Budget and Deadline There are two questions when spending a UTXO - how much fees to pay and what the confirmation target is, which gives us the concepts of budget and deadline. This is especially important when sweeping the outputs of a force close transaction - some of the outputs are time-sensitive, and may result in fund loss if not confirmed in time. On the other hand, we don’t want to pay more than what we can get back - if a sweeping transaction spends more than what is meant to be swept, we are losing money due to fees. To properly handle the case, the concept `budget` and `deadline` have been introduced to `lnd` since `v0.18.0` - for each new sweeping request, the sweeper requires the caller to specify a deadline and a budget so it can make economic decisions. A fee function is then created based on the budget and deadline, which proposes a fee rate to use for the sweeping transaction. When a new block arrives, unless the transaction is confirmed or the budget is used up, the sweeper will perform a fee bump on it via RBF. ## Package Structure On a high level, a UTXO is offered to the sweeper via `SweepInput`. The sweeper keeps track of the pending inputs. When a new block arrives, it asks the `UtxoAggregator` to group all the pending inputs into batches via `ClusterInputs`. Each batch is an `InputSet`, and is sent to the `Bumper`. The `Bumper` creates a `FeeFunction` and a sweeping transaction using the `InputSet`, and monitors its confirmation status. Every time it's not confirmed when a new block arrives, the `Bumper` will perform an RBF by calling `IncreaseFeeRate` on the `FeeFunction`. ```mermaid flowchart LR subgraph SweepInput UTXO1-->sweeper UTXO2-->sweeper UTXO3-->sweeper UTXO["..."]-->sweeper sweeper end subgraph ClusterInputs sweeper-->UtxoAggregator UtxoAggregator-->InputSet1 UtxoAggregator-->InputSet2 UtxoAggregator-->InputSet["..."] end subgraph Broadcast InputSet1-->Bumper InputSet2-->Bumper InputSet-->Bumper end subgraph IncreaseFeeRate FeeFunction-->Bumper end block["new block"] ==> ClusterInputs ``` #### `UtxoAggregator` and `InputSet` `UtxoAggregator` is an interface that handles the batching of inputs. `BudgetAggregator` implements this interface by grouping inputs with the same deadline together. Inputs with the same deadline express the same time sensitivity so it makes sense to sweep them in the same transaction. Once grouped, inputs in each batch are sorted based on their budgets. The only exception is inputs with the `ExclusiveGroup` flag set, which will be swept alone. Once the batching is finished, an `InputSet` is returned, which is an interface used to decide whether a wallet UTXO is needed or not when creating the sweeping transaction. `BudgetInputSet` implements this interface by checking the sum of the output values from these inputs against the sum of their budgets - if the total budget cannot be covered, one or more wallet UTXOs are needed. For instance, commitment and HTLC transactions usually have some proportion of their outputs timelocked, preventing them from being used to pay fees immediately. For these transactions, wallet UTXOs are often needed to get them confirmed in a timely manner. #### `Bumper` `Bumper` is a transaction creator, publisher, and monitor that works on an `InputSet`. Once a sweeping transaction is created using the `InputSet`, the `Bumper` will monitor its confirmation status and attempt an RBF if the transaction is not confirmed in the next block. It relies on the `FeeFunction` to determine the new fee rate every block, and this new fee rate may or may not meet the BIP 125 fee requirements - in that case, the `Bumper` will try to perform an RBF again in the coming blocks. `TxPublisher` implements the `Bumper` interface. When a transaction is created for the first time, unless its budget has been used up, `TxPublisher` will guarantee that the initial publish meets the RBF requirements. #### `FeeFunction` `FeeFunction` is an interface that specifies a function over a starting fee rate, an ending fee rate, and a width (the deadline delta). It's used by the `Bumper` to suggest a new fee rate for bumping the sweeping transaction. `LinearFeeFunction` implements this interface using a linear function - it calculates a fee rate delta using `(ending_fee_rate - starting_fee_rate) / deadline`, and increases the fee rate by this delta value everytime a new block arrives. Once the deadline is passed, `LinearFeeFunction` will cap its returning fee rate at the ending fee rate. The starting fee rate is the estimated fee rate from the fee estimator, which is the result from calling `estimatesmartfee`(`bitcoind`), `estimatefee`(`btcd`), or `feeurl` depending on the config. This fee estimator is called using the deadline as the conf target, and the returned fee rate is used as the starting fee rate. This behavior can be overridden by setting the `--sat_per_vbyte` via `bumpfee` cli when fee bumping a specific input, which allows users to bypass the fee estimation and set the starting fee rate directly. The ending fee rate is the value from dividing the budget by the size of the sweeping transaction, and capped at the `--sweeper.maxfeerate`. The ending fee rate can be overridden by setting the `--budget` via `bumpfee` cli. For instance, suppose `lnd` is using `bitcoind` as its fee estimator, and an input with a deadline of 1000 blocks and a budget of 200,000 sats is being swept in a transaction that has a size of 500 vbytes, the fee function will be initialized with: - a starting fee rate of 10 sat/vB, which is the result from calling `estimatesmartfee 1000`. - an ending fee rate of 400 sat/vB, which is the result of `200,000/500`. - a fee rate delta of 390 sat/kvB, which is the result of `(400 - 10) / 1000 * 1000`. ## Sweeping Outputs from a Force Close Transaction A force close transaction may have the following outputs: - Commit outputs, which are the `to_local` and `to_remote` outputs. - HTLC outputs, which are the `incoming_htlc` and `outgoing_htlc` outputs. - Anchor outputs, which are the local and remote anchor outputs. #### Sweeping Commit Outputs The only output we can spend is the `to_local` output. Because it can only be spent using our signature, there’s no time pressure here. By default, the sweeper will use a deadline of 1008 blocks as the confirmation target for non-time-sensitive outputs. To overwrite the default, users can specify a value using the config `--sweeper.nodeadlineconftarget`. To specify the budget, users can use `--sweeper.budget.tolocal` to set the max allowed fees in sats, or use `--sweeper.budget.tolocalratio` to set a proportion of the `to_local` value to be used as the budget. #### Sweeping HTLC Outputs When facing a local force close transaction, HTLCs are spent in a two-stage setup - the first stage is to spend the outputs using pre-signed HTLC success/timeout transactions, the second stage is to spend the outputs from these success/timeout transactions. All these outputs are automatically handled by `lnd`. In specific, - For an incoming HTLC in stage one, the deadline is specified using its CLTV from the timeout path. This output is time-sensitive. - For an outgoing HTLC in stage one, the deadline is derived from its corresponding incoming HTLC’s CLTV. This output is time-sensitive. - For both incoming and outgoing HTLCs in stage two, because they can only be spent by us, there is no time pressure to confirm them under a deadline. When facing a remote force close transaction, HTLCs can be directly spent from the commitment transaction, and both incoming and outgoing HTLCs are time-sensitive. By default, `lnd` will use 50% of the HTLC value as its budget. To customize it, users can specify `--sweeper.budget.deadlinehtlc` and `--sweeper.budget.deadlinehtlcratio` for time-sensitive HTLCs, and `--sweeper.budget.nodeadlinehtlc` and `--sweeper.budget.nodeadlinehtlcratio` for non-time-sensitive sweeps. #### Sweeping Anchor Outputs An anchor output is a special output that functions as “anchor” to speed up the unconfirmed force closing transaction via CPFP. If the force close transaction doesn't contain any HTLCs, the anchor output is generally uneconomical to sweep and will be ignored. However, if the force close transaction does contain time-sensitive outputs (HTLCs), the anchor output will be swept to CPFP the transaction and accelerate the force close process. For CPFP-purpose anchor sweeping, the deadline is the closest deadline value of all the HTLCs on the force close transaction. The budget, however, cannot be a ratio of the anchor output because the value is too small to contribute meaningful fees (330 sats). Since its purpose is to accelerate the force close transaction so the time-sensitive outputs can be swept, the budget is actually drawn from what we call “value under protection”, which is the sum of all HTLC outputs minus the sum of their budgets. By default, 50% of this value is used as the budget, to customize it, either use `--sweeper.budget.anchorcpfp` to specify sats, or use `--sweeper.budget.anchorcpfpratio` to specify a ratio.