Merge pull request #10 from lightningnetwork/feerate

BOLT 02/03: Add fee calculation details.

02-peer-protocol.md

@@ -78,7 +78,7 @@ desire to set up a new channel.
    * [8:channel-reserve-satoshis]
    * [4:htlc-minimum-msat]
    * [4:max-num-htlcs]
-   * [4:feerate-per-kb]
+   * [4:feerate-per-kw]
    * [2:to-self-delay]
    * [33:funding-pubkey]
    * [33:revocation-basepoint]
@@ -86,8 +86,8 @@ desire to set up a new channel.
    * [33:delayed-payment-basepoint]
 
 
-The `temporary-channel-id` is used to identify this channel until the funding transaction is established.  `funding-satoshis` is the amount the sender is putting into the channel.  `dust-limit-satoshis` is the threshold below which no HTLC output should be generated for this node’s commitment transaction; ie. HTLCs below this amount are not enforceable onchain. This reflects the reality that tiny outputs are not considered standard transactions and will not propagate through the bitcoin network.
-`max-htlc-value-in-inflight-msat` is a cap on total value of outstanding HTLCs, which allows a node to limit its exposure to HTLCs; similarly `max-num-htlcs` limits the number of outstanding HTLCs the other node can offer. `channel-reserve-satoshis` is the minimum amount that the other node is to keep as a direct payment. `htlc-minimum-msat` indicates the smallest value HTLC this node wil accept. `feerate-per-kb` indicates the initial fee rate which this side will pay for commitment and HTLC transactions (this can be adjusted later with a `update_fee` message).  `to-self-delay` is the number of block that the other nodes to-self outputs must be delayed, using `OP_CHECKSEQUENCEVERIFY` delays; this is how long it will have to wait in case of breakdown before redeeming its own funds.
+The `temporary-channel-id` is used to identify this channel until the funding transaction is established.  `funding-satoshis` is the amount the sender is putting into the channel.  `dust-limit-satoshis` is the threshold below which output should be generated for this node’s commitment or HTLC transaction; ie. HTLCs below this amount plus HTLC transaction fess are not enforceable onchain.  This reflects the reality that tiny outputs are not considered standard transactions and will not propagate through the bitcoin network.
+`max-htlc-value-in-inflight-msat` is a cap on total value of outstanding HTLCs, which allows a node to limit its exposure to HTLCs; similarly `max-num-htlcs` limits the number of outstanding HTLCs the other node can offer. `channel-reserve-satoshis` is the minimum amount that the other node is to keep as a direct payment. `htlc-minimum-msat` indicates the smallest value HTLC this node wil accept. `feerate-per-kw` indicates the initial fee rate which this side will pay for commitment and HTLC transactions as described in [BOLT #3](03-transactions.md#fee-calculation)(this can be adjusted later with a `update_fee` message).  `to-self-delay` is the number of block that the other nodes to-self outputs must be delayed, using `OP_CHECKSEQUENCEVERIFY` delays; this is how long it will have to wait in case of breakdown before redeeming its own funds.
 
 
 The `funding-pubkey` is the public key in the 2-of-2 multisig script of the funding transaction output.  The `revocation-basepoint` is combined with the revocation preimage for this commitment transaction to generate a unique revocation key for this commitment transaction. The `payment-basepoint` and `delayed-payment-basepoint` are similarly used to generate a series of keys for any payments to this node: `delayed-payment-basepoint` is used to for payments encumbered by a delau.  Varying these keys ensures that the transaction ID of each commitment transaction is unpredictable by an external observer, even if one commitment transaction is seen: this property is very useful for preserving privacy when outsourcing penalty transactions to third parties.
@@ -107,7 +107,7 @@ can irreversibly spend a commitment transaction output in case of
 misbehavior by the receiver.  The sender SHOULD set `minimum-depth` to
 an amount where the sender considers reorganizations to be low risk.
 `funding-pubkey`, `revocation-basepoint`, `payment-basepoint` and `delayed-payment-basepoint` MUST be valid DER-encoded
-compressed secp256k1 pubkeys. The sender SHOULD set `feerate-per-kb`
+compressed secp256k1 pubkeys. The sender SHOULD set `feerate-per-kw`
 to at least the rate it estimates would cause the transaction to be
 immediately included in a block.
 
@@ -129,7 +129,7 @@ The receiving node MAY fail the channel if it considers
 
 
 The receiver MUST fail the channel if
-considers `feerate-per-kb` too small for timely processing.  The
+considers `feerate-per-kw` too small for timely processing, or unreasonably large.  The
 receiver MUST fail the channel if `funding-pubkey`, `revocation-basepoint`, `payment-basepoint` or `delayed-payment-basepoint`
 are not be valid DER-encoded compressed secp256k1 pubkeys.
 
@@ -145,6 +145,7 @@ The *channel reserve* is specified by the peer's `channel-reserve-satoshis`; 1%
 
 The sender can unconditionally give initial funds to the receiver using a non-zero `push-msat`, and this is one case where the normal reserve mechanism doesn't apply.  However, like any other on-chain transaction, this payment is not certain until the funding transaction has been confirmed sufficiently (may be double-spent) and may require a separate method to prove payment via on-chain confirmation.
 
+The `feerate-per-kw` is generally only a concern to the sender (who pays the fees), but that is also the feerate paid by HTLC transactions; thus unresonably large fee rates can also penalize the reciepient.
 
 #### Future
 
@@ -283,7 +284,7 @@ There is a possibility of a race: the recipient can add new HTLCs
 before it receives the `update_fee`, and the sender may not be able to
 afford the fee on its own commitment transaction once the `update_fee`
 is acknowledged by the recipient.  In this case, the fee will be less
-than the fee rate.
+than the fee rate, as described in [BOLT #3](03-transactions.md#fee-calculation).
 
 The exact calculation used for deriving the fee from the fee rate is
 given in [BOLT #3].
@@ -292,7 +293,7 @@ given in [BOLT #3].
 1. type: 37 (`MSG_UPDATE_FEE`)
 2. data:
    * [8:channel-id]
-   * [4:feerate-per-kilobyte]
+   * [4:feerate-per-kw]
 
 #### Requirements
 
@@ -305,7 +306,7 @@ The node which is not responsible for paying the bitcoin fee MUST NOT
 send `update_fee`.
 
 A receiving node SHOULD fail the channel if the `update_fee` is too
-low for timely processing.
+low for timely processing, or unreasonably large.
 
 A receiving node MUST fail the channel if the sender is not
 responsible for paying the bitcoin fee.
@@ -328,8 +329,10 @@ margin, say 5x the expected fee requirement, but differing methods of
 fee estimation mean we don't specify an exact value.
 
 Since the fees are currently one-sided (the party which requested the
-channel creation always pays the fees), it is simplest to only allow
-them to set fee levels.
+channel creation always pays the fees for the commitment transaction),
+it is simplest to only allow them to set fee levels, but as the same
+fee rate applies to HTLC transactions, the receiving node must also
+care about the reasonableness of the fee.
 
 ## Channel Close
 

03-transactions.md

@@ -32,12 +32,12 @@ commitment transaction.
 
 ### Commitment Transaction Outputs
 
-The amounts for each output are rounded down to whole satoshis.  If this amount is less than the `dust-limit-satoshis` set by the owner of the commitment transaction, the output is not produced (thus the funds add to fees).
-
-To allow an opportunity for penalty transactions in case of a revoked commitment transaction, all outputs which return funds to the owner of the commitment transaction (aka "local node") must be delayed for `to-self-delay` blocks.  This delay is done in a second stage HTLC transaction.
+To allow an opportunity for penalty transactions in case of a revoked commitment transaction, all outputs which return funds to the owner of the commitment transaction (aka "local node") must be delayed for `to-self-delay` blocks.  This delay is done in a second stage HTLC transaction (HTLC-success for HTLCs accepted by the local node, HTLC-timeout for HTLCs offered by the local node).
 
 The reason for the separate transaction stage for HTLC outputs is so that HTLCs can time out or be fulfilled even though they are within the `to-self-delay` `OP_CHECKSEQUENCEVERIFY` delay.  Otherwise the required minimum timeout on HTLCs is lengthened by this delay, causing longer timeouts for HTLCs traversing the network.
 
+The amounts for each output are rounded down to whole satoshis.  If this amount, minus the fees for the HTLC transaction is less than the `dust-limit-satoshis` set by the owner of the commitment transaction, the output is not produced (thus the funds add to fees).
+
 #### To-Local Output
 
 This output sends funds back to the owner of this commitment transaction, thus must be timelocked using OP_CSV. If can be claimed, without delay, by the other party if they know the revocation key. The output is a version 0 P2WSH, with a witness script:
@@ -118,7 +118,7 @@ These HTLC transactions are almost identical, except the HTLC-Timeout transactio
    * txin[0] script bytes: 0
    * txin[0] witness stack: `<localsig> <remotesig> 0` (HTLC-Timeout) or `<localsig> <remotesig> <payment-preimage>` (HTLC-success).
 * txout count: 1
-   * txout[0] amount: the HTLC amount minus fees (see below)
+   * txout[0] amount: the HTLC amount minus fees (see [Fee Calculation](#fee-calculation)).
    * txout[0] script: version 0 P2WSH with witness script as below.
 
 The witness script for the output is:
@@ -136,6 +136,83 @@ The witness script for the output is:
 
 To spend this via penalty, the remote node uses a witness stack `<revocationsig> 1` and to collect the output the local node uses an input with nSequence `to-self-delay` and a witness stack `<local-delayedsig> 0`
 
+## Fee Calculation
+
+The fee calculation for both commitment transactions and HTLC
+transactions is based on the current `feerate-per-kw` and the
+*expected weight* of the transaction.
+
+The actual and expected weight vary for several reasons:
+* Bitcoin uses DER-encoded signatures which vary in size.
+* Bitcoin also uses variable-length integers, so a large number of outputs will take 3 bytes to encode rather than 1.
+* The `to-local` output may be below the dust limit once fees are extracted.
+
+Thus we use a simplified formula for *expected weight*, which assumes:
+* Signatures are 73 bytes long (the maximum length)
+* There is a small number of outputs (thus 1 byte to count them)
+* There is always a to-local output.
+
+The *expected weight* of a commitment transaction is calculated as follows:
+
+	transaction core: 4 + 1 + 1 + 4
+	transaction input: 32 + 4 + 1 + 4
+	transaction input witness: 1 + 74 + 74 + 1 + 1 + 34 + 34 + 1 + 1
+	transaction output: 8 + 1
+	transaction htlc output: 34
+	transaction to-local output: 34
+	transaction to-remote output: 25
+
+Multiplying non-witness data by 4, this gives a weight of:
+
+	597 + 136*num-htlc-outputs + 100*to-remote
+
+Where `to-remote` is 0 if the amount is below the local node's
+`dust-limit-satoshis`, or 1 otherwise.  `num-htlc-outputs` is the
+number of HTLCs whose amount (minus HTLC transaction fee) is greater or
+equal to the local `dust-limit-satoshis`.
+
+The *expected weight* of an HTLC transaction is calculated as follows:
+
+	transaction core: 4 + 1 + 1 + 4
+	transaction input: 32 + 4 + 1 + 4
+	transaction input witness: 1 + 74 + 74 + (1 for HTLC-timeout or 33 for HTLC-success)
+	transaction output: 8 + 1 + 34
+
+Multiplying non-witness data by 4, this gives a weight of:
+
+	526 (HTLC-timeout)
+	558 (HTLC-success)
+
+### Requirements
+
+The fee for an HTLC-timeout transaction MUST BE calculated to match:
+
+1. Multiply `feerate-per-kw` by 526 and divide by 1024 (rounding down).
+
+The fee for an HTLC-success transaction MUST BE calculated to match:
+
+1. Multiply `feerate-per-kw` by 558 and divide by 1024 (rounding down).
+
+The fee for a commitment transaction MUST BE calculated to match:
+
+1. Start with `weight` = 597, and `fee` = 0.
+
+2. If the amount to the remote node is greater or equal to the local
+   node's `dust-limit-satoshis`, add 136 to `weight`.
+   
+3. For every offered HTLC, if the HTLC amount plus the HTLC-timeout
+   transaction fee is greater or equal to the local node's
+   `dust-limit-satoshis`, then add 136 to `weight`, otherwise add
+   the HTLC amount to `fee`.
+
+4. For every accepted HTLC, if the HTLC amount plus the HTLC-success
+   transaction fee is greater or equal to the local node's
+   `dust-limit-satoshis`, then add 136 to `weight`, otherwise add
+   the HTLC amount to `fee`.
+
+5. Multiply `feerate-per-kw` by `weight`, divide by 1024 (rounding down),
+   and add to `fee`.
+
 # Key Derivation
 
 Each commitment transaction uses a unique set of keys; `<localkey>` and `<remotekey>`.  The HTLC-success and HTLC-timeout transactions use `<local-delayedkey>` and `<revocationkey>`.  These are changed every time depending on the