Update to bitcoind 23.x (#2466)

This release of bitcoind contains several bug fixes that let us simplify
our fee bumping logic:

- fixed a bug where bitcoind dropped non-wallet signatures
- added an option to fund transactions containing non-wallet inputs

It also has support for Taproot, which we want in eclair.

README.md

@@ -45,7 +45,7 @@ For more information please visit the [API documentation website](https://acinq.
 ## Documentation
 
 Please visit our [docs](./docs) folder to find detailed instructions on how to [configure](./docs/Configure.md) your
-node, connect to other nodes, open channels, send and receive payments, and help with more advanced scenarios. 
+node, connect to other nodes, open channels, send and receive payments, and help with more advanced scenarios.
 
 You will also find detailed [guides](./docs/Guides.md) and [frequently asked questions](./docs/FAQ.md) there.
 
@@ -57,11 +57,11 @@ Eclair relies on Bitcoin Core to interface with and monitor the blockchain and t
 
 This means that instead of re-implementing them, Eclair benefits from the verifications and optimisations (including fee management with RBF/CPFP, ...) that are implemented by Bitcoin Core. Eclair uses our own [bitcoin library](https://github.com/ACINQ/bitcoin-kmp) to verify data provided by Bitcoin Core.
 
-:warning: This also means that Eclair has strong requirements on how your Bitcoin Core node is configured (see below), and that you must back up your Bitcoin Core wallet as well as your Eclair node (see [here](#configure-bitcoin-core-wallet)): 
+:warning: This also means that Eclair has strong requirements on how your Bitcoin Core node is configured (see below), and that you must back up your Bitcoin Core wallet as well as your Eclair node (see [here](#configure-bitcoin-core-wallet)):
-- Eclair needs a _synchronized_, _segwit-ready_, 
+
-**_zeromq-enabled_**, _wallet-enabled_, _non-pruning_, _tx-indexing_ [Bitcoin Core](https://github.com/bitcoin/bitcoin) node. 
+* Eclair needs a _synchronized_, _segwit-ready_, **_zeromq-enabled_**, _wallet-enabled_, _non-pruning_, _tx-indexing_ [Bitcoin Core](https://github.com/bitcoin/bitcoin) node.
-- You must configure your Bitcoin node to use `bech32` (segwit) addresses. If your wallet has "non-segwit UTXOs" (outputs that are neither `p2sh-segwit` or `bech32`), you must send them to a `bech32` address before running Eclair.
+* You must configure your Bitcoin node to use `bech32` (segwit) addresses. If your wallet has "non-segwit UTXOs" (outputs that are neither `p2sh-segwit` or `bech32`), you must send them to a `bech32` address before running Eclair.
-- Eclair requires Bitcoin Core 0.20.1 or 0.21.1. (other versions of Bitcoin Core are *not* actively tested - use at your own risk).  If you are upgrading an existing wallet, you may need to create a new address and send all your funds to that address.
+* Eclair requires Bitcoin Core 23.0 or higher. If you are upgrading an existing wallet, you may need to create a new address and send all your funds to that address.
 
 Run bitcoind with the following minimal `bitcoin.conf`:
 
@@ -181,11 +181,12 @@ eclair-node-<version>-<commit_id>/bin/eclair-node.sh -Dlogback.configurationFile
 ### Backup
 
 You need to backup:
-- your Bitcoin Core wallet
-- your Eclair channels
 
-For Bitcoin Core, you need to backup the wallet file for the wallet that Eclair is using. You only need to do this once, when the wallet is 
+* your Bitcoin Core wallet
-created. See [Managing Wallets](https://github.com/bitcoin/bitcoin/blob/master/doc/managing-wallets.md) in the Bitcoin Core documentation for more information. 
+* your Eclair channels
+
+For Bitcoin Core, you need to backup the wallet file for the wallet that Eclair is using. You only need to do this once, when the wallet is
+created. See [Managing Wallets](https://github.com/bitcoin/bitcoin/blob/master/doc/managing-wallets.md) in the Bitcoin Core documentation for more information.
 
 For Eclair, the files that you need to backup are located in your data directory. You must backup:
 
@@ -210,7 +211,7 @@ before copying that file to your final backup location.
 ## Docker
 
 A [Dockerfile](Dockerfile) x86_64 image is built on each commit on [docker hub](https://hub.docker.com/r/acinq/eclair) for running a dockerized eclair-node.
-For arm64 platforms you can use an [arm64 Dockerfile](contrib/arm64v8.Dockerfile) to build your own arm64 container. 
+For arm64 platforms you can use an [arm64 Dockerfile](contrib/arm64v8.Dockerfile) to build your own arm64 container.
 
 You can use the `JAVA_OPTS` environment variable to set arguments to `eclair-node`.
 

docs/release-notes/eclair-vnext.md

@@ -4,6 +4,11 @@
 
 ## Major changes
 
+### Bitcoin Core 23 or higher required
+
+This release adds support for Bitcoin Core 23.x and removes support for previous Bitcoin Core versions.
+Please make sure you have updated your Bitcoin Core node before updating eclair, as eclair won't start when connected to older versions of `bitcoind`.
+
 ### Add support for channel aliases and zeroconf channels
 
 #### Channel aliases
@@ -146,7 +151,7 @@ Expired incoming invoices that are unpaid will be searched for and purged from t
 Thereafter searches for expired unpaid invoices to purge will run once every 24 hours. You can disable this feature, or
 change the search interval with two new settings:
 
-- `eclair.purge-expired-invoices.enabled = true
+- `eclair.purge-expired-invoices.enabled = true`
 - `eclair.purge-expired-invoices.interval = 24 hours`
 
 #### Skip anchor CPFP for empty commitment
@@ -164,7 +169,8 @@ If the mempool becomes congested and the feerate is too low, the commitment tran
 
 #### Public IP addresses can be DNS host names
 
-You can now specify a DNS host name as one of your `server.public-ips` addresses (see PR [#911](https://github.com/lightning/bolts/pull/911)). Note: you can not specify more than one DNS host name.
+You can now specify a DNS host name as one of your `server.public-ips` addresses (see PR [#911](https://github.com/lightning/bolts/pull/911)).
+Note: you can not specify more than one DNS host name.
 
 #### Support for testing on the signet network
 

eclair-core/pom.xml

@@ -88,9 +88,9 @@
                 <activeByDefault>true</activeByDefault>
             </activation>
             <properties>
-                <bitcoind.url>https://bitcoincore.org/bin/bitcoin-core-0.21.1/bitcoin-0.21.1-x86_64-linux-gnu.tar.gz</bitcoind.url>
+                <bitcoind.url>https://bitcoincore.org/bin/bitcoin-core-23.0/bitcoin-23.0-x86_64-linux-gnu.tar.gz</bitcoind.url>
-                <bitcoind.md5>e283a98b5e9f0b58e625e1dde661201d</bitcoind.md5>
+                <bitcoind.md5>fc4427b26a72fcc1556a9b07f3444013</bitcoind.md5>
-                <bitcoind.sha1>5101e29b39c33cc8e40d5f3b46dda37991b037a0</bitcoind.sha1>
+                <bitcoind.sha1>0fe0028a996909a46fbdde7e43aa842e419d29ce</bitcoind.sha1>
             </properties>
         </profile>
         <profile>
@@ -101,9 +101,9 @@
                 </os>
             </activation>
             <properties>
-                <bitcoind.url>https://bitcoincore.org/bin/bitcoin-core-0.21.1/bitcoin-0.21.1-osx64.tar.gz</bitcoind.url>
+                <bitcoind.url>https://bitcoincore.org/bin/bitcoin-core-23.0/bitcoin-23.0-x86_64-apple-darwin.tar.gz</bitcoind.url>
-                <bitcoind.md5>dfd1f323678eede14ae2cf6afb26ff6a</bitcoind.md5>
+                <bitcoind.md5>64f1ee12dd5da0607bfadabfa62ec8f1</bitcoind.md5>
-                <bitcoind.sha1>4273696f90a2648f90142438221f5d1ade16afa2</bitcoind.sha1>
+                <bitcoind.sha1>a0d14de1363df370dca1d31b4adecdf12a40e384</bitcoind.sha1>
             </properties>
         </profile>
         <profile>
@@ -114,9 +114,9 @@
                 </os>
             </activation>
             <properties>
-                <bitcoind.url>https://bitcoincore.org/bin/bitcoin-core-0.21.1/bitcoin-0.21.1-win64.zip</bitcoind.url>
+                <bitcoind.url>https://bitcoincore.org/bin/bitcoin-core-23.0/bitcoin-23.0-win64.zip</bitcoind.url>
-                <bitcoind.md5>1c6f5081ea68dcec7eddb9e6cdfc508d</bitcoind.md5>
+                <bitcoind.md5>000e6536eaca6c76bc518bed34cba599</bitcoind.md5>
-                <bitcoind.sha1>a782cd413fc736f05fad3831d6a9f59dde779520</bitcoind.sha1>
+                <bitcoind.sha1>1f14bb55f43ffc91185853e0240c5ef78da191be</bitcoind.sha1>
             </properties>
         </profile>
     </profiles>

eclair-core/src/main/scala/fr/acinq/eclair/Setup.scala

@@ -191,7 +191,7 @@ class Setup(val datadir: File,
     } yield (progress, ibd, chainHash, bitcoinVersion, unspentAddresses, blocks, headers)
     // blocking sanity checks
     val (progress, initialBlockDownload, chainHash, bitcoinVersion, unspentAddresses, blocks, headers) = await(future, 30 seconds, "bicoind did not respond after 30 seconds")
-    assert(bitcoinVersion >= 180000, "Eclair requires Bitcoin Core 0.18.0 or higher")
+    assert(bitcoinVersion >= 230000, "Eclair requires Bitcoin Core 23.0 or higher")
     assert(chainHash == nodeParams.chainHash, s"chainHash mismatch (conf=${nodeParams.chainHash} != bitcoind=$chainHash)")
     assert(unspentAddresses.forall(address => !isPay2PubkeyHash(address)), "Your wallet contains non-segwit UTXOs. You must send those UTXOs to a bech32 address to use Eclair (check out our README for more details).")
     if (chainHash != Block.RegtestGenesisBlock.hash) {

eclair-core/src/main/scala/fr/acinq/eclair/blockchain/bitcoind/rpc/BitcoinCoreClient.scala

@@ -469,11 +469,27 @@ class BitcoinCoreClient(val rpcClient: BitcoinJsonRPCClient) extends OnChainWall
 
 object BitcoinCoreClient {
 
-  case class FundTransactionOptions(feeRate: BigDecimal, replaceable: Boolean, lockUnspents: Boolean, changePosition: Option[Int])
+  /**
+   * When funding transactions that contain non-wallet inputs, we need to specify their maximum weight to let bitcoind
+   * compute the total weight of the (funded) transaction and set the fee accordingly.
+   */
+  case class InputWeight(txid: String, vout: Long, weight: Long)
+
+  object InputWeight {
+    def apply(outPoint: OutPoint, weight: Long): InputWeight = InputWeight(outPoint.txid.toHex, outPoint.index, weight)
+  }
+
+  case class FundTransactionOptions(feeRate: BigDecimal, replaceable: Boolean, lockUnspents: Boolean, changePosition: Option[Int], input_weights: Option[Seq[InputWeight]])
 
   object FundTransactionOptions {
-    def apply(feerate: FeeratePerKw, replaceable: Boolean = true, lockUtxos: Boolean = false, changePosition: Option[Int] = None): FundTransactionOptions = {
+    def apply(feerate: FeeratePerKw, replaceable: Boolean = true, lockUtxos: Boolean = false, changePosition: Option[Int] = None, inputWeights: Seq[InputWeight] = Nil): FundTransactionOptions = {
-      FundTransactionOptions(BigDecimal(FeeratePerKB(feerate).toLong).bigDecimal.scaleByPowerOfTen(-8), replaceable, lockUtxos, changePosition)
+      FundTransactionOptions(
+        BigDecimal(FeeratePerKB(feerate).toLong).bigDecimal.scaleByPowerOfTen(-8),
+        replaceable,
+        lockUtxos,
+        changePosition,
+        if (inputWeights.isEmpty) None else Some(inputWeights)
+      )
     }
   }
 

eclair-core/src/main/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunder.scala

@@ -22,7 +22,7 @@ import akka.actor.typed.{ActorRef, Behavior}
 import fr.acinq.bitcoin.scalacompat.{ByteVector32, OutPoint, Satoshi, Script, Transaction, TxOut}
 import fr.acinq.eclair.NotificationsLogger.NotifyNodeOperator
 import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient
-import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.FundTransactionOptions
+import fr.acinq.eclair.blockchain.bitcoind.rpc.BitcoinCoreClient.{FundTransactionOptions, InputWeight}
 import fr.acinq.eclair.blockchain.fee.FeeratePerKw
 import fr.acinq.eclair.channel.Commitments
 import fr.acinq.eclair.channel.publish.ReplaceableTxPrePublisher._
@@ -83,51 +83,11 @@ object ReplaceableTxFunder {
     }
   }
 
-  /**
-   * Adjust the amount of the change output of an anchor tx to match our target feerate.
-   * We need this because fundrawtransaction doesn't allow us to leave non-wallet inputs, so we have to add them
-   * afterwards which may bring the resulting feerate below our target.
-   */
-  def adjustAnchorOutputChange(unsignedTx: ClaimLocalAnchorWithWitnessData, commitTx: Transaction, amountIn: Satoshi, commitFeerate: FeeratePerKw, targetFeerate: FeeratePerKw, dustLimit: Satoshi): ClaimLocalAnchorWithWitnessData = {
-    require(unsignedTx.txInfo.tx.txOut.size == 1, "funded transaction should have a single change output")
-    // We take into account witness weight and adjust the fee to match our desired feerate.
-    val dummySignedClaimAnchorTx = addSigs(unsignedTx.txInfo, PlaceHolderSig)
-    // NB: we assume that our bitcoind wallet uses only P2WPKH inputs when funding txs.
-    val estimatedWeight = commitTx.weight() + dummySignedClaimAnchorTx.tx.weight() + claimP2WPKHOutputWitnessWeight * (dummySignedClaimAnchorTx.tx.txIn.size - 1)
-    val targetFee = weight2fee(targetFeerate, estimatedWeight) - weight2fee(commitFeerate, commitTx.weight())
-    val amountOut = dustLimit.max(amountIn - targetFee)
-    val updatedAnchorTx = unsignedTx.updateTx(unsignedTx.txInfo.tx.copy(txOut = Seq(unsignedTx.txInfo.tx.txOut.head.copy(amount = amountOut))))
-    updatedAnchorTx
-  }
-
   private def dummySignedCommitTx(commitments: Commitments): CommitTx = {
     val unsignedCommitTx = commitments.localCommit.commitTxAndRemoteSig.commitTx
     addSigs(unsignedCommitTx, PlaceHolderPubKey, PlaceHolderPubKey, PlaceHolderSig, PlaceHolderSig)
   }
 
-  /**
-   * Adjust the change output of an htlc tx to match our target feerate.
-   * We need this because fundrawtransaction doesn't allow us to leave non-wallet inputs, so we have to add them
-   * afterwards which may bring the resulting feerate below our target.
-   */
-  def adjustHtlcTxChange(unsignedTx: HtlcWithWitnessData, amountIn: Satoshi, targetFeerate: FeeratePerKw, dustLimit: Satoshi, commitmentFormat: CommitmentFormat): HtlcWithWitnessData = {
-    require(unsignedTx.txInfo.tx.txOut.size <= 2, "funded transaction should have at most one change output")
-    val dummySignedTx = unsignedTx.txInfo match {
-      case tx: HtlcSuccessTx => addSigs(tx, PlaceHolderSig, PlaceHolderSig, ByteVector32.Zeroes, commitmentFormat)
-      case tx: HtlcTimeoutTx => addSigs(tx, PlaceHolderSig, PlaceHolderSig, commitmentFormat)
-    }
-    // We adjust the change output to obtain the targeted feerate.
-    val estimatedWeight = dummySignedTx.tx.weight() + claimP2WPKHOutputWitnessWeight * (dummySignedTx.tx.txIn.size - 1)
-    val targetFee = weight2fee(targetFeerate, estimatedWeight)
-    val changeAmount = amountIn - dummySignedTx.tx.txOut.head.amount - targetFee
-    val updatedHtlcTx = if (dummySignedTx.tx.txOut.length == 2 && changeAmount >= dustLimit) {
-      unsignedTx.updateTx(unsignedTx.txInfo.tx.copy(txOut = Seq(unsignedTx.txInfo.tx.txOut.head, unsignedTx.txInfo.tx.txOut.last.copy(amount = changeAmount))))
-    } else {
-      unsignedTx.updateTx(unsignedTx.txInfo.tx.copy(txOut = Seq(unsignedTx.txInfo.tx.txOut.head)))
-    }
-    updatedHtlcTx
-  }
-
   /**
    * Adjust the main output of a claim-htlc tx to match our target feerate.
    * If the resulting output is too small, we skip the transaction.
@@ -174,7 +134,8 @@ object ReplaceableTxFunder {
         val commitTx = dummySignedCommitTx(commitments)
         val totalWeight = previousTx.signedTx.weight() + commitTx.tx.weight()
         weight2fee(targetFeerate, totalWeight) - commitTx.fee
-      case _ => weight2fee(targetFeerate, previousTx.signedTx.weight())
+      case _ =>
+        weight2fee(targetFeerate, previousTx.signedTx.weight())
     }
     previousTx.signedTxWithWitnessData match {
       case claimLocalAnchor: ClaimLocalAnchorWithWitnessData =>
@@ -353,23 +314,10 @@ private class ReplaceableTxFunder(nodeParams: NodeParams,
   }
 
   def signWalletInputs(locallySignedTx: ReplaceableTxWithWalletInputs, txFeerate: FeeratePerKw, amountIn: Satoshi): Behavior[Command] = {
-    locallySignedTx match {
+    val inputInfo = BitcoinCoreClient.PreviousTx(locallySignedTx.txInfo.input, locallySignedTx.txInfo.tx.txIn.head.witness)
-      case ClaimLocalAnchorWithWitnessData(anchorTx) =>
+    context.pipeToSelf(bitcoinClient.signTransaction(locallySignedTx.txInfo.tx, Seq(inputInfo))) {
-        val commitInfo = BitcoinCoreClient.PreviousTx(anchorTx.input, anchorTx.tx.txIn.head.witness)
+      case Success(signedTx) => SignWalletInputsOk(signedTx.tx)
-        context.pipeToSelf(bitcoinClient.signTransaction(anchorTx.tx, Seq(commitInfo))) {
+      case Failure(reason) => SignWalletInputsFailed(reason)
-          case Success(signedTx) => SignWalletInputsOk(signedTx.tx)
-          case Failure(reason) => SignWalletInputsFailed(reason)
-        }
-      case htlcTx: HtlcWithWitnessData =>
-        val inputInfo = BitcoinCoreClient.PreviousTx(htlcTx.txInfo.input, htlcTx.txInfo.tx.txIn.head.witness)
-        context.pipeToSelf(bitcoinClient.signTransaction(htlcTx.txInfo.tx, Seq(inputInfo), allowIncomplete = true).map(signTxResponse => {
-          // NB: bitcoind versions older than 0.21.1 messes up the witness stack for our htlc input, so we need to restore it.
-          // See https://github.com/bitcoin/bitcoin/issues/21151
-          htlcTx.txInfo.tx.copy(txIn = htlcTx.txInfo.tx.txIn.head +: signTxResponse.tx.txIn.tail)
-        })) {
-          case Success(signedTx) => SignWalletInputsOk(signedTx)
-          case Failure(reason) => SignWalletInputsFailed(reason)
-        }
     }
     Behaviors.receiveMessagePartial {
       case SignWalletInputsOk(signedTx) =>
@@ -405,75 +353,46 @@ private class ReplaceableTxFunder(nodeParams: NodeParams,
 
   private def addInputs(anchorTx: ClaimLocalAnchorWithWitnessData, targetFeerate: FeeratePerKw, commitments: Commitments): Future[(ClaimLocalAnchorWithWitnessData, Satoshi)] = {
     val dustLimit = commitments.localParams.dustLimit
-    val commitFeerate = commitments.localCommit.spec.commitTxFeerate
     val commitTx = dummySignedCommitTx(commitments).tx
-    // We want the feerate of the package (commit tx + tx spending anchor) to equal targetFeerate.
-    // Thus we have: anchorFeerate = targetFeerate + (weight-commit-tx / weight-anchor-tx) * (targetFeerate - commitTxFeerate)
-    // If we use the smallest weight possible for the anchor tx, the feerate we use will thus be greater than what we want,
-    // and we can adjust it afterwards by raising the change output amount.
-    val anchorFeerate = targetFeerate + FeeratePerKw(targetFeerate.feerate - commitFeerate.feerate) * commitTx.weight() / claimAnchorOutputMinWeight
     // NB: fundrawtransaction requires at least one output, and may add at most one additional change output.
     // Since the purpose of this transaction is just to do a CPFP, the resulting tx should have a single change output
-    // (note that bitcoind doesn't let us publish a transaction with no outputs).
+    // (note that bitcoind doesn't let us publish a transaction with no outputs). To work around these limitations, we
-    // To work around these limitations, we start with a dummy output and later merge that dummy output with the optional
+    // start with a dummy output and later merge that dummy output with the optional change output added by bitcoind.
-    // change output added by bitcoind.
+    val txNotFunded = anchorTx.txInfo.tx.copy(txOut = TxOut(dustLimit, Script.pay2wpkh(PlaceHolderPubKey)) :: Nil)
-    // NB: fundrawtransaction doesn't support non-wallet inputs, so we have to remove our anchor input and re-add it later.
+    // The anchor transaction is paying for the weight of the commitment transaction.
-    // That means bitcoind will not take our anchor input's weight into account when adding inputs to set the fee.
+    val anchorWeight = Seq(InputWeight(anchorTx.txInfo.input.outPoint, anchorInputWeight + commitTx.weight()))
-    // That's ok, we can increase the fee later by decreasing the output amount. But we need to ensure we'll have enough
+    bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(targetFeerate, lockUtxos = true, inputWeights = anchorWeight)).flatMap(fundTxResponse => {
-    // to cover the weight of our anchor input, which is why we set it to the following value.
-    val dummyChangeAmount = weight2fee(anchorFeerate, claimAnchorOutputMinWeight) + dustLimit
-    val txNotFunded = Transaction(2, Nil, TxOut(dummyChangeAmount, Script.pay2wpkh(PlaceHolderPubKey)) :: Nil, 0)
-    bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(anchorFeerate, lockUtxos = true)).flatMap(fundTxResponse => {
       // We merge the outputs if there's more than one.
       fundTxResponse.changePosition match {
         case Some(changePos) =>
           val changeOutput = fundTxResponse.tx.txOut(changePos)
-          val txSingleOutput = fundTxResponse.tx.copy(txOut = Seq(changeOutput.copy(amount = changeOutput.amount + dummyChangeAmount)))
+          val txSingleOutput = fundTxResponse.tx.copy(txOut = Seq(changeOutput))
-          Future.successful(fundTxResponse.copy(tx = txSingleOutput))
+          // We ask bitcoind to sign the wallet inputs to learn their final weight and adjust the change amount.
+          bitcoinClient.signTransaction(txSingleOutput, allowIncomplete = true).map(signTxResponse => {
+            val dummySignedTx = addSigs(anchorTx.updateTx(signTxResponse.tx).txInfo, PlaceHolderSig)
+            val packageWeight = commitTx.weight() + dummySignedTx.tx.weight()
+            val anchorTxFee = weight2fee(targetFeerate, packageWeight) - weight2fee(commitments.localCommit.spec.commitTxFeerate, commitTx.weight())
+            val changeAmount = dustLimit.max(fundTxResponse.amountIn - anchorTxFee)
+            val fundedTx = fundTxResponse.tx.copy(txOut = Seq(changeOutput.copy(amount = changeAmount)))
+            (anchorTx.updateTx(fundedTx), fundTxResponse.amountIn)
+          })
         case None =>
           bitcoinClient.getChangeAddress().map(pubkeyHash => {
-            val txSingleOutput = fundTxResponse.tx.copy(txOut = Seq(TxOut(dummyChangeAmount, Script.pay2wpkh(pubkeyHash))))
+            val fundedTx = fundTxResponse.tx.copy(txOut = Seq(TxOut(dustLimit, Script.pay2wpkh(pubkeyHash))))
-            fundTxResponse.copy(tx = txSingleOutput)
+            (anchorTx.updateTx(fundedTx), fundTxResponse.amountIn)
           })
       }
-    }).map(fundTxResponse => {
-      require(fundTxResponse.tx.txOut.size == 1, "funded transaction should have a single change output")
-      // NB: we insert the anchor input in the *first* position because our signing helpers only sign input #0.
-      val unsignedTx = anchorTx.updateTx(fundTxResponse.tx.copy(txIn = anchorTx.txInfo.tx.txIn.head +: fundTxResponse.tx.txIn))
-      val totalAmountIn = fundTxResponse.amountIn + AnchorOutputsCommitmentFormat.anchorAmount
-      (adjustAnchorOutputChange(unsignedTx, commitTx, totalAmountIn, commitFeerate, targetFeerate, dustLimit), totalAmountIn)
     })
   }
 
   private def addInputs(htlcTx: HtlcWithWitnessData, targetFeerate: FeeratePerKw, commitments: Commitments): Future[(HtlcWithWitnessData, Satoshi)] = {
-    // NB: fundrawtransaction doesn't support non-wallet inputs, so we clear the input and re-add it later.
+    val htlcInputWeight = Seq(InputWeight(htlcTx.txInfo.input.outPoint, htlcTx.txInfo match {
-    val txNotFunded = htlcTx.txInfo.tx.copy(txIn = Nil, txOut = htlcTx.txInfo.tx.txOut.head.copy(amount = commitments.localParams.dustLimit) :: Nil)
+      case _: HtlcSuccessTx => commitments.commitmentFormat.htlcSuccessInputWeight
-    val htlcTxWeight = htlcTx.txInfo match {
+      case _: HtlcTimeoutTx => commitments.commitmentFormat.htlcTimeoutInputWeight
-      case _: HtlcSuccessTx => commitments.commitmentFormat.htlcSuccessWeight
+    }))
-      case _: HtlcTimeoutTx => commitments.commitmentFormat.htlcTimeoutWeight
+    bitcoinClient.fundTransaction(htlcTx.txInfo.tx, FundTransactionOptions(targetFeerate, lockUtxos = true, changePosition = Some(1), inputWeights = htlcInputWeight)).map(fundTxResponse => {
-    }
+      val unsignedTx = htlcTx.updateTx(fundTxResponse.tx)
-    // We want the feerate of our final HTLC tx to equal targetFeerate. However, we removed the HTLC input from what we
+      (unsignedTx, fundTxResponse.amountIn)
-    // send to fundrawtransaction, so bitcoind will not know the total weight of the final tx. In order to make up for
-    // this difference, we need to tell bitcoind to target a higher feerate that takes into account the weight of the
-    // input we removed.
-    // That feerate will satisfy the following equality:
-    // feerate * weight_seen_by_bitcoind = target_feerate * (weight_seen_by_bitcoind + htlc_input_weight)
-    // So: feerate = target_feerate * (1 + htlc_input_weight / weight_seen_by_bitcoind)
-    // Because bitcoind will add at least one P2WPKH input, weight_seen_by_bitcoind >= htlc_tx_weight + p2wpkh_weight
-    // Thus: feerate <= target_feerate * (1 + htlc_input_weight / (htlc_tx_weight + p2wpkh_weight))
-    // NB: we don't take into account the fee paid by our HTLC input: we will take it into account when we adjust the
-    // change output amount (unless bitcoind didn't add any change output, in that case we will overpay the fee slightly).
-    val weightRatio = 1.0 + (htlcInputMaxWeight.toDouble / (htlcTxWeight + claimP2WPKHOutputWeight))
-    bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(targetFeerate * weightRatio, lockUtxos = true, changePosition = Some(1))).map(fundTxResponse => {
-      // We add the HTLC input (from the commit tx) and restore the HTLC output.
-      // NB: we can't modify them because they are signed by our peer (with SIGHASH_SINGLE | SIGHASH_ANYONECANPAY).
-      val txWithHtlcInput = fundTxResponse.tx.copy(
-        txIn = htlcTx.txInfo.tx.txIn ++ fundTxResponse.tx.txIn,
-        txOut = htlcTx.txInfo.tx.txOut ++ fundTxResponse.tx.txOut.tail
-      )
-      val unsignedTx = htlcTx.updateTx(txWithHtlcInput)
-      val totalAmountIn = fundTxResponse.amountIn + unsignedTx.txInfo.amountIn
-      (adjustHtlcTxChange(unsignedTx, totalAmountIn, targetFeerate, commitments.localParams.dustLimit, commitments.commitmentFormat), totalAmountIn)
     })
   }
 

eclair-core/src/main/scala/fr/acinq/eclair/transactions/Scripts.scala

@@ -16,12 +16,12 @@
 
 package fr.acinq.eclair.transactions
 
+import fr.acinq.bitcoin.Script.LOCKTIME_THRESHOLD
+import fr.acinq.bitcoin.SigHash._
+import fr.acinq.bitcoin.TxIn.{SEQUENCE_LOCKTIME_DISABLE_FLAG, SEQUENCE_LOCKTIME_MASK, SEQUENCE_LOCKTIME_TYPE_FLAG}
 import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey
 import fr.acinq.bitcoin.scalacompat.Script._
 import fr.acinq.bitcoin.scalacompat._
-import fr.acinq.bitcoin.SigHash._
-import fr.acinq.bitcoin.TxIn.{SEQUENCE_LOCKTIME_DISABLE_FLAG, SEQUENCE_LOCKTIME_TYPE_FLAG, SEQUENCE_LOCKTIME_MASK}
-import fr.acinq.bitcoin.Script.LockTimeThreshold
 import fr.acinq.eclair.transactions.Transactions.{AnchorOutputsCommitmentFormat, CommitmentFormat, DefaultCommitmentFormat}
 import fr.acinq.eclair.{BlockHeight, CltvExpiry, CltvExpiryDelta}
 import scodec.bits.ByteVector
@@ -85,7 +85,7 @@ object Scripts {
    * @return the block height before which this tx cannot be published.
    */
   def cltvTimeout(tx: Transaction): BlockHeight =
-    if (tx.lockTime <= LockTimeThreshold) {
+    if (tx.lockTime <= LOCKTIME_THRESHOLD) {
       // locktime is a number of blocks
       BlockHeight(tx.lockTime)
     }

eclair-core/src/main/scala/fr/acinq/eclair/transactions/Transactions.scala

@@ -16,12 +16,12 @@
 
 package fr.acinq.eclair.transactions
 
+import fr.acinq.bitcoin.ScriptFlags
+import fr.acinq.bitcoin.SigHash._
+import fr.acinq.bitcoin.SigVersion._
 import fr.acinq.bitcoin.scalacompat.Crypto.{PrivateKey, PublicKey, ripemd160}
 import fr.acinq.bitcoin.scalacompat.Script._
 import fr.acinq.bitcoin.scalacompat._
-import fr.acinq.bitcoin.SigHash._
-import fr.acinq.bitcoin.SigVersion._
-import fr.acinq.bitcoin.ScriptFlags
 import fr.acinq.eclair._
 import fr.acinq.eclair.blockchain.fee.FeeratePerKw
 import fr.acinq.eclair.transactions.CommitmentOutput._
@@ -45,6 +45,8 @@ object Transactions {
     def htlcOutputWeight: Int
     def htlcTimeoutWeight: Int
     def htlcSuccessWeight: Int
+    def htlcTimeoutInputWeight: Int
+    def htlcSuccessInputWeight: Int
     // @formatter:on
   }
 
@@ -56,6 +58,8 @@ object Transactions {
     override val htlcOutputWeight = 172
     override val htlcTimeoutWeight = 663
     override val htlcSuccessWeight = 703
+    override val htlcTimeoutInputWeight = 449
+    override val htlcSuccessInputWeight = 488
   }
 
   /**
@@ -67,6 +71,8 @@ object Transactions {
     override val htlcOutputWeight = 172
     override val htlcTimeoutWeight = 666
     override val htlcSuccessWeight = 706
+    override val htlcTimeoutInputWeight = 452
+    override val htlcSuccessInputWeight = 491
   }
 
   object AnchorOutputsCommitmentFormat {
@@ -196,16 +202,12 @@ object Transactions {
   /**
    * these values are specific to us (not defined in the specification) and used to estimate fees
    */
-  val claimP2WPKHOutputWitnessWeight = 109
   val claimP2WPKHOutputWeight = 438
+  val anchorInputWeight = 279
   // The smallest transaction that spends an anchor contains 2 inputs (the commit tx output and a wallet input to set the feerate)
   // and 1 output (change). If we're using P2WPKH wallet inputs/outputs with 72 bytes signatures, this results in a weight of 717.
   // We round it down to 700 to allow for some error margin (e.g. signatures smaller than 72 bytes).
   val claimAnchorOutputMinWeight = 700
-  // The biggest htlc input is an HTLC-success with anchor outputs:
-  // 143 bytes (accepted_htlc_script) + 327 bytes (success_witness) + 41 bytes (commitment_input) = 511 bytes
-  // See https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md#expected-weight-of-htlc-timeout-and-htlc-success-transactions
-  val htlcInputMaxWeight = 511
   val htlcDelayedWeight = 483
   val claimHtlcSuccessWeight = 571
   val claimHtlcTimeoutWeight = 545
@@ -292,7 +294,7 @@ object Transactions {
 
   /**
    * @param commitTxNumber         commit tx number
-   * @param isInitiator               true if local node initiated the channel open
+   * @param isInitiator            true if local node initiated the channel open
    * @param localPaymentBasePoint  local payment base point
    * @param remotePaymentBasePoint remote payment base point
    * @return the obscured tx number as defined in BOLT #3 (a 48 bits integer)
@@ -310,7 +312,7 @@ object Transactions {
 
   /**
    * @param commitTx               commit tx
-   * @param isInitiator               true if local node initiated the channel open
+   * @param isInitiator            true if local node initiated the channel open
    * @param localPaymentBasePoint  local payment base point
    * @param remotePaymentBasePoint remote payment base point
    * @return the actual commit tx number that was blinded and stored in locktime and sequence fields

eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoinCoreClientSpec.scala

@@ -22,6 +22,7 @@ import akka.testkit.TestProbe
 import fr.acinq.bitcoin
 import fr.acinq.bitcoin.scalacompat.Crypto.PublicKey
 import fr.acinq.bitcoin.scalacompat.{Block, BtcDouble, ByteVector32, MilliBtcDouble, OutPoint, Satoshi, SatoshiLong, Script, ScriptWitness, Transaction, TxIn, TxOut}
+import fr.acinq.bitcoin.{SigHash, SigVersion}
 import fr.acinq.eclair.blockchain.OnChainWallet.{FundTransactionResponse, MakeFundingTxResponse, OnChainBalance, SignTransactionResponse}
 import fr.acinq.eclair.blockchain.WatcherSpec.{createSpendManyP2WPKH, createSpendP2WPKH}
 import fr.acinq.eclair.blockchain.bitcoind.BitcoindService.BitcoinReq
@@ -82,8 +83,12 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A
       bitcoinClient.fundTransaction(txNotFunded, FundTransactionOptions(TestConstants.feeratePerKw)).pipeTo(sender.ref)
       val fundTxResponse = sender.expectMsgType[FundTransactionResponse]
       assert(fundTxResponse.changePosition.nonEmpty)
-      assert(fundTxResponse.amountIn > 0.sat)
       assert(fundTxResponse.fee > 0.sat)
+      val amountIn = fundTxResponse.tx.txIn.map(txIn => {
+        bitcoinClient.getTransaction(txIn.outPoint.txid).pipeTo(sender.ref)
+        sender.expectMsgType[Transaction].txOut(txIn.outPoint.index.toInt).amount
+      }).sum
+      assert(amountIn == fundTxResponse.amountIn)
       fundTxResponse.tx.txIn.foreach(txIn => assert(txIn.signatureScript.isEmpty && txIn.witness.isNull))
       fundTxResponse.tx.txIn.foreach(txIn => assert(txIn.sequence == bitcoin.TxIn.SEQUENCE_FINAL - 2))
 
@@ -115,7 +120,7 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A
       val fundTxResponse1 = sender.expectMsgType[FundTransactionResponse]
       bitcoinClient.fundTransaction(fundTxResponse1.tx, FundTransactionOptions(TestConstants.feeratePerKw * 2)).pipeTo(sender.ref)
       val fundTxResponse2 = sender.expectMsgType[FundTransactionResponse]
-      assert(fundTxResponse1.tx !== fundTxResponse2.tx)
+      assert(fundTxResponse1.tx != fundTxResponse2.tx)
       assert(fundTxResponse1.fee < fundTxResponse2.fee)
     }
     {
@@ -124,13 +129,152 @@ class BitcoinCoreClientSpec extends TestKitBaseClass with BitcoindService with A
       bitcoinClient.fundTransaction(txManyOutputs, FundTransactionOptions(TestConstants.feeratePerKw, replaceable = false, changePosition = Some(1))).pipeTo(sender.ref)
       val fundTxResponse = sender.expectMsgType[FundTransactionResponse]
       assert(fundTxResponse.tx.txOut.size == 3)
-      assert(fundTxResponse.changePosition == Some(1))
+      assert(fundTxResponse.changePosition.contains(1))
       assert(!Set(230000 sat, 410000 sat).contains(fundTxResponse.tx.txOut(1).amount))
       assert(Set(230000 sat, 410000 sat) == Set(fundTxResponse.tx.txOut.head.amount, fundTxResponse.tx.txOut.last.amount))
       fundTxResponse.tx.txIn.foreach(txIn => assert(txIn.sequence == bitcoin.TxIn.SEQUENCE_FINAL - 1))
     }
   }
 
+  test("fund transactions with external inputs") {
+    val sender = TestProbe()
+    val defaultWallet = new BitcoinCoreClient(bitcoinrpcclient)
+    val walletExternalFunds = new BitcoinCoreClient(createWallet("external_inputs", sender))
+
+    // We receive some funds on an address that belongs to our wallet.
+    Seq(25 millibtc, 15 millibtc, 20 millibtc).foreach(amount => {
+      walletExternalFunds.getReceiveAddress().pipeTo(sender.ref)
+      val walletAddress = sender.expectMsgType[String]
+      defaultWallet.sendToAddress(walletAddress, amount, 1).pipeTo(sender.ref)
+      sender.expectMsgType[ByteVector32]
+    })
+
+    // We receive more funds on an address that does not belong to our wallet.
+    val externalInputWeight = 310
+    val (alicePriv, bobPriv, carolPriv) = (randomKey(), randomKey(), randomKey())
+    val (outpoint1, inputScript1) = {
+      val script = Script.createMultiSigMofN(1, Seq(alicePriv.publicKey, bobPriv.publicKey))
+      val txNotFunded = Transaction(2, Nil, Seq(TxOut(250_000 sat, Script.pay2wsh(script))), 0)
+      defaultWallet.fundTransaction(txNotFunded, FundTransactionOptions(FeeratePerKw(2500 sat), changePosition = Some(1))).pipeTo(sender.ref)
+      val fundedTx = sender.expectMsgType[FundTransactionResponse].tx
+      defaultWallet.signTransaction(fundedTx, Nil).pipeTo(sender.ref)
+      val signedTx = sender.expectMsgType[SignTransactionResponse].tx
+      defaultWallet.publishTransaction(signedTx).pipeTo(sender.ref)
+      sender.expectMsg(signedTx.txid)
+      (OutPoint(signedTx, 0), script)
+    }
+
+    // We make sure these utxos are confirmed.
+    generateBlocks(1)
+
+    // We're able to spend those funds and ask bitcoind to fund the corresponding transaction.
+    val (tx2, inputScript2) = {
+      val targetFeerate = FeeratePerKw(5000 sat)
+      val outputScript = Script.createMultiSigMofN(1, Seq(alicePriv.publicKey, carolPriv.publicKey))
+      val txNotFunded = Transaction(2, Seq(TxIn(outpoint1, Nil, 0)), Seq(TxOut(300_000 sat, Script.pay2wsh(outputScript))), 0)
+      val smallExternalInputWeight = 200
+      assert(smallExternalInputWeight < externalInputWeight)
+      walletExternalFunds.fundTransaction(txNotFunded, FundTransactionOptions(targetFeerate, inputWeights = Seq(InputWeight(outpoint1, smallExternalInputWeight)), changePosition = Some(1))).pipeTo(sender.ref)
+      val fundedTx1 = sender.expectMsgType[FundTransactionResponse]
+      assert(fundedTx1.tx.txIn.length >= 2)
+      val amountIn1 = fundedTx1.tx.txIn.map(txIn => {
+        defaultWallet.getTransaction(txIn.outPoint.txid).pipeTo(sender.ref)
+        sender.expectMsgType[Transaction].txOut(txIn.outPoint.index.toInt).amount
+      }).sum
+      assert(amountIn1 == fundedTx1.amountIn)
+      // If we specify a bigger weight, bitcoind uses a bigger fee.
+      walletExternalFunds.fundTransaction(txNotFunded, FundTransactionOptions(targetFeerate, inputWeights = Seq(InputWeight(outpoint1, externalInputWeight)), changePosition = Some(1))).pipeTo(sender.ref)
+      val fundedTx2 = sender.expectMsgType[FundTransactionResponse]
+      assert(fundedTx2.tx.txIn.length >= 2)
+      assert(fundedTx1.fee < fundedTx2.fee)
+      val amountIn2 = fundedTx2.tx.txIn.map(txIn => {
+        defaultWallet.getTransaction(txIn.outPoint.txid).pipeTo(sender.ref)
+        sender.expectMsgType[Transaction].txOut(txIn.outPoint.index.toInt).amount
+      }).sum
+      assert(amountIn2 == fundedTx2.amountIn)
+      // We sign our external input.
+      val externalSig = Transaction.signInput(fundedTx2.tx, 0, inputScript1, SigHash.SIGHASH_ALL, 250_000 sat, SigVersion.SIGVERSION_WITNESS_V0, alicePriv)
+      val partiallySignedTx = fundedTx2.tx.updateWitness(0, Script.witnessMultiSigMofN(Seq(alicePriv, bobPriv).map(_.publicKey), Seq(externalSig)))
+      // And let bitcoind sign the wallet input.
+      walletExternalFunds.signTransaction(partiallySignedTx, Nil).pipeTo(sender.ref)
+      val signedTx = sender.expectMsgType[SignTransactionResponse].tx
+      walletExternalFunds.publishTransaction(signedTx).pipeTo(sender.ref)
+      sender.expectMsg(signedTx.txid)
+      // The weight of our external input matches our estimation and the resulting feerate is correct.
+      val actualExternalInputWeight = signedTx.weight() - signedTx.copy(txIn = signedTx.txIn.tail).weight()
+      assert(actualExternalInputWeight * 0.9 <= externalInputWeight && externalInputWeight <= actualExternalInputWeight * 1.1)
+      walletExternalFunds.getMempoolTx(signedTx.txid).pipeTo(sender.ref)
+      val mempoolTx = sender.expectMsgType[MempoolTx]
+      val expectedFee = Transactions.weight2fee(targetFeerate, signedTx.weight())
+      val actualFee = mempoolTx.fees
+      assert(expectedFee * 0.9 <= actualFee && actualFee <= expectedFee * 1.1, s"expected fee=$expectedFee actual fee=$actualFee")
+      (signedTx, outputScript)
+    }
+
+    // We're also able to spend unconfirmed external funds and ask bitcoind to fund the corresponding transaction.
+    val tx3 = {
+      val targetFeerate = FeeratePerKw(10_000 sat)
+      val externalOutpoint = OutPoint(tx2, 0)
+      val txNotFunded = Transaction(2, Seq(TxIn(externalOutpoint, Nil, 0)), Seq(TxOut(300_000 sat, Script.pay2wpkh(randomKey().publicKey))), 0)
+      walletExternalFunds.fundTransaction(txNotFunded, FundTransactionOptions(targetFeerate, inputWeights = Seq(InputWeight(externalOutpoint, externalInputWeight)), changePosition = Some(1))).pipeTo(sender.ref)
+      val fundedTx = sender.expectMsgType[FundTransactionResponse]
+      assert(fundedTx.tx.txIn.length >= 2)
+      // We sign our external input.
+      val externalSig = Transaction.signInput(fundedTx.tx, 0, inputScript2, SigHash.SIGHASH_ALL, 300_000 sat, SigVersion.SIGVERSION_WITNESS_V0, alicePriv)
+      val partiallySignedTx = fundedTx.tx.updateWitness(0, Script.witnessMultiSigMofN(Seq(alicePriv, carolPriv).map(_.publicKey), Seq(externalSig)))
+      // And let bitcoind sign the wallet input.
+      walletExternalFunds.signTransaction(partiallySignedTx, Nil).pipeTo(sender.ref)
+      val signedTx = sender.expectMsgType[SignTransactionResponse].tx
+      walletExternalFunds.publishTransaction(signedTx).pipeTo(sender.ref)
+      sender.expectMsg(signedTx.txid)
+      // The resulting feerate takes into account our unconfirmed parent as well.
+      walletExternalFunds.getMempoolTx(signedTx.txid).pipeTo(sender.ref)
+      val mempoolTx = sender.expectMsgType[MempoolTx]
+      assert(mempoolTx.fees == fundedTx.fee)
+      assert(mempoolTx.fees < mempoolTx.ancestorFees)
+      // TODO: uncomment the lines below once bitcoind takes into account unconfirmed ancestors in feerate estimation (expected in Bitcoin Core 25).
+      // val actualFee = mempoolTx.ancestorFees
+      // val expectedFee = Transactions.weight2fee(targetFeerate, signedTx.weight() + tx2.weight())
+      // assert(expectedFee * 0.9 <= actualFee && actualFee <= expectedFee * 1.1, s"expected fee=$expectedFee actual fee=$actualFee")
+      signedTx
+    }
+
+    // We can RBF our unconfirmed transaction by asking bitcoind to fund it again.
+    {
+      val targetFeerate = FeeratePerKw(15_000 sat)
+      // We simply remove the change output, but keep the rest of the transaction unchanged.
+      val txNotFunded = tx3.copy(txOut = tx3.txOut.take(1))
+      val inputWeights = txNotFunded.txIn.map(txIn => {
+        val weight = txNotFunded.weight() - txNotFunded.copy(txIn = txNotFunded.txIn.filterNot(_.outPoint == txIn.outPoint)).weight()
+        InputWeight(txIn.outPoint, weight)
+      })
+      walletExternalFunds.fundTransaction(txNotFunded, FundTransactionOptions(targetFeerate, inputWeights = inputWeights, changePosition = Some(1))).pipeTo(sender.ref)
+      val fundedTx = sender.expectMsgType[FundTransactionResponse]
+      assert(fundedTx.tx.txIn.length >= 2)
+      assert(fundedTx.tx.txOut.length == 2)
+      // We sign our external input.
+      val externalSig = Transaction.signInput(fundedTx.tx, 0, inputScript2, SigHash.SIGHASH_ALL, 300_000 sat, SigVersion.SIGVERSION_WITNESS_V0, alicePriv)
+      val partiallySignedTx = fundedTx.tx.updateWitness(0, Script.witnessMultiSigMofN(Seq(alicePriv, carolPriv).map(_.publicKey), Seq(externalSig)))
+      // And let bitcoind sign the wallet input.
+      walletExternalFunds.signTransaction(partiallySignedTx, Nil).pipeTo(sender.ref)
+      val signedTx = sender.expectMsgType[SignTransactionResponse].tx
+      walletExternalFunds.publishTransaction(signedTx).pipeTo(sender.ref)
+      sender.expectMsg(signedTx.txid)
+      // We have replaced the previous transaction.
+      walletExternalFunds.getMempoolTx(tx3.txid).pipeTo(sender.ref)
+      assert(sender.expectMsgType[Failure].cause.getMessage.contains("Transaction not in mempool"))
+      // The resulting feerate takes into account our unconfirmed parent as well.
+      walletExternalFunds.getMempoolTx(signedTx.txid).pipeTo(sender.ref)
+      val mempoolTx = sender.expectMsgType[MempoolTx]
+      assert(mempoolTx.fees == fundedTx.fee)
+      assert(mempoolTx.fees < mempoolTx.ancestorFees)
+      // TODO: uncomment the lines below once bitcoind takes into account unconfirmed ancestors in feerate estimation (expected in Bitcoin Core 25).
+      // val actualFee = mempoolTx.ancestorFees
+      // val expectedFee = Transactions.weight2fee(targetFeerate, signedTx.weight() + tx2.weight())
+      // assert(expectedFee * 0.9 <= actualFee && actualFee <= expectedFee * 1.1, s"expected fee=$expectedFee actual fee=$actualFee")
+    }
+  }
+
   test("absence of rounding") {
     val txIn = Transaction(1, Nil, Nil, 42)
     val hexOut = "02000000013361e994f6bd5cbe9dc9e8cb3acdc12bc1510a3596469d9fc03cfddd71b223720000000000feffffff02c821354a00000000160014b6aa25d6f2a692517f2cf1ad55f243a5ba672cac404b4c0000000000220020822eb4234126c5fc84910e51a161a9b7af94eb67a2344f7031db247e0ecc2f9200000000"

eclair-core/src/test/scala/fr/acinq/eclair/blockchain/bitcoind/BitcoindService.scala

@@ -56,7 +56,7 @@ trait BitcoindService extends Logging {
 
   val PATH_BITCOIND = sys.env.get("BITCOIND_DIR") match {
     case Some(customBitcoinDir) => new File(customBitcoinDir, "bitcoind")
-    case None => new File(TestUtils.BUILD_DIRECTORY, "bitcoin-0.21.1/bin/bitcoind")
+    case None => new File(TestUtils.BUILD_DIRECTORY, "bitcoin-23.0/bin/bitcoind")
   }
   logger.info(s"using bitcoind: $PATH_BITCOIND")
   val PATH_BITCOIND_DATADIR = new File(INTEGRATION_TMP_DIR, "datadir-bitcoin")

eclair-core/src/test/scala/fr/acinq/eclair/channel/InteractiveTxBuilderSpec.scala

@@ -694,10 +694,6 @@ class InteractiveTxBuilderSpec extends TestKitBaseClass with AnyFunSuiteLike wit
       f.forwardAlice2Bob[TxAddOutput]
       // Alice <-- tx_complete --- Bob
       f.forwardBob2Alice[TxComplete]
-      // Alice --- tx_add_output --> Bob
-      f.forwardAlice2Bob[TxAddOutput]
-      // Alice <-- tx_complete --- Bob
-      f.forwardBob2Alice[TxComplete]
       // Alice --- tx_complete --> Bob
       f.forwardAlice2Bob[TxComplete]
       // Alice --- commit_sig --> Bob
@@ -708,7 +704,8 @@ class InteractiveTxBuilderSpec extends TestKitBaseClass with AnyFunSuiteLike wit
       val txB1 = bob2alice.expectMsgType[Succeeded].sharedTx.asInstanceOf[PartiallySignedSharedTransaction]
       alice ! ReceiveTxSigs(txB1.localSigs)
       val txA1 = alice2bob.expectMsgType[Succeeded].sharedTx.asInstanceOf[FullySignedSharedTransaction]
-      assert(targetFeerate * 0.9 <= txA1.feerate && txA1.feerate <= targetFeerate * 1.25)
+      // Bitcoin Core didn't add a change output, which results in a bigger over-payment of the on-chain fees.
+      assert(targetFeerate * 0.9 <= txA1.feerate && txA1.feerate <= targetFeerate * 1.5)
       val probe = TestProbe()
       walletA.publishTransaction(txA1.signedTx).pipeTo(probe.ref)
       probe.expectMsg(txA1.signedTx.txid)

eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxFunderSpec.scala

@@ -53,39 +53,6 @@ class ReplaceableTxFunderSpec extends TestKitBaseClass with AnyFunSuiteLike {
     (CommitTx(commitInput, commitTx), anchorTx)
   }
 
-  test("adjust anchor tx change amount", Tag("fuzzy")) {
-    val (commitTx, anchorTx) = createAnchorTx()
-    val dustLimit = 600 sat
-    val commitFeerate = FeeratePerKw(2500 sat)
-    val targetFeerate = FeeratePerKw(10000 sat)
-    for (_ <- 1 to 100) {
-      val walletInputsCount = 1 + Random.nextInt(5)
-      val walletInputs = (1 to walletInputsCount).map(_ => TxIn(OutPoint(randomBytes32(), 0), Nil, 0))
-      val amountIn = dustLimit * walletInputsCount + Random.nextInt(25_000_000).sat
-      val amountOut = dustLimit + Random.nextLong(amountIn.toLong).sat
-      val unsignedTx = ClaimLocalAnchorWithWitnessData(anchorTx.copy(tx = anchorTx.tx.copy(
-        txIn = anchorTx.tx.txIn ++ walletInputs,
-        txOut = TxOut(amountOut, Script.pay2wpkh(PlaceHolderPubKey)) :: Nil,
-      )))
-      val adjustedTx = adjustAnchorOutputChange(unsignedTx, commitTx.tx, amountIn, commitFeerate, targetFeerate, dustLimit)
-      assert(adjustedTx.txInfo.tx.txIn.size == unsignedTx.txInfo.tx.txIn.size)
-      assert(adjustedTx.txInfo.tx.txOut.size == 1)
-      assert(adjustedTx.txInfo.tx.txOut.head.amount >= dustLimit)
-      if (adjustedTx.txInfo.tx.txOut.head.amount > dustLimit) {
-        // Simulate tx signing to check final feerate.
-        val signedTx = {
-          val anchorSigned = addSigs(adjustedTx.txInfo, PlaceHolderSig)
-          val signedWalletInputs = anchorSigned.tx.txIn.tail.map(txIn => txIn.copy(witness = Script.witnessPay2wpkh(PlaceHolderPubKey, PlaceHolderSig)))
-          anchorSigned.tx.copy(txIn = anchorSigned.tx.txIn.head +: signedWalletInputs)
-        }
-        // We want the package anchor tx + commit tx to reach our target feerate, but the commit tx already pays a (smaller) fee
-        val targetFee = weight2fee(targetFeerate, signedTx.weight() + commitTx.tx.weight()) - weight2fee(commitFeerate, commitTx.tx.weight())
-        val actualFee = amountIn - signedTx.txOut.map(_.amount).sum
-        assert(targetFee * 0.9 <= actualFee && actualFee <= targetFee * 1.1, s"actualFee=$actualFee targetFee=$targetFee amountIn=$amountIn tx=$signedTx")
-      }
-    }
-  }
-
   private def createHtlcTxs(): (HtlcSuccessWithWitnessData, HtlcTimeoutWithWitnessData) = {
     val preimage = randomBytes32()
     val paymentHash = Crypto.sha256(preimage)
@@ -113,46 +80,6 @@ class ReplaceableTxFunderSpec extends TestKitBaseClass with AnyFunSuiteLike {
     (htlcSuccess, htlcTimeout)
   }
 
-  test("adjust htlc tx change amount", Tag("fuzzy")) {
-    val dustLimit = 600 sat
-    val targetFeerate = FeeratePerKw(10000 sat)
-    val (htlcSuccess, htlcTimeout) = createHtlcTxs()
-    for (_ <- 1 to 100) {
-      val walletInputsCount = 1 + Random.nextInt(5)
-      val walletInputs = (1 to walletInputsCount).map(_ => TxIn(OutPoint(randomBytes32(), 0), Nil, 0))
-      val walletAmountIn = dustLimit * walletInputsCount + Random.nextInt(25_000_000).sat
-      val changeOutput = TxOut(Random.nextLong(walletAmountIn.toLong).sat, Script.pay2wpkh(PlaceHolderPubKey))
-      val unsignedHtlcSuccessTx = htlcSuccess.updateTx(htlcSuccess.txInfo.tx.copy(
-        txIn = htlcSuccess.txInfo.tx.txIn ++ walletInputs,
-        txOut = htlcSuccess.txInfo.tx.txOut ++ Seq(changeOutput)
-      ))
-      val unsignedHtlcTimeoutTx = htlcTimeout.updateTx(htlcTimeout.txInfo.tx.copy(
-        txIn = htlcTimeout.txInfo.tx.txIn ++ walletInputs,
-        txOut = htlcTimeout.txInfo.tx.txOut ++ Seq(changeOutput)
-      ))
-      for (unsignedTx <- Seq(unsignedHtlcSuccessTx, unsignedHtlcTimeoutTx)) {
-        val totalAmountIn = unsignedTx.txInfo.input.txOut.amount + walletAmountIn
-        val adjustedTx = adjustHtlcTxChange(unsignedTx, totalAmountIn, targetFeerate, dustLimit, ZeroFeeHtlcTxAnchorOutputsCommitmentFormat)
-        assert(adjustedTx.txInfo.tx.txIn.size == unsignedTx.txInfo.tx.txIn.size)
-        assert(adjustedTx.txInfo.tx.txOut.size == 1 || adjustedTx.txInfo.tx.txOut.size == 2)
-        if (adjustedTx.txInfo.tx.txOut.size == 2) {
-          // Simulate tx signing to check final feerate.
-          val signedTx = {
-            val htlcSigned = adjustedTx.txInfo match {
-              case tx: HtlcSuccessTx => addSigs(tx, PlaceHolderSig, PlaceHolderSig, ByteVector32.Zeroes, ZeroFeeHtlcTxAnchorOutputsCommitmentFormat)
-              case tx: HtlcTimeoutTx => addSigs(tx, PlaceHolderSig, PlaceHolderSig, ZeroFeeHtlcTxAnchorOutputsCommitmentFormat)
-            }
-            val signedWalletInputs = htlcSigned.tx.txIn.tail.map(txIn => txIn.copy(witness = Script.witnessPay2wpkh(PlaceHolderPubKey, PlaceHolderSig)))
-            htlcSigned.tx.copy(txIn = htlcSigned.tx.txIn.head +: signedWalletInputs)
-          }
-          val targetFee = weight2fee(targetFeerate, signedTx.weight())
-          val actualFee = totalAmountIn - signedTx.txOut.map(_.amount).sum
-          assert(targetFee * 0.9 <= actualFee && actualFee <= targetFee * 1.1, s"actualFee=$actualFee targetFee=$targetFee amountIn=$walletAmountIn tx=$signedTx")
-        }
-      }
-    }
-  }
-
   private def createClaimHtlcTx(): (ClaimHtlcSuccessWithWitnessData, ClaimHtlcTimeoutWithWitnessData) = {
     val preimage = randomBytes32()
     val paymentHash = Crypto.sha256(preimage)

eclair-core/src/test/scala/fr/acinq/eclair/channel/publish/ReplaceableTxPublisherSpec.scala

@@ -307,7 +307,7 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w
   }
 
   test("not enough funds to increase commit tx feerate") {
-    withFixture(Seq(10.5 millibtc), ChannelTypes.AnchorOutputsZeroFeeHtlcTx()) { f =>
+    withFixture(Seq(10.4 millibtc), ChannelTypes.AnchorOutputsZeroFeeHtlcTx()) { f =>
       import f._
 
       // close channel and wait for the commit tx to be published, anchor will not be published because we don't have enough funds
@@ -388,9 +388,9 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w
       // channel funding
       10 millibtc,
       // bumping utxos
-      25000 sat,
+      15000 sat,
-      22000 sat,
+      12000 sat,
-      15000 sat
+      10000 sat
     )
     withFixture(utxos, ChannelTypes.AnchorOutputsZeroFeeHtlcTx()) { f =>
       import f._
@@ -481,7 +481,7 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w
   }
 
   test("commit tx not confirming, adding other wallet inputs") {
-    withFixture(Seq(10.5 millibtc, 5 millibtc), ChannelTypes.AnchorOutputsZeroFeeHtlcTx()) { f =>
+    withFixture(Seq(10.4 millibtc, 5 millibtc), ChannelTypes.AnchorOutputsZeroFeeHtlcTx()) { f =>
       import f._
 
       val (commitTx, anchorTx) = closeChannelWithoutHtlcs(f, aliceBlockHeight() + 30)
@@ -650,7 +650,7 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w
 
       // our parent will stop us when receiving the TxRejected message.
       publisher2 ! Stop
-      awaitCond(getLocks(probe, walletRpcClient).isEmpty)
+      awaitCond(!getLocks(probe, walletRpcClient).exists(_.txid != commitTx.tx.txid))
 
       // the first publishing attempt succeeds
       generateBlocks(5)
@@ -794,7 +794,7 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w
     w.replyTo ! WatchParentTxConfirmedTriggered(currentBlockHeight(probe), 0, commitTx)
     val htlcSuccessTx = getMempoolTxs(1).head
     val htlcSuccessTargetFee = Transactions.weight2fee(targetFeerate, htlcSuccessTx.weight.toInt)
-    assert(htlcSuccessTargetFee * 0.9 <= htlcSuccessTx.fees && htlcSuccessTx.fees <= htlcSuccessTargetFee * 1.4, s"actualFee=${htlcSuccessTx.fees} targetFee=$htlcSuccessTargetFee")
+    assert(htlcSuccessTargetFee * 0.9 <= htlcSuccessTx.fees && htlcSuccessTx.fees <= htlcSuccessTargetFee * 1.2, s"actualFee=${htlcSuccessTx.fees} targetFee=$htlcSuccessTargetFee")
 
     generateBlocks(4)
     system.eventStream.publish(CurrentBlockHeight(currentBlockHeight(probe)))
@@ -823,7 +823,7 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w
     w.replyTo ! WatchParentTxConfirmedTriggered(currentBlockHeight(probe), 0, commitTx)
     val htlcTimeoutTx = getMempoolTxs(1).head
     val htlcTimeoutTargetFee = Transactions.weight2fee(targetFeerate, htlcTimeoutTx.weight.toInt)
-    assert(htlcTimeoutTargetFee * 0.9 <= htlcTimeoutTx.fees && htlcTimeoutTx.fees <= htlcTimeoutTargetFee * 1.4, s"actualFee=${htlcTimeoutTx.fees} targetFee=$htlcTimeoutTargetFee")
+    assert(htlcTimeoutTargetFee * 0.9 <= htlcTimeoutTx.fees && htlcTimeoutTx.fees <= htlcTimeoutTargetFee * 1.2, s"actualFee=${htlcTimeoutTx.fees} targetFee=$htlcTimeoutTargetFee")
 
     generateBlocks(4)
     system.eventStream.publish(CurrentBlockHeight(currentBlockHeight(probe)))
@@ -961,15 +961,15 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w
       assert(htlcSuccessTx1.fees < htlcSuccessTx2.fees)
       assert(htlcSuccessInputs1 == htlcSuccessInputs2)
       val htlcSuccessTargetFee = Transactions.weight2fee(targetFeerate, htlcSuccessTx2.weight.toInt)
-      assert(htlcSuccessTargetFee * 0.9 <= htlcSuccessTx2.fees && htlcSuccessTx2.fees <= htlcSuccessTargetFee * 1.4, s"actualFee=${htlcSuccessTx2.fees} targetFee=$htlcSuccessTargetFee")
+      assert(htlcSuccessTargetFee * 0.9 <= htlcSuccessTx2.fees && htlcSuccessTx2.fees <= htlcSuccessTargetFee * 1.1, s"actualFee=${htlcSuccessTx2.fees} targetFee=$htlcSuccessTargetFee")
     }
   }
 
   test("htlc success tx not confirming, adding other wallet inputs") {
-    withFixture(Seq(10.2 millibtc, 2 millibtc), ChannelTypes.AnchorOutputsZeroFeeHtlcTx()) { f =>
+    withFixture(Seq(1_010_000 sat, 10_000 sat), ChannelTypes.AnchorOutputsZeroFeeHtlcTx()) { f =>
       import f._
 
-      val initialFeerate = FeeratePerKw(15_000 sat)
+      val initialFeerate = FeeratePerKw(3_000 sat)
       setFeerate(initialFeerate)
       val (commitTx, htlcSuccess, _) = closeChannelWithHtlcs(f, aliceBlockHeight() + 15)
 
@@ -986,7 +986,7 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w
       assert(htlcSuccessTx1.txid == htlcSuccessTxId1)
 
       // New blocks are found, which makes us aim for a more aggressive block target, so we bump the fees.
-      val targetFeerate = FeeratePerKw(75_000 sat)
+      val targetFeerate = FeeratePerKw(10_000 sat)
       setFeerate(targetFeerate, blockTarget = 2)
       system.eventStream.publish(CurrentBlockHeight(aliceBlockHeight() + 10))
       val htlcSuccessTxId2 = listener.expectMsgType[TransactionPublished].tx.txid
@@ -995,9 +995,9 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w
       val htlcSuccessInputs2 = getMempool().head.txIn.map(_.outPoint).toSet
       assert(htlcSuccessTx2.txid == htlcSuccessTxId2)
       assert(htlcSuccessTx1.fees < htlcSuccessTx2.fees)
-      assert(htlcSuccessInputs1 !== htlcSuccessInputs2)
+      assert(htlcSuccessInputs1 != htlcSuccessInputs2)
       val htlcSuccessTargetFee = Transactions.weight2fee(targetFeerate, htlcSuccessTx2.weight.toInt)
-      assert(htlcSuccessTargetFee * 0.9 <= htlcSuccessTx2.fees && htlcSuccessTx2.fees <= htlcSuccessTargetFee * 1.4, s"actualFee=${htlcSuccessTx2.fees} targetFee=$htlcSuccessTargetFee")
+      assert(htlcSuccessTargetFee * 0.9 <= htlcSuccessTx2.fees && htlcSuccessTx2.fees <= htlcSuccessTargetFee * 1.1, s"actualFee=${htlcSuccessTx2.fees} targetFee=$htlcSuccessTargetFee")
     }
   }
 
@@ -1089,7 +1089,7 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w
       w.replyTo ! WatchParentTxConfirmedTriggered(currentBlockHeight(probe), 0, commitTx)
       val htlcSuccessTx = getMempoolTxs(1).head
       val htlcSuccessTargetFee = Transactions.weight2fee(targetFeerate, htlcSuccessTx.weight.toInt)
-      assert(htlcSuccessTargetFee * 0.9 <= htlcSuccessTx.fees && htlcSuccessTx.fees <= htlcSuccessTargetFee * 1.4, s"actualFee=${htlcSuccessTx.fees} targetFee=$htlcSuccessTargetFee")
+      assert(htlcSuccessTargetFee * 0.9 <= htlcSuccessTx.fees && htlcSuccessTx.fees <= htlcSuccessTargetFee * 1.1, s"actualFee=${htlcSuccessTx.fees} targetFee=$htlcSuccessTargetFee")
     }
   }
 
@@ -1118,7 +1118,7 @@ class ReplaceableTxPublisherSpec extends TestKitBaseClass with AnyFunSuiteLike w
 
       // our parent will stop us when receiving the TxRejected message.
       publisher2 ! Stop
-      awaitCond(getLocks(probe, walletRpcClient).isEmpty)
+      awaitCond(!getLocks(probe, walletRpcClient).exists(_.txid != commitTx.txid))
 
       // the first publishing attempt succeeds
       generateBlocks(5)

eclair-core/src/test/scala/fr/acinq/eclair/transactions/TransactionsSpec.scala

@@ -16,10 +16,10 @@
 
 package fr.acinq.eclair.transactions
 
+import fr.acinq.bitcoin.SigHash._
 import fr.acinq.bitcoin.scalacompat.Crypto.{PrivateKey, ripemd160, sha256}
 import fr.acinq.bitcoin.scalacompat.Script.{pay2wpkh, pay2wsh, write}
 import fr.acinq.bitcoin.scalacompat.{Btc, ByteVector32, Crypto, MilliBtc, MilliBtcDouble, OutPoint, Protocol, Satoshi, SatoshiLong, Script, ScriptWitness, Transaction, TxIn, TxOut, millibtc2satoshi}
-import fr.acinq.bitcoin.SigHash._
 import fr.acinq.eclair._
 import fr.acinq.eclair.blockchain.fee.FeeratePerKw
 import fr.acinq.eclair.channel.Helpers.Funding
@@ -208,9 +208,12 @@ class TransactionsSpec extends AnyFunSuite with Logging {
         txIn = claimAnchorOutputTx.tx.txIn :+ TxIn(OutPoint(randomBytes32(), 3), ByteVector.empty, 0, p2wpkhWitness),
         txOut = Seq(TxOut(1500 sat, Script.pay2wpkh(randomKey().publicKey)))
       ))
-      val weight = Transaction.weight(addSigs(claimAnchorOutputTxWithFees, PlaceHolderSig).tx)
+      val signedTx = addSigs(claimAnchorOutputTxWithFees, PlaceHolderSig).tx
+      val weight = Transaction.weight(signedTx)
       assert(weight == 717)
       assert(weight >= claimAnchorOutputMinWeight)
+      val inputWeight = Transaction.weight(signedTx) - Transaction.weight(signedTx.copy(txIn = signedTx.txIn.tail))
+      assert(inputWeight == anchorInputWeight)
     }
   }
 

pom.xml

@@ -72,7 +72,7 @@
         <akka.version>2.6.18</akka.version>
         <akka.http.version>10.2.7</akka.http.version>
         <sttp.version>3.4.1</sttp.version>
-        <bitcoinlib.version>0.24</bitcoinlib.version>
+        <bitcoinlib.version>0.25</bitcoinlib.version>
         <guava.version>31.1-jre</guava.version>
         <kamon.version>2.4.6</kamon.version>
     </properties>