wallet: new JSON commands reserveinputs and unreserveinputs.

reserveinputs marks UTXOs reserved for 12 hours, so we won't select them
for spending: unreserveinputs marks them available again.

Exposes param_psbt() for wider use.

Disabled the test_sign_and_send_psbt since we're altering the API;
the final patch restores it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

doc/lightning-reserveinputs.7

@@ -3,58 +3,32 @@
 lightning-reserveinputs - Construct a transaction and reserve the UTXOs it spends
 .SH SYNOPSIS
 
-\fBreserveinputs\fR \fIoutputs\fR [\fIfeerate\fR] [\fIminconf\fR] [\fIutxos\fR]
+\fBreserveinputs\fR \fIpsbt\fR
 
 .SH DESCRIPTION
 
-The \fBreserveinputs\fR RPC command creates an unsigned PSBT which
+The \fBreserveinputs\fR RPC command places (or increases) reservations on any
-spends funds from c-lightning’s internal wallet to the outputs specified
+inputs specified in \fIpsbt\fR which are known to lightningd\.  It will fail
-in \fIoutputs\fR\.
+with an error it any of the inputs are known to be spent\.
-
-
-The \fIoutputs\fR is the array of output that include \fIdestination\fR
-and \fIamount\fR({\fIdestination\fR: \fIamount\fR})\. Its format is like:
-[{address1: amount1}, {address2: amount2}]
-or
-[{address: \fIall\fR}]\.
-It supports any number of outputs\.
-
-
-The \fIdestination\fR of output is the address which can be of any Bitcoin accepted
-type, including bech32\.
-
-
-The \fIamount\fR of output is the amount to be sent from the internal wallet
-(expressed, as name suggests, in amount)\. The string \fIall\fR can be used to specify
-all available funds\. Otherwise, it is in amount precision; it can be a whole
-number, a whole number ending in \fIsat\fR, a whole number ending in \fI000msat\fR,
-or a number with 1 to 8 decimal places ending in \fIbtc\fR\.
-
-
-\fIfeerate\fR is an optional feerate to use\. It can be one of the strings
-\fIurgent\fR (aim for next block), \fInormal\fR (next 4 blocks or so) or \fIslow\fR
-(next 100 blocks or so) to use lightningd’s internal estimates: \fInormal\fR
-is the default\.
-
-
-Otherwise, \fIfeerate\fR is a number, with an optional suffix: \fIperkw\fR means
-the number is interpreted as satoshi-per-kilosipa (weight), and \fIperkb\fR
-means it is interpreted bitcoind-style as satoshi-per-kilobyte\. Omitting
-the suffix is equivalent to \fIperkb\fR\.
-
-
-\fIminconf\fR specifies the minimum number of confirmations that reserved UTXOs 
-should have\. Default is 1\.
-
-
-\fIutxos\fR specifies the utxos to be used to fund the transaction, as an array
-of "txid:vout"\. These must be drawn from the node's available UTXO set\.
 
 .SH RETURN VALUE
 
-On success, an object with attributes \fIpsbt\fR and \fIfeerate_per_kw\fR will be
+On success, an \fIreservations\fR array is returned, with an entry for each input
-returned\. The inputs of the \fIpsbt\fR have been marked as reserved in the internal wallet\.
+which was reserved:
 
+.RS
+.IP \[bu]
+\fItxid\fR is the input transaction id\.
+.IP \[bu]
+\fIvout\fR is the input index\.
+.IP \[bu]
+\fIwas_reserved\fR indicates whether the input was already reserved\.
+.IP \[bu]
+\fIreserved\fR indicates that the input is now reserved (i\.e\. true)\.
+.IP \[bu]
+\fIreserved_to_block\fR indicates what blockheight the reservation will expire\.
+
+.RE
 
 On failure, an error is reported and no UTXOs are reserved\.
 
@@ -63,12 +37,7 @@ The following error codes may occur:
 
 .RS
 .IP \[bu]
--1: Catchall nonspecific error\.
+-32602: Invalid parameter, such as specifying a spent input in \fIpsbt\fR\.
-.IP \[bu]
-301: There are not enough funds in the internal wallet (including
-fees) to create the transaction\.
-.IP \[bu]
-302: The dust limit is not met\.
 
 .RE
 .SH AUTHOR

doc/lightning-reserveinputs.7.md

@@ -4,61 +4,32 @@ lightning-reserveinputs -- Construct a transaction and reserve the UTXOs it spen
 SYNOPSIS
 --------
 
-**reserveinputs** *outputs* \[*feerate*\] \[*minconf*\] \[*utxos*\]
+**reserveinputs** *psbt*
 
 DESCRIPTION
 -----------
 
-The **reserveinputs** RPC command creates an unsigned PSBT which
+The **reserveinputs** RPC command places (or increases) reservations on any
-spends funds from c-lightning’s internal wallet to the outputs specified
+inputs specified in *psbt* which are known to lightningd.  It will fail
-in *outputs*.
+with an error it any of the inputs are known to be spent.
-
-The *outputs* is the array of output that include *destination*
-and *amount*(\{*destination*: *amount*\}). Its format is like:
-\[\{address1: amount1\}, \{address2: amount2\}\]
-or
-\[\{address: *all*\}\].
-It supports any number of outputs.
-
-The *destination* of output is the address which can be of any Bitcoin accepted
-type, including bech32.
-
-The *amount* of output is the amount to be sent from the internal wallet
-(expressed, as name suggests, in amount). The string *all* can be used to specify
-all available funds. Otherwise, it is in amount precision; it can be a whole
-number, a whole number ending in *sat*, a whole number ending in *000msat*,
-or a number with 1 to 8 decimal places ending in *btc*.
-
-*feerate* is an optional feerate to use. It can be one of the strings
-*urgent* (aim for next block), *normal* (next 4 blocks or so) or *slow*
-(next 100 blocks or so) to use lightningd’s internal estimates: *normal*
-is the default.
-
-Otherwise, *feerate* is a number, with an optional suffix: *perkw* means
-the number is interpreted as satoshi-per-kilosipa (weight), and *perkb*
-means it is interpreted bitcoind-style as satoshi-per-kilobyte. Omitting
-the suffix is equivalent to *perkb*.
-
-*minconf* specifies the minimum number of confirmations that reserved UTXOs 
-should have. Default is 1.
-
-*utxos* specifies the utxos to be used to fund the transaction, as an array
-of "txid:vout". These must be drawn from the node's available UTXO set.
 
 
 RETURN VALUE
 ------------
 
-On success, an object with attributes *psbt* and *feerate_per_kw* will be
+On success, an *reservations* array is returned, with an entry for each input
-returned. The inputs of the *psbt* have been marked as reserved in the internal wallet.
+which was reserved:
+
+- *txid* is the input transaction id.
+- *vout* is the input index.
+- *was_reserved* indicates whether the input was already reserved.
+- *reserved* indicates that the input is now reserved (i.e. true).
+- *reserved_to_block* indicates what blockheight the reservation will expire.
 
 On failure, an error is reported and no UTXOs are reserved.
 
 The following error codes may occur:
-- -1: Catchall nonspecific error.
+- -32602: Invalid parameter, such as specifying a spent input in *psbt*.
-- 301: There are not enough funds in the internal wallet (including
-fees) to create the transaction.
-- 302: The dust limit is not met.
 
 AUTHOR
 ------

doc/lightning-unreserveinputs.7

@@ -7,31 +7,40 @@ lightning-unreserveinputs - Release reserved UTXOs
 
 .SH DESCRIPTION
 
-The \fBunreserveinputs\fR RPC command releases UTXOs which were previously 
+The \fBunreserveinputs\fR RPC command releases (or reduces reservation)
-marked as reserved, generally by \fBlightning-reserveinputs\fR(7)\.
+on UTXOs which were previously marked as reserved, generally by
+\fBlightning-reserveinputs\fR(7)\.
 
 
 The inputs to unreserve are the inputs specified in the passed-in \fIpsbt\fR\.
 
 .SH RETURN VALUE
 
-On success, an object with \fIoutputs\fR will be returned\.
+On success, an \fIreservations\fR array is returned, with an entry for each input
+which was reserved:
 
+.RS
+.IP \[bu]
+\fItxid\fR is the input transaction id\.
+.IP \[bu]
+\fIvout\fR is the input index\.
+.IP \[bu]
+\fIwas_reserved\fR indicates whether the input was already reserved (generally true)
+.IP \[bu]
+\fIreserved\fR indicates that the input is now reserved (may still be true, if it was previously reserved for a long time)\.
+.IP \[bu]
+\fIreserved_to_block\fR (if \fIreserved\fR is still true) indicates what blockheight the reservation will expire\.
 
-\fIoutputs\fR will include an entry for each input specified in the \fIpsbt\fR,
+.RE
-indicating the \fItxid\fR and \fIvout\fR for that input plus a boolean result
- \fIunreserved\fR, which will be true if that UTXO was successfully unreserved
-by this call\.
 
-
+On failure, an error is reported and no UTXOs are unreserved\.
-Note that restarting lightningd will unreserve all UTXOs by default\.
 
 
 The following error codes may occur:
 
 .RS
 .IP \[bu]
--1: An unparseable PSBT\.
+-32602: Invalid parameter, i\.e\. an unparseable PSBT\.
 
 .RE
 .SH AUTHOR

doc/lightning-unreserveinputs.7.md

@@ -9,25 +9,28 @@ SYNOPSIS
 DESCRIPTION
 -----------
 
-The **unreserveinputs** RPC command releases UTXOs which were previously 
+The **unreserveinputs** RPC command releases (or reduces reservation)
-marked as reserved, generally by lightning-reserveinputs(7).
+on UTXOs which were previously marked as reserved, generally by
+lightning-reserveinputs(7).
 
 The inputs to unreserve are the inputs specified in the passed-in *psbt*.
 
 RETURN VALUE
 ------------
 
-On success, an object with *outputs* will be returned.
+On success, an *reservations* array is returned, with an entry for each input
+which was reserved:
 
-*outputs* will include an entry for each input specified in the *psbt*,
+- *txid* is the input transaction id.
-indicating the *txid* and *vout* for that input plus a boolean result
+- *vout* is the input index.
- *unreserved*, which will be true if that UTXO was successfully unreserved
+- *was_reserved* indicates whether the input was already reserved (generally true)
-by this call.
+- *reserved* indicates that the input is now reserved (may still be true, if it was previously reserved for a long time).
+- *reserved_to_block* (if *reserved* is still true) indicates what blockheight the reservation will expire.
 
-Note that restarting lightningd will unreserve all UTXOs by default.
+On failure, an error is reported and no UTXOs are unreserved.
 
 The following error codes may occur:
-- -1: An unparseable PSBT.
+- -32602: Invalid parameter, i.e. an unparseable PSBT.
 
 AUTHOR
 ------

tests/test_wallet.py

@@ -564,6 +564,7 @@ def test_reserveinputs(node_factory, bitcoind, chainparams):
     assert len(l1.rpc.listfunds()['outputs']) == 12
 
 
+@pytest.mark.xfail(strict=True)
 def test_sign_and_send_psbt(node_factory, bitcoind, chainparams):
     """
     Tests for the sign + send psbt RPCs

wallet/Makefile

@@ -11,11 +11,14 @@ WALLET_LIB_SRC :=		\
 	wallet/wallet.c		\
 	wallet/walletrpc.c
 
+WALLET_LIB_SRC_NOHDR :=		\
+	wallet/reservation.c
+
 WALLET_DB_DRIVERS :=		\
 	wallet/db_postgres.c	\
 	wallet/db_sqlite3.c
 
-WALLET_LIB_OBJS := $(WALLET_LIB_SRC:.c=.o) $(WALLET_DB_DRIVERS:.c=.o)
+WALLET_LIB_OBJS := $(WALLET_LIB_SRC:.c=.o) $(WALLET_LIB_SRC_NOHDR:.c=.o) $(WALLET_DB_DRIVERS:.c=.o)
 WALLET_LIB_HEADERS := $(WALLET_LIB_SRC:.c=.h)
 
 # Make sure these depend on everything.

wallet/reservation.c

@@ -0,0 +1,154 @@
+/* Dealing with reserving UTXOs */
+#include <common/json_command.h>
+#include <common/json_helpers.h>
+#include <common/jsonrpc_errors.h>
+#include <lightningd/jsonrpc.h>
+#include <lightningd/lightningd.h>
+#include <wallet/wallet.h>
+#include <wallet/walletrpc.h>
+
+static bool was_reserved(enum output_status oldstatus,
+			 const u32 *reserved_til,
+			 u32 current_height)
+{
+	if (oldstatus != output_state_reserved)
+		return false;
+
+	return *reserved_til > current_height;
+}
+
+static void json_add_reservestatus(struct json_stream *response,
+				   const struct utxo *utxo,
+				   enum output_status oldstatus,
+				   u32 old_res,
+				   u32 current_height)
+{
+	json_object_start(response, NULL);
+	json_add_txid(response, "txid", &utxo->txid);
+	json_add_u32(response, "vout", utxo->outnum);
+	json_add_bool(response, "was_reserved",
+		      was_reserved(oldstatus, &old_res, current_height));
+	json_add_bool(response, "reserved",
+		      is_reserved(utxo, current_height));
+	if (utxo->reserved_til)
+		json_add_u32(response, "reserved_to_block",
+			     *utxo->reserved_til);
+	json_object_end(response);
+}
+
+static struct command_result *json_reserveinputs(struct command *cmd,
+						 const char *buffer,
+						 const jsmntok_t *obj UNNEEDED,
+						 const jsmntok_t *params)
+{
+	struct json_stream *response;
+	struct wally_psbt *psbt;
+	struct utxo **utxos = tal_arr(cmd, struct utxo *, 0);
+
+	if (!param(cmd, buffer, params,
+		   p_req("psbt", param_psbt, &psbt),
+		   NULL))
+		return command_param_failed();
+
+	for (size_t i = 0; i < psbt->tx->num_inputs; i++) {
+		struct bitcoin_txid txid;
+		struct utxo *utxo;
+
+		wally_tx_input_get_txid(&psbt->tx->inputs[i], &txid);
+		utxo = wallet_utxo_get(cmd, cmd->ld->wallet,
+				       &txid, psbt->tx->inputs[i].index);
+		if (!utxo)
+			continue;
+		if (utxo->status == output_state_spent)
+			return command_fail(cmd, JSONRPC2_INVALID_PARAMS,
+					    "%s:%u already spent",
+					    type_to_string(tmpctx,
+							   struct bitcoin_txid,
+							   &utxo->txid),
+					    utxo->outnum);
+		tal_arr_expand(&utxos, utxo);
+	}
+
+	response = json_stream_success(cmd);
+	json_array_start(response, "reservations");
+	for (size_t i = 0; i < tal_count(utxos); i++) {
+		enum output_status oldstatus;
+		u32 old_res;
+
+		oldstatus = utxos[i]->status;
+		old_res = utxos[i]->reserved_til ? *utxos[i]->reserved_til : 0;
+
+		if (!wallet_reserve_utxo(cmd->ld->wallet,
+					 utxos[i],
+					 get_block_height(cmd->ld->topology))) {
+			fatal("Unable to reserve %s:%u!",
+			      type_to_string(tmpctx,
+					     struct bitcoin_txid,
+					     &utxos[i]->txid),
+			      utxos[i]->outnum);
+		}
+		json_add_reservestatus(response, utxos[i], oldstatus, old_res,
+				       get_block_height(cmd->ld->topology));
+	}
+	json_array_end(response);
+	return command_success(cmd, response);
+}
+
+static const struct json_command reserveinputs_command = {
+	"reserveinputs",
+	"bitcoin",
+	json_reserveinputs,
+	"Reserve utxos (or increase their reservation)",
+	false
+};
+AUTODATA(json_command, &reserveinputs_command);
+
+static struct command_result *json_unreserveinputs(struct command *cmd,
+						   const char *buffer,
+						   const jsmntok_t *obj UNNEEDED,
+						   const jsmntok_t *params)
+{
+	struct json_stream *response;
+	struct wally_psbt *psbt;
+
+	if (!param(cmd, buffer, params,
+		   p_req("psbt", param_psbt, &psbt),
+		   NULL))
+		return command_param_failed();
+
+	response = json_stream_success(cmd);
+	json_array_start(response, "reservations");
+	for (size_t i = 0; i < psbt->tx->num_inputs; i++) {
+		struct bitcoin_txid txid;
+		struct utxo *utxo;
+		enum output_status oldstatus;
+		u32 old_res;
+
+		wally_tx_input_get_txid(&psbt->tx->inputs[i], &txid);
+		utxo = wallet_utxo_get(cmd, cmd->ld->wallet,
+				       &txid, psbt->tx->inputs[i].index);
+		if (!utxo || utxo->status != output_state_reserved)
+			continue;
+
+		oldstatus = utxo->status;
+		old_res = *utxo->reserved_til;
+
+		wallet_unreserve_utxo(cmd->ld->wallet,
+				      utxo,
+				      get_block_height(cmd->ld->topology));
+
+		json_add_reservestatus(response, utxo, oldstatus, old_res,
+				       get_block_height(cmd->ld->topology));
+	}
+	json_array_end(response);
+	return command_success(cmd, response);
+}
+
+static const struct json_command unreserveinputs_command = {
+	"unreserveinputs",
+	"bitcoin",
+	json_unreserveinputs,
+	"Unreserve utxos (or at least, reduce their reservation)",
+	false
+};
+AUTODATA(json_command, &unreserveinputs_command);

wallet/walletrpc.c

@@ -1218,45 +1218,11 @@ static const struct json_command listtransactions_command = {
 };
 AUTODATA(json_command, &listtransactions_command);
 
-static struct command_result *json_reserveinputs(struct command *cmd,
+struct command_result *param_psbt(struct command *cmd,
-						 const char *buffer,
+				  const char *name,
-						 const jsmntok_t *obj UNNEEDED,
+				  const char *buffer,
-						 const jsmntok_t *params)
+				  const jsmntok_t *tok,
-{
+				  struct wally_psbt **psbt)
-	struct command_result *res;
-	struct json_stream *response;
-	struct unreleased_tx *utx;
-
-	u32 feerate;
-
-	res = json_prepare_tx(cmd, buffer, params, false, &utx, &feerate);
-	if (res)
-		return res;
-
-	/* Unlike json_txprepare, we don't keep the utx object
-	 * around, so we remove the auto-cleanup that happens
-	 * when the utxo objects are free'd */
-	wallet_persist_utxo_reservation(cmd->ld->wallet, utx->wtx->utxos);
-
-	response = json_stream_success(cmd);
-	json_add_psbt(response, "psbt", utx->tx->psbt);
-	json_add_u32(response, "feerate_per_kw", feerate);
-	return command_success(cmd, response);
-}
-static const struct json_command reserveinputs_command = {
-	"reserveinputs",
-	"bitcoin",
-	json_reserveinputs,
-	"Reserve inputs and pass back the resulting psbt",
-	false
-};
-AUTODATA(json_command, &reserveinputs_command);
-
-static struct command_result *param_psbt(struct command *cmd,
-					 const char *name,
-					 const char *buffer,
-					 const jsmntok_t *tok,
-					 struct wally_psbt **psbt)
 {
 	/* Pull out the token into a string, then pass to
 	 * the PSBT parser; PSBT parser can't handle streaming
@@ -1265,55 +1231,12 @@ static struct command_result *param_psbt(struct command *cmd,
 	if (psbt_from_b64(psbt_buff, psbt))
 		return NULL;
 
-	return command_fail(cmd, LIGHTNINGD, "'%s' should be a PSBT, not '%.*s'",
+	return command_fail(cmd, JSONRPC2_INVALID_PARAMS,
+			    "'%s' should be a PSBT, not '%.*s'",
 			    name, json_tok_full_len(tok),
 			    json_tok_full(buffer, tok));
 }
 
-static struct command_result *json_unreserveinputs(struct command *cmd,
-						   const char *buffer,
-						   const jsmntok_t *obj UNNEEDED,
-						   const jsmntok_t *params)
-{
-	struct json_stream *response;
-	struct wally_psbt *psbt;
-
-	/* for each input in the psbt, attempt to 'unreserve' it */
-	if (!param(cmd, buffer, params,
-		   p_req("psbt", param_psbt, &psbt),
-		   NULL))
-		return command_param_failed();
-
-	response = json_stream_success(cmd);
-	json_array_start(response, "outputs");
-	for (size_t i = 0; i < psbt->tx->num_inputs; i++) {
-		struct wally_tx_input *in;
-		struct bitcoin_txid txid;
-		bool unreserved;
-
-		in = &psbt->tx->inputs[i];
-		wally_tx_input_get_txid(in, &txid);
-		unreserved = wallet_unreserve_output(cmd->ld->wallet,
-						     &txid, in->index);
-		json_object_start(response, NULL);
-		json_add_txid(response, "txid", &txid);
-		json_add_u64(response, "vout", in->index);
-		json_add_bool(response, "unreserved", unreserved);
-		json_object_end(response);
-	}
-	json_array_end(response);
-
-	return command_success(cmd, response);
-}
-static const struct json_command unreserveinputs_command = {
-	"unreserveinputs",
-	"bitcoin",
-	json_unreserveinputs,
-	"Unreserve inputs, freeing them up to be reused",
-	false
-};
-AUTODATA(json_command, &unreserveinputs_command);
-
 static struct command_result *match_psbt_inputs_to_utxos(struct command *cmd,
 							 struct wally_psbt *psbt,
 							 struct utxo ***utxos)

wallet/walletrpc.h

@@ -1,9 +1,12 @@
 #ifndef LIGHTNING_WALLET_WALLETRPC_H
 #define LIGHTNING_WALLET_WALLETRPC_H
 #include "config.h"
+#include <common/json.h>
 
+struct command;
 struct json_stream;
 struct utxo;
+struct wally_psbt;
 
 void json_add_utxos(struct json_stream *response,
 		    struct wallet *wallet,
@@ -12,4 +15,9 @@ void json_add_utxos(struct json_stream *response,
 /* We evaluate reserved timeouts lazily, so use this. */
 bool is_reserved(const struct utxo *utxo, u32 current_height);
 
+struct command_result *param_psbt(struct command *cmd,
+				  const char *name,
+				  const char *buffer,
+				  const jsmntok_t *tok,
+				  struct wally_psbt **psbt);
 #endif /* LIGHTNING_WALLET_WALLETRPC_H */