Commit Graph

885 Commits

Author SHA1 Message Date
Christian Decker
1da977a04c db: Guard against accessing NULL partid and total_msat
These were spamming my logs and could result in misleading results being
returned on `listpays` and `listsendpays`.
2020-07-28 16:17:39 +09:30
Rusty Russell
73a5f5b313 fundpsbt: make parameters more usable.
fundpsbt forces the caller to manually add their weight * feerate
to the satoshis they ask for.  That means no named feerates.

Instead, create a startweight parameter and do the calc for them
internally, and return the feerate we used (and, while we're at it,
the estimated final weight).

This API change is best done now, as it would otherwise have to
be appended as a parameter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-17 13:53:46 +02:00
Rusty Russell
371cabf976 txprepare: revert 1fb9a078b6 (psbt field)
We're actually going to deprecate this, so don't add new features!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: ***REMOVE*** JSON-API: `txprepare` returns a psbt version of the created transaction
2020-07-15 18:49:02 +09:30
Rusty Russell
10e62af7c7 fundpsbt: add reserve arg.
It's easier for us to call it atomically than have the user loop and
retry!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell
bd19ec2292 fundpsbt: new JSON API to gather UTXOs.
Technically, they could do this themselves, but it's much nicer to have one
place to do it (and it makes sure we get the required information into the
PSBT, which is actually not entirely accessible through listfunds, as that
doesn't want to consult with the HSM for close outputs).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON RPC: new low-level coin selection `fundpsbt` routine.
2020-07-15 18:49:02 +09:30
Rusty Russell
be17a9392f reserveinputs: add exclusive flag.
This is the normal case: you only want to reserve inputs which
are not already reserved.  This saves you iterating through the
results and unreserving some if you weren't exclusive.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell
56ea215ba0 wallet: new JSON commands reserveinputs and unreserveinputs.
reserveinputs marks UTXOs reserved for 12 hours, so we won't select them
for spending: unreserveinputs marks them available again.

Exposes param_psbt() for wider use.

Disabled the test_sign_and_send_psbt since we're altering the API;
the final patch restores it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell
27b48959d7 wallet: add wallet_find_utxo().
This is a new fundamental routine to obtain UTXOs from the database.

It's not the most efficient approach, as it returns a single UTXO at a
time, but it can consolidate all our UTXO handling (becoming more
complex by the reservation timeout logic).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell
50ff0b26fd wallet: explicit routines to reserve/unreserve a UTXO.
These keep the struct utxo in sync with the database, explicitly:
these will be the only places where utxo->status is set.

The old routines will be removed at the end.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
niftynei
7ebdc14397 utxos: add a 'reserved_til' marker for utxos
Allow a utxo to be reserved until explicitly unreserved or until a timer
runs out. Currently unused.

We explicitly do not unreserve these at startup.
2020-07-15 18:49:02 +09:30
niftynei
f5f85b389d elements,pset: populate elements specific data for PSBTs
PSETs have a bit different requirements. The witness_utxo needs
the asset tag + values, and these should also be added to the PSET
struct separately as well. To do this, we create a new 'init' method for
elements inputs, which takes care of the elements specific things.
2020-07-13 11:37:24 +09:30
niftynei
14de198bd1 wally-tx: add type-to-string for a wally-tx
and then use it to print out things
2020-07-13 11:37:24 +09:30
Rusty Russell
94de18ace7 listfunds: add scriptpubkey if it's known.
(Which it is, since 0.7.3).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON: `listfunds` now has a 'scriptpubkey' field.
2020-07-08 21:07:20 +02:00
Rusty Russell
a9427f1a8d bitcoin/feerate: new exposure for feerate parsing outside lightningd.
This exposes the numeric part of param_feerate() as param_feerate_val().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-08 21:07:20 +02:00
Rusty Russell
e12defa7fb wallet: no longer expose wallet_add_utxo function.
All users are now internal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-06 19:25:05 +02:00
Rusty Russell
4271fc8652 wallet: add explicit API for onchaind to register UTXOs.
This is the only place outside the wallet code where we create
a 'struct utxo', so it makes sense for us to move that logic inside
the wallet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-06 19:25:05 +02:00
Rusty Russell
1708fb91e5 wallet: clean up json output creation, part 2.
It looked like we weren't printing the address on closing outputs.

But we are, because the 'scriptPubkey' field is in the 'outputs' db
table since 0.7.3 (66a47d2761).

So make the logic clearer, and remove a completely bogus comment (UTXOs
with closing_info are definitely spendable!).

We export the json_add_utxos() for future use, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-06 19:25:05 +02:00
Rusty Russell
5b82271187 wallet: clean up json output creation, part 1.
We're not allowed to command_fail() once we've started json_success.

That's OK, because encoding a known output can only fail if something is
badly, badly wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-06 19:25:05 +02:00
Rusty Russell
7aa8ffa2a0 bitcoin: add weight calculation helpers.
These are pulled from wallet/wallet.c, with the fix now that we grind sigs.

This reduces the fees we pay slightly, as you can see in the coinmoves changes.

I now print out all the coin moves in suitable format before we match:
you only see this if the test fails, but it's really helpful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-06 19:25:05 +02:00
ZmnSCPxj jxPCSnmZ
d0c85033d2 wallet/walletrpc.c: txprepared transactions now use current tip blockheight by default.
Changelog-Changed: `txprepare` now prepares transactions whose `nLockTime` is set to the tip blockheight, instead of using 0. `fundchannel` will use `nLockTime` set to the tip blockheight as well.
2020-07-01 15:17:33 +00:00
Christian Decker
7b899da801 db: Retrieve peer ID if it exists or create the peer if not
We were assuming `wallet_channel_insert` that there cannot be a matching peer
if our in-memory representation isn't bound to it (`dbid == 0`). If we then
attempt to create the peer, and we already had one it'd cause a unique
constraint violation. As far as I can tell this could end up happening if we
have an uncommitted channel, and then exited without cleanup (`tal_destructor`
on the uncommitted channel not running). This could then leave the peer in the
DB. This is because the constraint that every peer has at least one channel is
not enforce at DB level, but rather in destructors that may or may not run.

Changelog-Fixed: Fixed a failing assertion if we reconnect to a peer that we had a channel with before, and then attempt to insert the peer into the DB twice.
2020-07-01 09:22:29 +09:30
Rusty Russell
ae0cccb293 wallet: fix memleak if we get shut down before finishing utxo cleanup.
"backtrace": [
                       "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
                       "wallet/wallet.c:154 (wallet_stmt2output)",
                       "wallet/wallet.c:275 (wallet_get_utxos)",
                       "wallet/wallet.c:3792 (wallet_clean_utxos)",
                       "lightningd/lightningd.c:914 (main)"
                   ],
                   "label": "wallet/wallet.c:154:struct utxo",
                   "parents": [
                       "wallet/wallet.c:273:struct utxo*[]"
                   ],
                   "value": "0x24c1be8"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-01 09:19:39 +09:30
ZmnSCPxj jxPCSnmZ
d14460ef3d wallet/walletrpc.c: Show input annotations for inputs.
Changelog-None
2020-06-30 17:59:17 +02:00
niftynei
9830c94778 rpc: new signpsbt + sendpsbt rpcs
Changelog-Added: JSON-RPC: new call `signpsbt` which will add the wallet's signatures to a provided psbt
Changelog-Added: JSON-RPC: new call `sendpsbt` which will finalize and send a signed PSBT
2020-06-29 16:10:05 +02:00
niftynei
fd8a716695 wallet: have wallet_extract_outputs take wally_tx, not bitcoin_tx
With the incursion of PSBTs, we're moving away from bitcoin_tx
2020-06-29 16:10:05 +02:00
niftynei
103dce63ef reserve/unreserve input: new RPC commands for reserving inputs/outputs
Reserve and unreserve wallet UTXOs using a PSBT which includes those
inputs.

Note that currently we unreserve inputs everytime the node restarts.
This will be addressed in a future commit.

Changelog-Added: JSON-RPC: Adds two new rpc methods, `reserveinputs` and `unreserveinputs`, which allow for reserving or unreserving wallet UTXOs
2020-06-29 16:10:05 +02:00
niftynei
b90be4f6c8 prepare-tx: pass back the feerate, as json_tx_prepare sometimes sets it
Unused here, but we'll use it in the next commit so that we can always
pass back the effective / used feerate to the caller of `reserveinputs`

This makes opening a channel much easier if we've internally determined
the feerate
2020-06-29 16:10:05 +02:00
niftynei
0bd0de54fa psbt: have withdraw_tx use psbt's to create signed txs
this will allow us to add inputs that aren't ours to a tx that we sign
and finalize
2020-06-23 14:49:32 +02:00
niftynei
85dca84327 psbt-json: remove reliance on bitcoin_tx, use straight wally_psbt struct 2020-06-23 14:49:32 +02:00
niftynei
c3ae44e296 psbt: don't crash if we can't add a partial sig
instead return a boolean indicating the success/failure of a sig set
2020-06-23 14:49:32 +02:00
lisa neigut
16656a85cf withdraw: refactor change output handling
We're not using the change_outnum for withdraw tx's (and the way
we were calculating it was broken as of the addition of 'multiple
outputs'). This removes the change output knowhow from withdraw_tx
entirely, and pushes the responsibility up to the caller to
include the change output in the output set if desired.

Consequently, we also remove the change output knowhow from hsmd.
2020-06-23 14:49:32 +02:00
niftynei
2900da6112 migrations: ignore channels that don't have a peer_id
We erase peer data after the last channel close transaction for that
peer is 100 blocks deep. We were failing to finish the migration because
the peer_id lookup on these was failing.

Now we ignore any channel with a null peer_id.

Fixes #3768
2020-06-17 14:30:41 +02:00
niftynei
431463b57a listfunds: also list reserved outputs
Currently 'listfunds' lies, a teensy eeinsy bit, in that it doesn't list
all of the funds in a wallet (it omits reserved wallet UTXOs). This
change makes the reserved outputs visible by listing them in the
'outputs' section along with a new field, 'reserved', which denotes the
UTXO's state

Changelog-Changed: JSON-RPC: `listfunds` 'outputs' now includes reserved outputs, designated as 'reserved' = true
2020-06-16 15:58:58 +02:00
niftynei
bb589e0eaf psbt: database migration for converting last_tx to a psbt
We update the `last_tx` in `channels` to be psbt format, instead
of a linearized transaction.

We need the amount of the input populated, which we have since
this is the 'funding' amount. Ideally we'd also populate the funding
scriptPubkey, but to do that we'd need to access the HSM module to fetch
our local funding pubkey, which isn't initialized at the time that the
database migrations are run.

Since the only field the HSM uses currently when signing these is the
amount field, it's ok to just leave it out.

needs a test!
2020-06-11 13:13:13 +02:00
niftynei
8fa04a710a psbt: move channels.last_tx field to be a psbt
note: missing migration at the moment lol
2020-06-11 13:13:13 +02:00
niftynei
894a46e8e3 psbt: populate last commitment transaction's input info at db
when re-populating a channel's data from the database, since we don't
store the psbt data (with input scripts + amounts), we need to
re-populate it.

the right solution is to patch the psbt into the database; for now we
'monkey-patch' it in.
2020-06-11 13:13:13 +02:00
niftynei
a04f0fe250 psbt: remove input_amounts from bitcoin tx
Instead we will stash them into the PSBT as a utxo/witness record (which
includes the amount)
2020-06-11 13:13:13 +02:00
niftynei
78d95b51aa nit: align spacing for SQL stmts 2020-05-29 16:20:23 -05:00
fiatjaf
96452eafb7 sort listinvoices and listsendpays by order of creation. 2020-05-29 15:30:59 +02:00
Antoine Poinsot
5dfd2436db wallet: show input txid in little endian for json_listtransactions
Changelog-Fixed: jsonrpc: `listtransactions` now displays all txids as little endian

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-05-22 15:11:51 +02:00
lisa neigut
1fb9a078b6 txprepare: return psbt serialized version of tx as well
Changelog-Added: JSON-API: `txprepare` returns a psbt version of the created transaction
2020-05-21 18:45:07 +09:30
Antoine Poinsot
f598caa60d config: don't ignore the --commit-fee option.
We did not take the value of --commit-fee into account : this removes
the unused option from lightningd and instead registers it in bcli,
where we set the actual feerate of commitment transactions. This also
corrects the documentation.

Changelog-Fixed: config: we now take the --commit-fee parameter into account.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-05-20 06:09:24 +09:30
niftynei
fbe50e087a setup: create a common setup which will handle the wally-context
Since we now over-write the wally malloc/free functions, we need to do
so for tests as well. Here we pull up all of the common setup/teardown
logic into a separate place, and update the tests that use libwally to
use the new common_setup core

Changelog-None
2020-05-19 13:35:42 +09:30
Rusty Russell
cfb320c972 wire: move remaining bitcoin functions out to bitcoin/ files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
fda5f0b427 common/channel_id: move channel_id into its own file.
The definition was in wire/wire.h, and helper functions in fromwire.c!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
lisa neigut
8acbbca05d coins: use the chain's BIP173 name instead of a 'unit of account'
Updates the unit of account to be the chain_id, which is the BIP173 name
of the chain that the coins moved on.

Suggested-By: @rustyrussell
2020-05-12 15:46:58 +09:30
lisa neigut
de065580f6 coins: update API surface for creating coin movements
Canonicalize the signature for the 'tag-type' of coin moves by unique
constructor/method calls.

Suggested-By: @rustyrussell
2020-05-12 15:46:58 +09:30
lisa neigut
aab9893661 coins: have we_fulfilled be fully 'ternary'
note that 'null' 'we_fulfilled's are going to be legacy from this
release forward.
2020-05-12 15:46:58 +09:30
lisa neigut
8537e77ac7 coins: re-write API interface for htlc notices
Wrap up more logic internally to the method call for htlcs. Also, don't
touch part id if we're not the 'origin'

Suggested-By: @rustyrussell
2020-05-12 15:46:58 +09:30
lisa neigut
ffd9467f14 coin moves: persist the coin movement index counter to disk
Should make it easier to track when coin moves in the plugin are
disjoint from what c-lightning says it's broadcast already.
2020-05-12 15:46:58 +09:30