Commit Graph

122 Commits

Author SHA1 Message Date
Christian Decker
542f3225e3 pytest: Parameterize process waits for hsmtool calls
We were sometimes waiting only 5 seconds, which is way too short on a
heavily loaded machine such as CI. Making it 30 seconds and collecting
it in a single place so we can adjust more easily.
2021-01-29 10:29:09 +10:30
Christian Decker
2b12cac31e pytest: Skip hsm encryption test if we don't have a TTY 2021-01-29 10:29:09 +10:30
Antoine Poinsot
56c223886c lightning: confirm password on hsm_secret encryption
Changelog-changed: lightningd: the `--encrypted-hsm` now asks you to confirm your password when first set
Changelog-changed: hsmtool: the `encrypt` now asks you to confirm your password
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
19afe15968 hsmtool: only write 32 bytes to hsm_secret on generatehsm
Fixes #4302
Changelog-fixed: hsmtool: the `generatehsm` command now generates an appropriately-sized hsm_secret
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 19:02:28 +01:00
Antoine Poinsot
aa2586a45a pytest: reproduce issue #4302
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 19:02:28 +01:00
Antoine Poinsot
1513a2d07e hsmtool: input encryption password from stdin
This slightly breaks the API, but still accept the input: we just don't
take it into account anymore.

For `dumponchaindescriptors`, we have to still take the old place of the
`network` parameter into account to not entirely break the API.

Changelog-Added: hsmtool: password must now be entered on stdin. Password passed on the command line are discarded.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 16:15:40 +01:00
Christian Decker
71fafd23fd pytest: Add a test to reproduce #4258 2020-12-09 06:56:21 +10:30
Antoine Poinsot
fc964edc06 pytest: test onchain wallet descriptor import on bitcoind
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-10 10:34:13 -06:00
niftynei
26bc4f5239 tx,bugfix: correct signature length estimate
71-bytes for a signature already includes the sighash byte.

 2-bytes	30 44 (DER- prefix thing)
34-bytes	02 20 6e29c8df67fffdda1613cef1413eb1a9ef3627f1fc5e4d910837274eafcc7b2a (r)
34-bytes	02 20 4b8563d79b92fdd830a546862439f80b24132d09318af2c7220c791067067e29 (s)
 1-byte		01 (sighash)
==
71-bytes
2020-10-21 11:24:16 +10:30
niftynei
e2a6fd7112 common: pull up param_psbt
Usable other places, not just in wallet
2020-10-20 12:50:31 +10:30
Rusty Russell
2d897be10a pytest: fix experimental test.
Broken back in 3b8c0a7397 which
changed the behavior of funding not to mark as immediately spent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-08 11:00:23 +10:30
niftynei
eab7fcf8e5 test_utxopsbt: account for fee output diff when elements 2020-09-10 12:31:36 +09:30
niftynei
4311c658d0 test_txprepare: skip over fee outputs (elements) 2020-09-10 12:31:36 +09:30
niftynei
250af67908 elements: run sign + send psbt check for elements tests
Now that we're using an 'elements' witness_utxo and issue 3998 has been
resolved, this should work as expected.
2020-09-10 12:31:36 +09:30
ZmnSCPxj jxPCSnmZ
c6186829df tests/test_wallet.py: Simple multiwithdraw tests. 2020-09-09 20:36:08 +09:30
Christian Decker
c7f5889a32 pytest: Skip test_sign_and_send_psbt for elements
This test is flaky because the generated PSBT seems to not have the change
output adjusted, or it is missing.

Tracking-Issue: ElementsProject/lightning#3998
2020-09-09 20:17:26 +09:30
niftynei
437ca83fa5 elementsd: add a fee output to a fund/utxopsbt for elements transactions
fundpsbt / utxopsbt create a (typically) output-less PSBT,
however for elements we require the fees to be encapsulated in an
output.

this patch updates fundpsbt / utxopsbt to add a fee output for elements
transactions. includes test updates.

Fixes #3998
2020-09-09 16:50:14 +09:30
Rusty Russell
83298c030a wallet: switch over to withdraw in module, remove lots of unused code.
This removes the reservation cleanup at startup, too, now they're all
using 'reserved_til'.

This changes test_withdraw, since it asserted that outputs were marked
spent as soon as we broadcast a transaction: now they're reserved until
it's mined.  Similarly, test_addfunds_from_block assumed we'd see funds
as soon as we broadcast the tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `withdraw` now randomizes input and output order, not BIP69.
2020-09-08 10:14:42 +09:30
Rusty Russell
6b2a3f4dfb txprepare: remove old code, switch to plugin.
Some minor phrasing differences cause test changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: txprepare reservations stay across restarts: use fundpsbt/reservepsbt/unreservepsbt
Changelog-Removed: txprepare `destination` `satoshi` argument form removed (deprecated v0.7.3)
2020-09-08 10:14:42 +09:30
Rusty Russell
843d3d6745 pytest: remove test_txprepare_restart
This tested that we unreserved everything on restart, which we no
longer do.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell
7435d50970 signpsbt: add signonly parameter to restrict/enforce what inputs to sign.
This is an extra safety check for dual funding, where we only want to sign
the inputs we provided!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `signpsbt` takes an optional `signonly` array to limit what inputs to sign.
2020-08-18 11:38:25 -05:00
Rusty Russell
aab3808668 utxopsbt: let caller specify locktime, add tests and python binding.
Changelog-Added: JSON-RPC: `utxopsbt` takes a new `locktime` parameter
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-18 11:38:25 -05:00
Rusty Russell
14baaaa8ba fundpsbt: let caller specify locktime.
Required for dual funding where the opener sets it.

Changelog-Added: JSON-RPC: `fundpsbt` takes a new `locktime` parameter
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-18 11:38:25 -05:00
Rusty Russell
0fa5cde782 pytest: fix occasional error in test_sign_and_send_psbt:
The modded_psbt is not necessarily invalid:

       # Try a modified (invalid) PSBT string
        modded_psbt = psbt[:-3] + 'A' + psbt[-3:]
        with pytest.raises(RpcError, match=r"should be a PSBT, not"):
>           l1.rpc.signpsbt(modded_psbt)
E           AssertionError: Pattern 'should be a PSBT, not' not found in 'RPC call failed: method: signpsbt, payload: {'psbt': 'cHNidP8BAM0CAAAABJ9446mTRp/ml8OxSLC1hEvrcxG1L02AG7YZ4syHon2sAQAAAAD9////JFJH/NjKwjwrP9myuU68G7t8Q4VIChH0KUkZ5hSAyqcAAAAAAP3///8Uhrj0XDRhGRno8V7qEe4hHvZcmEjt3LQSIXWc+QU2tAEAAAAA/f///wstLikuBrgZJI83VPaY8aM7aPe5U6TMb06+jvGYzQLEAQAAAAD9////AcDGLQAAAAAAFgAUyQltQ/QI6lJgICYsza18hRa5KoEAAAAAAAEBH0BCDwAAAAAAFgAUqc1Qh7Q5kY1noDksmj7cJmHaIbQAAQEfQEIPAAAAAAAWABS3bdYeQbXvBSryHNoyYIiMBwu5rwABASBAQg8AAAAAABepFD1r0NuqAA+R7zDiXrlP7J+/PcNZhwEEFgAUKvGgVL/ThjWE/P1oORVXh/ObucYAAQEgQEIPAAAAAAAXqRRsrE5ugA1VJnAith5+msRMUTwl8ocBBBYAFMrfGCiLi0ZnOCY83ERKJ1sLYMY8AAAA='}, error: {'code': -1, 'message': 'No wallet inputs to sign'}'

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-13 10:39:56 +09:30
Rusty Russell
73a5f5b313 fundpsbt: make parameters more usable.
fundpsbt forces the caller to manually add their weight * feerate
to the satoshis they ask for.  That means no named feerates.

Instead, create a startweight parameter and do the calc for them
internally, and return the feerate we used (and, while we're at it,
the estimated final weight).

This API change is best done now, as it would otherwise have to
be appended as a parameter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-17 13:53:46 +02:00
Rusty Russell
371cabf976 txprepare: revert 1fb9a078b6 (psbt field)
We're actually going to deprecate this, so don't add new features!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: ***REMOVE*** JSON-API: `txprepare` returns a psbt version of the created transaction
2020-07-15 18:49:02 +09:30
Rusty Russell
e85c895c5b pytest: restore test_sign_and_send_psbt.
It uses reservations heavily, and assumed we generated change, etc.
It's now a simpler test, in many ways.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell
31d7e013bd pytest: test fundpsbt.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell
4ee527a59c pytest: test reserve and unreserve.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell
56ea215ba0 wallet: new JSON commands reserveinputs and unreserveinputs.
reserveinputs marks UTXOs reserved for 12 hours, so we won't select them
for spending: unreserveinputs marks them available again.

Exposes param_psbt() for wider use.

Disabled the test_sign_and_send_psbt since we're altering the API;
the final patch restores it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
niftynei
bea05b1250 elements,tests: skip over fee type outputs
They don't contain addresses
2020-07-13 11:37:24 +09:30
niftynei
eec5dfe6aa elements: update output amounts for elements test
The amount we expect back, post change, is different for an elements tx
2020-07-13 11:37:24 +09:30
Rusty Russell
7aa8ffa2a0 bitcoin: add weight calculation helpers.
These are pulled from wallet/wallet.c, with the fix now that we grind sigs.

This reduces the fees we pay slightly, as you can see in the coinmoves changes.

I now print out all the coin moves in suitable format before we match:
you only see this if the test fails, but it's really helpful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-06 19:25:05 +02:00
niftynei
8185474bfb liquid-compat: update coinmove check to be liquid compatible
🎉 :waterfall:
2020-07-01 19:50:02 -05:00
ZmnSCPxj jxPCSnmZ
d0c85033d2 wallet/walletrpc.c: txprepared transactions now use current tip blockheight by default.
Changelog-Changed: `txprepare` now prepares transactions whose `nLockTime` is set to the tip blockheight, instead of using 0. `fundchannel` will use `nLockTime` set to the tip blockheight as well.
2020-07-01 15:17:33 +00:00
Rusty Russell
dd4a1a3510 tests: fix flake in test_txprepare_restart
Detection of reserved outputs is async, going via bitcoind.  Wait for them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-30 13:50:30 +00:00
niftynei
2e9c387f45 coin_moves: update withdrawal logic to account for 'variable owner' txs
Our existing coin_moves tracking logic assumed that any tx we had an
input in belonged to *all* of our wallet (not a bad assumption as long
as there was no way to update a tx that spends our wallets)

Now that we've got `signpsbt` implemented, however, we need to be
careful about how we account for withdrawals. For now we do a best guess
at what the feerate is, and lump all of our spent outputs as a
'withdrawal' when it's impossible to disambiguate
2020-06-29 16:10:05 +02:00
niftynei
9830c94778 rpc: new signpsbt + sendpsbt rpcs
Changelog-Added: JSON-RPC: new call `signpsbt` which will add the wallet's signatures to a provided psbt
Changelog-Added: JSON-RPC: new call `sendpsbt` which will finalize and send a signed PSBT
2020-06-29 16:10:05 +02:00
niftynei
103dce63ef reserve/unreserve input: new RPC commands for reserving inputs/outputs
Reserve and unreserve wallet UTXOs using a PSBT which includes those
inputs.

Note that currently we unreserve inputs everytime the node restarts.
This will be addressed in a future commit.

Changelog-Added: JSON-RPC: Adds two new rpc methods, `reserveinputs` and `unreserveinputs`, which allow for reserving or unreserving wallet UTXOs
2020-06-29 16:10:05 +02:00
Antoine Poinsot
4302afd9a5 rpc: don't go below feerate_floor when converting vbytes
We passed below the floor when the user specified `1000perkb`.
Matt Whitlock says :

    I was withdrawing with feerate=1000perkb, which should be the minimum-allowed fee rate. Indeed, bitcoin-cli getmempoolinfo reports:

    {
      "loaded": true,
      "size": 15097,
      "bytes": 9207924,
      "usage": 32831760,
      "maxmempool": 64000000,
      "mempoolminfee": 0.00001000,
      "minrelaytxfee": 0.00001000
    }

Changelog-fixed: rpc: The `feerate` parameters now correctly handle the standardness minimum when passed as `perkb`.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
Reported-by: Matt Whitlock
2020-06-18 14:07:00 +02:00
Antoine Poinsot
0b99e2cc42 pytest: reproduce issue #3591
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-06-18 14:07:00 +02:00
lisa neigut
1fb9a078b6 txprepare: return psbt serialized version of tx as well
Changelog-Added: JSON-API: `txprepare` returns a psbt version of the created transaction
2020-05-21 18:45:07 +09:30
darosior
eead65350f wallet: allow to withdraw with unconfirmed utxos
Changelog-Fixed: Passing 0 as minconf to withdraw allows you to use unconfirmed transaction outputs, even if explicitly passed as the `utxos` parameter
2020-04-01 22:58:18 -05:00
darosior
cd15cec2f3 txprepare: don't crash if we are passed unconfirmed utxos
Changelog-added: `txprepare` doesn't crash lightningd anymore if you pass unconfirmed utxos
2020-02-21 17:52:02 +01:00
darosior
9e9e23c81a pytest: test txprepare with unconfirmed utxos 2020-02-21 17:52:02 +01:00
darosior
d2638c99af pytest: test withdrawal transactions' nLockTime 2020-02-03 00:45:27 +00:00
lisa neigut
d36af2c340 txprepare: make output finding map for withdraw_tx variable len
The number of outputs got updated, but the map used to calculate the
change output's location did not (still assumes only one output). This
patch fixes this to make the output map a variable size.

Changelog-Fixed: JSON API: `txprepare` no longer crashes when more than two outputs are specified
2020-01-27 22:59:41 +01:00
lisa neigut
f15d34465d txprepare: add test for multiple withdraw-to outputs
adds (currently failing test) for a txprepare with more than two outputs
2020-01-27 22:59:41 +01:00
lisa neigut
1763960c4a txprepare: don't crash on empty args
add test for empty args check
2020-01-27 22:59:41 +01:00
darosior
dc3aa33927 pytest: use pyln.client for functional tests 2020-01-21 22:38:14 +01:00
Christian Decker
3def5393d8 pytest: Stabilize the hsm_encryption test
Tailing the log when we expect the process to exit can be racy (in this case
we were waiting for the last line that'd be printed).
2019-12-11 16:18:34 +01:00
Christian Decker
15f04d5fff pytest: Give details about which node exited with a return code != 0 2019-12-11 16:18:34 +01:00
Christian Decker
c84473f82c hsm: Stabilize the hsm encryption and decryption tests
We were using sleeps to hope we catch the password prompt. This makes the test
flaky. So I added a help text followed by a `fflush` to make sure we catcht he
right moment, instead of guessing. The `fflush` is also useful for debugging
if a user ever pipes the output to a file it'd get buffered and the user would
wait forever. The same applies for automated systems such as `expect` or
`pexpect` based scripts that enter the password on prompt.
2019-11-29 15:06:39 +01:00
Rusty Russell
aab83e729b lightningd: change config-dir from plugin / wallet / hsm POV into <network> subdir
Changelog-changed: .lightningd plugins and files moved into <network>/ subdir
Changelog-changed: WARNING: If you don't have a config file, you now may need to specify the network to lightning-cli
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Christian Decker
3482d254e3 pytest: Fix a test assuming a specific DB order
Postgresql does not guarantee insertion order when querying, so we need to
actually go and look for the correct transaction.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-11-22 03:48:54 +00:00
Michael Schmoock
d7609e61b5 pytest: add test for raising listtransactions after funding 2019-11-15 03:53:56 +00:00
darosior
f97d548582 pytest: test hsm_secret management with hsmtool 2019-11-12 05:29:19 +00:00
darosior
1171b5dba2 pytest: clean up hsm_secret_encryption test 2019-11-12 05:29:19 +00:00
darosior
1c27545f9e pytest: test hsm_secret encryption 2019-10-09 22:00:38 -05:00
trueptolemy
722974a36f pytest: Test compact of the old-style txprepare 2019-10-09 16:51:42 -05:00
Rusty Russell
93865bb0f3 wallet: minor style fixes, and remove null JSON fields.
Our policy is generally to omit fields which aren't sensible.
Also, @niftynei points out the spacing in for loops.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-07 20:03:24 -05:00
Christian Decker
bf859bd919 pytest: Test transaction annotations for deposit and channel open 2019-10-07 20:03:24 -05:00
Christian Decker
8a69ea7bf5 elements: Parametrize the sample addresses we test against
Since elements addresses look quite different from the bitcoin mainnet
addresses I just added a sample to the chainparams fixture. In addition I
extracted some of the fixed strings to reference chainparams instead.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
lisa neigut
5c70890efa rpc: add 'utxos' parameter to txprepare + withdraw
Allow a user to select the utxo set that will be added to a
transaction, via the `utxos` parameter. Optional.

Format for utxos should be of the form ["txid:vout","..."]
2019-09-11 23:56:27 +00:00
trueptolemy
74055eada5 pytest: Test txprepare with multiple outputs 2019-09-05 16:05:36 -05:00
trueptolemy
cdcafdaf74 API: txprepare now support mutiple outputs 2019-09-05 16:05:36 -05:00
lisa neigut
10ed2ebdb4 wallet: track our change outputs
Add change scripts to our txfilter so that we mark them as confirmed
(and can subsequently spend them)
2019-09-05 15:48:49 +02:00
lisa neigut
01e0ba8232 test: add check that withdraw's change is confirmed
We're not tracking change outputs for withdraws, so they're not
confirming. This test exposes the problem.
2019-09-05 15:48:49 +02:00
Rusty Russell
e1dbc0b12b wallet: clean reserved inputs on startup.
We reserve inputs when we're going to send a transaction, but we don't
unreserve them if we crash.  This is most graphically demonstrated by
the txprepare case, which makes it easier to trigger.

Instead, we should query bitcoind to see whether the tx made it out or
not, as we would do manually with dev-rescan-outputs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-06 04:47:44 +00:00
Rusty Russell
65505adbab pytest: test that we unreserve txprepare inputs across shutdown/crash.
We fail this at the moment, since we rely on shutdown to do the cleanups
for us.

(Also had to fix the unclean shutdown path: the caller checks the rc unless
 mayfail is set, and of course it's not zero since we just SIGTERM'd it).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-06 04:47:44 +00:00
Rusty Russell
cdab75ecee pytest: test txprepare/txdiscard/txsend.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-06 04:47:44 +00:00
Rusty Russell
0543149b89 pytest: move wallet tests into new file tests/test_wallet.py
We're going to add some more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-06 04:47:44 +00:00