Commit Graph

9363 Commits

Author SHA1 Message Date
Rusty Russell
f2d2db7b4e plugins/offer: handle receiving an invoice in an onion_message.
And if we have a matching `send_invoice` offer, try to pay it!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell
6d1fe7e599 plugin/offers: hoist send_onion_reply into core.
We want to reuse it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell
43f5304539 plugins/offer: allow offer to create send_invoice, refund and single_use offers.
refund offers are implied send_invoice offers.  And send_invoice offers
are implied single-use offers, though it can also make sense to have
a non-send_invoice offer be single-use.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell
a15878b3a4 sendonion: remove underscores from JSON arg name.
I fixed the "sendpay" one but not this one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Christian Decker
6b6e0fe34c pytest: Mark test_lockup_drain as disabled
It's been causing me headaches for some time, and it's just getting
annoying. Will file an issue to track progress.
2021-01-08 19:28:30 +01:00
Christian Decker
91e406ed6f doc: Document existence of erring_direction in sendpay result
Fixes #3537
2021-01-08 19:28:30 +01:00
Christian Decker
16801e31e9 configure: Prefer pytest over pytest3
When looking for a pytest executable we should be looking for the ones
that `pip` installs in a virtualenv (`pytest` and `py.test`) before we
look for the ones that `apt` installs system-wide (`pytest3` and
`py.test3`) because these may not be part of the virtualenv that all
other packages are installed in.
2021-01-08 19:28:30 +01:00
niftynei
d0dcb7cb42 tests: wait for drop to chain test_onchain_middleman_their_unilateral_in
this was catching the wrong 'sendrawtx exit 0' and consequently
not mining the close tx into a block
2021-01-08 19:28:30 +01:00
Christian Decker
a91254de11 pyln: Add ammag key to onion keyset
This was missing, and is required to wrap error responses.
2021-01-08 19:28:30 +01:00
Rusty Russell
1a8978100e invoice: add ctlv option.
This is required if we want to create a "bouncer" plugin (in my copious free time!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `invoice` now takes an optional `cltv` parameter.
2021-01-08 10:45:56 +01:00
Christian Decker
4b2efd69d7 pay: Fix a crash when waitblockheight times out
Fixes #4309
Changelog-Fixed: pay: Fixed an issue where waiting for the blockchain height to sync could time out.
2021-01-08 09:19:24 +01:00
Christian Decker
ca204eacfa pytest: Reproduce waitblockheight timeout issue #4309 2021-01-08 09:19:24 +01:00
Christian Decker
0d1b45dfcc json-rpc: Give waitblockheight an error code for timeouts
We need to differentiate timeouts from other results, so make it recognizable.
2021-01-08 09:19:24 +01:00
Christian Decker
8e908ff652 Revert "wallet: Remove in-memory utxoset filter"
This reverts commit c239a7161b.

The goal of c239a716 was to reduce the memory footprint of our
internal UTXO set tracking, and testing against the sqlite3 backend
showed no performance impact. We have since found that the added
roundtrips to the DB server with postgres was having a considerable
performance impact, and backups would also bloat due to the increased
number of queries.

Undoing this change skips the noop updates that were causing this
regression.

Changelog-Fixed: db: Fixed a performance regression during block sync, resulting in many more queries against the DB than necessary.
2021-01-08 12:08:32 +10:30
Rusty Russell
3b7d0e7a62 common/json: make json_scan return an error string.
This makes for more useful errors.  It prints where it was up to in
the guide, but doesn't print the entire JSON it's scanning.

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell
53582a0f81 lightningd/bitcoind: use json_scan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell
11a4d46c44 chaintopology: fix crash if bcli doesn't include errmsg.
Interestingly, we required that "sendrawtx" include "errmsg" field even
on success, otherwise we crashed in broadcast_remainder.

We only actually insist on an "errmsg" if success is false.  And this
logic here is weird (the !success) was added by darosior in
947f5ddde1, which makes the msg checks redundant.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell
fd7d1a1cc0 common: remove now-unused json_delve.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell
09b18bf64f libplugin: replace rpc_delve with rpc_scan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell
b61da8c5a9 plugins: use json_scan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell
35e8949df3 plugins/bcli: convert to json_scan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell
a5befb0072 common: enhance json_scan with simple array helpers.
In several places we want to access the first element of an array.
This uses a '[indexnum:xxx]' form which is a bit weird, but works similarly
to the way we specify member matches.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell
553daf17e0 common/json: new helper to scan JSON: json_scan.
This takes a JSON-style format string, and does intelligent parsing,
removing a lot of boilerplate from code which needs to deal with JSON.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell
24b6132ee8 common: add length-limited versions of json_streq and json_get_member.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Rusty Russell
2b6cdfeb5a json_to_psbt: fix API to match other allocating json_to functions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-07 19:32:47 +01:00
Christian Decker
70410b8ee8 pytest: Test LightningRpc and plugin command notification support 2021-01-07 08:32:40 +01:00
Christian Decker
b6650425b9 pyln: Add notifications support to LightningRpc
Changelog-Added: pyln-client: Added support for command notifications to LightningRpc via the `notify` context-manager.
2021-01-07 08:32:40 +01:00
Christian Decker
84b3653606 pyln: Add command notification support for plugins
Changelog-Added: pyln-client: Plugin methods can now report progress or status via the `Request.notify` function
2021-01-07 08:32:40 +01:00
Christian Decker
a1d33c3781 pytest: Add a test for listinvoices
We now have some more logic in the query, so let's test it
exhaustively.
2021-01-06 18:48:06 +01:00
Christian Decker
2981184fac docs: Document bolt11 and payment_hash arguments of listinvoices 2021-01-06 18:48:06 +01:00
Christian Decker
30abda2750 jsonrpc: Allow filtering via bolt11 or payment_hash in listinvoices
A user reported that it is sometimes cumbersome to search an invoice
based on the payment hash or the bolt11 string in the full list, which
may be required when we don't have the label available.

This adds support for querying / filtering based on the `payment_hash`
or `bolt11` string.

Changelog-Added: JSON-RPC: `listinvoices` can now query for an invoice matching a `payment_hash` or a `bolt11` string, in addition to `label`
2021-01-06 18:48:06 +01:00
Rusty Russell
106afd005f channeld: stop taken() leak.
I saw this message:

```
lightning_channeld: outstanding taken(): channeld/channeld.c:3087:blinding
lightning_channeld: outstanding taken(): channeld/channeld.c:3087:blinding
lightning_channeld: outstanding taken(): channeld/channeld.c:3087:blinding
```

The caller does take(blinding), but blinding can be NULL.  We should
move the code around to do the take() earlier anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-06 15:34:02 +01:00
niftynei
0726d2912e df-mfc: remove guard-rails for failures
This will cause blow ups for v2 multifundchannel attempts with failures,
but allows us to return the expected errors for single-shot
fundchannel attempts.

Error handling is coming, i promise
2021-01-06 14:50:02 +01:00
niftynei
b040a893ed mfc: avoid off-by-one rounding errors when finding change fee
Since we round down in `amount_tx_fee`, find the change fee as the
difference between what we've already paid and what the combined/total
fee would be if the change weight were also added.
2021-01-06 14:50:02 +01:00
niftynei
3793687550 mfc-df: if the psbt we've got has inputs from our peer, remove them
We only want to attempt to unreserve inputs that are ours (otherwise we
log a broken error)
2021-01-06 14:50:02 +01:00
niftynei
6e3c517611 mfc-df: if there's an error with a openchannel_signed, fail
Ideally we'd 'cure' the error and re-attempt, except that if this was a
bitcoin-backend 'failure to broadcast' then it really needs user
intervention to figure out what's wrong -- it's possible that the
peer successfully broadcast the transaction
2021-01-06 14:50:02 +01:00
Antoine Poinsot
a4f07a3e78 tests/fuzz: add a fuzz target for hsm_encryption
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
506d524508 libfuzz: add a NULL-termination in to_string
It's more useful if we actually want to use the output as, well, a
string..

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
45bb1bfa3c hsmd: cleanup encrypted hsm_secret detection
This makes use of the constant defined in the previous commits to more
accurately detect plaintext, encrypted, and invalid seeds. We now error
on invalid seeds.

Changelog-changed: hsmd: we now error at startup on invalid hsm_secret
Changelog-changed: hsmtool: all commands now error on invalid hsm_secret
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
d2a903992c hsmd: regroup hsm_secret decryption logic
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
c6bc22b0f5 hsmd: group hsm_secret encryption
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
a441485a35 lightningd: regroup hsm_secret password input logic
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
917f78a4f8 lightningd: group hsm_secret encryption key derivation
This avoids duplication of both logic and error-prone values, such as
the salt. Grouping all hsm encryption logic into a public API will also
allow us to fuzz it.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
56c223886c lightning: confirm password on hsm_secret encryption
Changelog-changed: lightningd: the `--encrypted-hsm` now asks you to confirm your password when first set
Changelog-changed: hsmtool: the `encrypt` now asks you to confirm your password
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
38204f9b6d hsmtool: use errx() instead of err() everywhere
errx() was printing the confusing errno as well ("Error could not [...] :Success")

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Rusty Russell
d41b383cf7 pyln-testing: suppress mypy on FileLock
3.0.12 here:
```
pyln/testing/utils.py:9: error: Cannot find implementation or library stub for module named 'filelock'
pyln/testing/utils.py:9: error: Cannot find implementation or library stub for module named 'filelock'
pyln/testing/utils.py:9: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-06 12:15:39 +01:00
niftynei
0c9072272f df-plugin: add signonly to signing in df_accepter
We're the accepter, so we look up which inputs we need to sign and only
sign those.
2021-01-05 19:12:00 +01:00
niftynei
d98bd39d60 dual_open: neaten spacing for macro 2021-01-05 19:12:00 +01:00
niftynei
55bbdf1866 df: update an error message to give more info 2021-01-05 19:12:00 +01:00
niftynei
713bb7491b json: allow helper to take/free psbt
Makes things a bit cleaner elsewhere
2021-01-05 19:12:00 +01:00