Commit Graph

9125 Commits

Author SHA1 Message Date
Antoine Poinsot
45bb1bfa3c hsmd: cleanup encrypted hsm_secret detection
This makes use of the constant defined in the previous commits to more
accurately detect plaintext, encrypted, and invalid seeds. We now error
on invalid seeds.

Changelog-changed: hsmd: we now error at startup on invalid hsm_secret
Changelog-changed: hsmtool: all commands now error on invalid hsm_secret
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
d2a903992c hsmd: regroup hsm_secret decryption logic
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
c6bc22b0f5 hsmd: group hsm_secret encryption
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
a441485a35 lightningd: regroup hsm_secret password input logic
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
917f78a4f8 lightningd: group hsm_secret encryption key derivation
This avoids duplication of both logic and error-prone values, such as
the salt. Grouping all hsm encryption logic into a public API will also
allow us to fuzz it.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
56c223886c lightning: confirm password on hsm_secret encryption
Changelog-changed: lightningd: the `--encrypted-hsm` now asks you to confirm your password when first set
Changelog-changed: hsmtool: the `encrypt` now asks you to confirm your password
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
38204f9b6d hsmtool: use errx() instead of err() everywhere
errx() was printing the confusing errno as well ("Error could not [...] :Success")

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Rusty Russell
d41b383cf7 pyln-testing: suppress mypy on FileLock
3.0.12 here:
```
pyln/testing/utils.py:9: error: Cannot find implementation or library stub for module named 'filelock'
pyln/testing/utils.py:9: error: Cannot find implementation or library stub for module named 'filelock'
pyln/testing/utils.py:9: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-06 12:15:39 +01:00
niftynei
0c9072272f df-plugin: add signonly to signing in df_accepter
We're the accepter, so we look up which inputs we need to sign and only
sign those.
2021-01-05 19:12:00 +01:00
niftynei
d98bd39d60 dual_open: neaten spacing for macro 2021-01-05 19:12:00 +01:00
niftynei
55bbdf1866 df: update an error message to give more info 2021-01-05 19:12:00 +01:00
niftynei
713bb7491b json: allow helper to take/free psbt
Makes things a bit cleaner elsewhere
2021-01-05 19:12:00 +01:00
niftynei
e3a1d1a7f3 pyln-client: to_whole_satoshi returns the rounded up satoshi value
A fractional satoshi value isn't really useful; rounding up loses
precision but that's why you called "whole satoshi", wasn't it?

Changelog-Changed: pyln-client: Millisatoshi has new method, `to_whole_satoshi`; *rounds value up* to the nearest whole satoshi
2021-01-05 19:12:00 +01:00
niftynei
cce0020568 pyln-client: add missing min_witness_weight to fundpsbt/utxopsbt
We need to use it for the 'df_accepter' plugin, so we get the feerate
correct.

Changelog-Added: pyln-client: `fundpsbt`/`utxopsbt` now support `min_witness_weight` param
2021-01-05 19:12:00 +01:00
Antoine Poinsot
19afe15968 hsmtool: only write 32 bytes to hsm_secret on generatehsm
Fixes #4302
Changelog-fixed: hsmtool: the `generatehsm` command now generates an appropriately-sized hsm_secret
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 19:02:28 +01:00
Antoine Poinsot
aa2586a45a pytest: reproduce issue #4302
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 19:02:28 +01:00
Antoine Poinsot
dd1def335c hsmtool: reuse read_stdin_pass for generatehsm passphrase
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 16:15:40 +01:00
Antoine Poinsot
1513a2d07e hsmtool: input encryption password from stdin
This slightly breaks the API, but still accept the input: we just don't
take it into account anymore.

For `dumponchaindescriptors`, we have to still take the old place of the
`network` parameter into account to not entirely break the API.

Changelog-Added: hsmtool: password must now be entered on stdin. Password passed on the command line are discarded.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 16:15:40 +01:00
Christian Decker
5b3af00e28 pytest: Really only check DB statements if the env-var is "1" 2021-01-04 11:25:50 +01:00
Christian Decker
94820ad559 pytest: Disable test_funding_cancel_race on slow machines
The CI regularly trips up on this very taxing test (100 nodes) so only
run it if we have the required horsepower.
2021-01-04 11:25:50 +01:00
Christian Decker
dbb83a1bc3 pyln: Do not rstrip() the return value of .append()
This was causing the following error

```
Exception in thread Thread-553:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/tmp/cirrus-ci-build/contrib/pyln-testing/pyln/testing/utils.py", line 232, in tail
    self.err_logs.append(line.rstrip().decode('UTF-8', 'replace')).rstrip()
AttributeError: 'NoneType' object has no attribute 'rstrip'

[gw5] [ 33%] FAILED tests/test_misc.py::test_bitcoin_failure
```

Notice the second call to `.rstrip()` on the return value of `.append()`
2021-01-04 11:25:50 +01:00
Christian Decker
b3fd47e5f6 pytest: Stabilize test_routing_gossip
We were getting bad gossip because some nodes discarded the channel
announcement for being in the future. This is because the node was, at
that time, below the confirmation height. It'd then discard the
followup messages because not preceded by an announcement, and getting
upset about that.
2021-01-04 11:25:50 +01:00
Christian Decker
1ff57f07f8 pyln: Check that the stderr line is not none in pyln-testing tail
I stumbled over this in a test run and it seems benign.
2021-01-04 11:25:50 +01:00
Christian Decker
7e867e5ee6 pytest: Add throttler to limit load on the test system
Both my machine and apparently the CI tester machines regularly run
into issues with load on the system, causing timeouts (and
unresponsiveness). The throttler throttles the speed with which new
instances of c-lightning get started to avoid overloading. Since the
plugin used for parallelism when testing spawns multiple processes we
need to lock on the fs. Since we have that file open already, we'll
also write a couple of performance metics to it.
2021-01-04 11:25:50 +01:00
Christian Decker
f2a0a4abfc pyln: Add logging to bitcoind RPC calls
Useful if we want to debug a bit better
2021-01-04 11:25:50 +01:00
Christian Decker
8b09ac38aa pyln: Log mempool when we generate blocks in BitcoinD fixture
We sometimes have very specific sequences of tx broadcasts and blocks
being generated to confirm them. If the confirmation is missed the
test can completely get out of sync. Make debugging this easier by
logging what we confirmed.
2021-01-04 11:25:50 +01:00
fiatjaf
6af8adde26 fix memset for route_hop json parsing when no channel_id exists.
Changelog-None
2021-01-04 11:24:55 +01:00
niftynei
6030d0542a bcli: print error message in debug log for broadcast failures 2020-12-24 11:38:49 +01:00
niftynei
47d9ff88df bcli: only print error if exitcode is non-zero 2020-12-24 11:38:49 +01:00
niftynei
08c8581461 bcli: allocate temp string off of short-lived cmd
`cmd` gets cleaned up almost immediately, and is the right scope for a
temporary string allocation
2020-12-24 11:38:49 +01:00
niftynei
f9a2489424 bcli: print sendrawtx every time (even on failures) 2020-12-24 11:38:49 +01:00
Rusty Russell
52af729641 plugins/offer and plugins/fetchinvoice: send and recv errors.
This also lets us extend our testing to cover error cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
96caf9f4ab fetchinvoice: return the next period for recurring offers.
This is useful for the caller to know when to call again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
19af1d10e6 invoice: hack in merkle of invoice as "payment_secret" (EXPERIMENTAL_FEATURES)
This lets actually pay the invoice that fetchinvoice returns.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
2baa410e2b pytest: finally test fetchinvoice send, and offer plugin receive.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
69ddf818c7 plugins/fetchinvoice: catch invoice_request reply.
Using onionmessage hook, we get the response and either present it
to the user (invoice) or return the error to the user.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
3eada358e9 plugins/fetchinvoice: plugin to send an invoice_request for a given offer
Doesn't catch the reply yet, but prepares the invoice request based on
the offer and sends it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
43b71de897 lightningd: low-level createinvoicerequest API (EXPERIMENTAL_FEATURES)
This is similar to the createinvoice API, except we don't need to save
invoice requests in the database.  We may, however, have to look up
payment_key for recurring invoice requests, and sign the message with
the payment_key.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
59efd160c1 hsmd: code to sign bolt12 messages with a tweaked key.
Invoices are signed with our own key, but we use a transient payer_key with a
tweak for invoice_requests (and refunds).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
9d75fbe237 plugins/offers: add code to catch invoice_request onion messages.
We prepare an invoice, but there's no code to send an error reply yet
(future patch).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
08e110b568 JSON: offers plugin offer command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
26e0cbeb7c plugin/offers: skeleton for offers plugin (EXPERIMENTAL_FEATURES)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
1293852542 common/bolt12: add calculations for offer periods.
It's not trivial, but fortunately gmtime and mktime exist already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
353e4cf3c2 wire/Makefile: make generated wire files depend on Makefile.
That way they rebuild when we change flags in Makefile.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Karol Hosiawa
e582c0ac0a Added listfunds test 2020-12-22 13:00:02 +01:00
Karol Hosiawa
4a3dec660f Added spent option to listfunds in pyln-client 2020-12-22 13:00:02 +01:00
Karol Hosiawa
a2c208e121 Added spent option to listfunds
Changelog-Added: JSON-RPC: The `listfunds` method now includes spent outputs if the `spent` parameter is set to true.
2020-12-22 13:00:02 +01:00
Rusty Russell
eb4062bb14 setchannelfee: fix crash when channel is not in valid state.
You can't fail a cmd when you've already started streaming
a successful response:

lightningd: ccan/ccan/json_out/json_out.c:343: json_out_finished: Assertion `tal_count(jout->wrapping) == 0' failed.
lightningd: FATAL SIGNAL 6 (version v0.9.2-119-gf7cdf1d)
0x1847d1 send_backtrace
	common/daemon.c:38
0x184877 crashdump
	common/daemon.c:51
0x5bda03f ???
	???:0
0x5bd9fb7 ???
	???:0
0x5bdb920 ???
	???:0
0x5bcb489 ???
	???:0
0x5bcb501 ???
	???:0
0x1e07a8 json_out_finished
	ccan/ccan/json_out/json_out.c:343
0x18db0a json_stream_double_cr
	common/json_stream.c:95
0x18dbf3 json_stream_close
	common/json_stream.c:117
0x12fd98 command_raw_complete
	lightningd/jsonrpc.c:459
0x12fec9 command_failed
	lightningd/jsonrpc.c:488
0x12ffb9 command_fail
	lightningd/jsonrpc.c:503
0x14dc20 json_setchannelfee
	lightningd/peer_control.c:2052

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSONRPC: `setchannelfee` would fail an assertion if channel wasn't in normal state.
2020-12-21 17:10:28 -06:00
fiatjaf
0fb4c1112d add a note to "unknown parameter" error about -k autodetect failure. 2020-12-21 12:17:43 +00:00
Rusty Russell
77478408f9 pyln: add RpcException for finer method failure control.
Allows caller to set code and exact message to be returned.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln-client: plugins can now raise RpcException for finer control over error returns.
2020-12-16 12:37:14 +01:00