Commit Graph

1117 Commits

Author SHA1 Message Date
lisa neigut
f6ff5e5b19 connectd: make failure message more descriptive 2019-12-04 15:32:31 -06:00
Saibato
2c16b410ab Add basic tests
Signed-off-by: Saibato <saibato.naga@pm.me>
2019-12-03 23:35:18 +01:00
fiatjaf
c0368a717a also return preimage on listinvoices. 2019-12-03 01:16:07 +00:00
darosior
d9aeeb4d14 pytest: skip test_sendonion if not DEVELOPER 2019-12-02 14:41:13 +01:00
darosior
4661dfa590 tests: update requirements
cheroot release(/"changes"?) notes:
    #218 via PR #219: Fix HTTP parser to return 400 on invalid major-only HTTP version in Request-Line.
    #198 via 9f7affe: Fix race condition when toggling stats counting in the middle of request processing.
    Improve post Python 3.9 compatibility checks.
    Fix support of abstract namespace sockets.
    #222 via 621f4ee: Fix socket.SO_PEERCRED constant fallback value under PowerPC
    Revisit PR #85 under PR #221. Now backports.functools_lru_cache is only required on Python 3.2 and earlier.
    CherryPy #1206 via PR #204: Fix race condition in threadpool shrink code.
    PR #224: Refactored “open URL” behavior in webtest to rely on retry_call. Callers can no longer pass raise_subcls or ssl_context positionally, but must pass them as keyword arguments.
    #231 via PR #232: Remove custom setup.cfg parser handling, allowing the project (including sdist) to build/run on setuptools 41.4. Now building cheroot requires setuptools 30.3 or later (for declarative config support) and preferably 34.4 or later (as indicated in pyproject.toml).
    Workers are now request-based, addressing the long-standing issue with keep-alive connections (#91 via PR #199).
    Deprecated use of negative timeouts as alias for infinite timeouts in ThreadPool.stop.
    CherryPy #1662 via PR #74: For OPTION requests, bypass URI as path if it does not appear absolute.
    CherryPy #1818: Restore support for None default argument to WebCase.getPage().

https://github.com/cherrypy/cheroot/blob/master/CHANGES.rst

flaky changes:
    Bugfixes - Reraise KeyboardInterrupt when running tests under pytest.

https://github.com/box/flaky/blob/v3.6.1/HISTORY.rst#361-2019-08-06

python-bitcoinlib:
    New RPC `generatetoaddress(self,numblocks,addr)`.
    Fixed Python 2.7 incompatibility.
    Various OpenSSL fixes, including a memory leak.

https://github.com/petertodd/python-bitcoinlib/blob/python-bitcoinlib-v0.10.2/release-notes.md#v0102

pytest release notes:
    A lot of misc fixes, see https://docs.pytest.org/en/latest/changelog.html.
2019-12-02 14:41:13 +01:00
Christian Decker
21b5b59c8c json-rpc: Rename onion payload type to style
Suggested-by: Rusty Russell <@rustyrussell>
2019-12-01 15:40:47 +01:00
Christian Decker
16b6c31010 json-rpc: Add the error onion if we stored it in the DB
If we can't decode the onion, because the onion got corrupted or we used
`sendonion` without specifying the `shared_secrets` used, the best we can do
is tell the caller instead.
2019-12-01 15:40:47 +01:00
Christian Decker
27547ce8d4 pay: Allow sendonion callers to provide shared_secrets
This means that c-lightning can now internally decrypt an eventual error
message, and not force the caller to implement the decryption. The main
difficulty was that we now have a new state (channels and nodes not specified,
while shared_secrets are specified) which needed to be handled.
2019-12-01 15:40:47 +01:00
Christian Decker
b8ce175fd2 pay: Handle payment failures resulting from sendonion correctly
We are breaking with a couple of assumptions, namely that we have the
`path_secrets` to decode the error onion. If this happens we just want it to
error out.
2019-12-01 15:40:47 +01:00
Christian Decker
cb912c9ae8 pytest: Add a test for the sendonion method 2019-12-01 15:40:47 +01:00
Christian Decker
8daf9b2577 pytest: Add a test for createonion 2019-12-01 15:40:47 +01:00
Jarret Dyrbye
d712f732d8 pylightning - allow unicode symbols in the description to be passed as-is to the daemon
addresses issue #2753.

Formatting the JSON with the default parameters will escape the unicode
symbols in a way that c-lightning won't allow, leading to an exception.

Changelog-Fixed: `pylightning` now handles unicode characters in JSON-RPC requests and responses correctly.
2019-11-29 21:38:17 +01:00
Christian Decker
dc1f27c0b3 pytest: Add failing test for unicode in JSON-RPC through pylightning 2019-11-29 21:38:17 +01:00
Christian Decker
c84473f82c hsm: Stabilize the hsm encryption and decryption tests
We were using sleeps to hope we catch the password prompt. This makes the test
flaky. So I added a help text followed by a `fflush` to make sure we catcht he
right moment, instead of guessing. The `fflush` is also useful for debugging
if a user ever pipes the output to a file it'd get buffered and the user would
wait forever. The same applies for automated systems such as `expect` or
`pexpect` based scripts that enter the password on prompt.
2019-11-29 15:06:39 +01:00
Michael Schmoock
6ed320189e feat: restructure plugin and options in listconfigs
This will change the command `listconfigs` output in several ways:

 - Deprecated the duplicated "plugin" JSON output by replacing it with
 - a "plugins" array with substructures for each plugin with:
 - path, name and their options

Changelog-Changed: JSON-RPC: `listconfigs` now structures plugins and include their options
Changelog-Deprecated: JSON-RPC: `listconfigs` duplicated "plugin" paths
2019-11-25 18:15:02 +01:00
Rusty Russell
8393d21503 common/features: add payment_secret feature if EXPERIMENTAL.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell
3b37c9d426 common/features: clean up feature handling for different cases.
The spec is (RSN!) going to explicitly denote where each feature should
be presented, so create that infrastructure.

Incorporate the new proposed bolt11 features, which need this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell
e5247a68b6 lightningd: check payment secret on htlc receipt.
We don't set the secret to compulsory (yet!) but put code in for the
future.  Meanwhile, if there is a secret, check it is correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell
14997f6237 lightningd: fix handling of truncated config options.
Do the same thing '--help' does with them; append `...`.

Valgrind noticed that we weren't NUL-terminarting if answer was over
78 characters.

Changelog-Fixed: JSONRPC: listconfigs appends '...' to truncated config options.
2019-11-23 22:42:34 +00:00
Rusty Russell
badeaf1a67 pytest: upgrade tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell
57a32ee044 pytest: test for config parsing errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell
aab83e729b lightningd: change config-dir from plugin / wallet / hsm POV into <network> subdir
Changelog-changed: .lightningd plugins and files moved into <network>/ subdir
Changelog-changed: WARNING: If you don't have a config file, you now may need to specify the network to lightning-cli
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell
dc23c308e4 config: Read both top-level and network-subdir config files.
This lets you have a default, but also a network-specific config.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Options: `config` and <network>/`config` read by default.
2019-11-23 22:42:34 +00:00
Rusty Russell
34c89cb226 config: Add include directive support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: configuration files now support `include`.
2019-11-23 22:42:34 +00:00
Christian Decker
5a233a1b27 pytest: Fix TLV hook tests in experimental mode
I missed this test breaking with `EXPERIMENTAL_FEATURES` in #3261
2019-11-22 03:48:54 +00:00
Christian Decker
3482d254e3 pytest: Fix a test assuming a specific DB order
Postgresql does not guarantee insertion order when querying, so we need to
actually go and look for the correct transaction.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-11-22 03:48:54 +00:00
lisa neigut
de16d0f0b4 openchannel hook: add new close_to field
Rounds out the application of `upfront_shutdown_script`, allowing
an accepting node to specify a close_to address.

Prior to this, only the opening node could specify one.

Changelog-Added: Plugins: Allow the 'accepter' to specify an upfront_shutdown_script for a channel via a `close_to` field in the openchannel hook result
2019-11-22 00:41:15 +00:00
Rusty Russell
654faa6174 lightningd: don't start if bitcoind is behind.
This leads to all sorts of problems; in particular it's incredibly
slow (days, weeks!)  if bitcoind is a long way back.  This also changes
the behaviour of a rescan argument referring to a future block: we will
also refuse to start in that case, which I think is the correct behavior.

We already ignore bitcoind if it goes backwards while we're running.

Also cover a false positive memleak.

Changelog-Fixed: If bitcoind goes backwards (e.g. reindex) refuse to start (unless forced with --rescan).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-21 05:18:29 +00:00
Rusty Russell
d4b48a6640 pytest: test that we refuse to follow bitcoind backwards.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-21 05:18:29 +00:00
Christian Decker
dda792c766 plugin: Pass the full raw_payload including realm to htlc_accepted
So far we've only handled legacy payloads, which meant we could drop the realm
byte since it was always 0x00. Once we start handling TLV payloads the first
byte, i.e., the former realm byte, is important since it gives us the length
of the payload. This is a breaking change, however I don't think there's
anyone using the `raw_payload` as of yet.

Changelog-Changed: JSON-RPC: the `raw_payload` now includes the first byte, i.e., the realm byte, of the payload as well. This allows correct decoding of a TLV payload in the plugins.
2019-11-18 12:00:15 +01:00
Rusty Russell
3437f7e25d devtools/gossipwith: change timeout to seconds.
I always get this wrong, then wonder why it's dying!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 11:01:20 +01:00
Rusty Russell
709c98f539 logging: remove spaces from subsystem names.
Spaces just make life a little harder for everyone.

(Plus, fix documentation: it's 'jsonrpc' not 'json' subsystem).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
f91ff993ea pytest: deflake test_htlc_send_timeout
We can definitely get a pong from l1 (should test be slow enough):
it's l3 we are concerned about.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
4fc498f901 lightningd: enable io logging on subdaemons iff we're going to print it.
This simplifies our tests, too, since we don't need a magic option to
enable io logging in subdaemons.

Note that test_bad_onion still takes too long, due to a separate minor
bug, so that's marked and left dev-only for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
ef7a820ab1 log: make formatting more consistent.
1. Printed form is always "[<nodeid>-]<prefix>: <string>"
2. "jcon fd %i" becomes "jsonrpc #%i".
3. "jsonrpc" log is only used once, and is removed.
4. "database" log prefix is use for db accesses.
5. "lightningd(%i)" becomes simply "lightningd" without the pid.
6. The "lightningd_" prefix is stripped from subd log prefixes, and pid removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Logging: formatting made uniform: [NODEID-]SUBSYSTEM: MESSAGE
Changelog-removed: `lightning_` prefixes removed from subdaemon names, including in listpeers `owner` field.
2019-11-18 04:50:22 +00:00
Rusty Russell
eed654f684 connectd, gossipd: use per-peer logging.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
e433d4ddc1 lightningd: have logging include an optional node_id for each entry.
A log can have a default node_id, which can be overridden on a per-entry
basis.  This changes the format of logging, so some tests need rework.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Michael Schmoock
d7609e61b5 pytest: add test for raising listtransactions after funding 2019-11-15 03:53:56 +00:00
Rusty Russell
21555b277d tests: test that we only use tlv onion when advertized.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Rusty Russell
bb06bec891 features: set OPT_VAR_ONION (bit 9) iff EXPERIMENTAL_FEATURES
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Rusty Russell
2a2259083a lightningd: handle tlv-style payloads.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: JSON API: `htlc_accepted` hook has `type` (currently `legacy` or `tlv`) and other fields directly inside `onion`.
Changelog-deprecated: JSON API: `htlc_accepted` hook `per_hop_v0` object deprecated, as is `short_channel_id` for the final hop.
2019-11-14 10:15:33 +01:00
lisa neigut
a333df449a listpeers: show close_to address
If a 'upfront_shutdown_script' was specified, show the address +
scriptpubky in `listpeers`

Changelog-added: JSON API: `listpeers` channels now include `close_to` and `close_to_addr` iff a `close_to` address was specified at channel open
2019-11-13 03:31:20 +00:00
Christian Decker
9378be72a9 pyln: Allow users to override the LightningNode class
Quite a few of the things in the LightningNode class are tailored to their use
in the c-lightning tests, so I decided to split those customizations out into
a sub-class, and adding one more fixture that just serves the class. This
allows us to override the LightningNode implementation in our own tests, while
still having sane defaults for other users.
2019-11-12 21:23:55 +01:00
Christian Decker
d58339b872 pytest: Migrate to in-tree pyln-testing 2019-11-12 21:23:55 +01:00
Christian Decker
b7e7e535c8 pytest: Migrate env to pyln-testing 2019-11-12 21:23:55 +01:00
Christian Decker
f6a016c17d pytest: Consolidate access to environment variables
We use `env()` to look up configuration variables and allow them to be
overridden by the environment.
2019-11-12 21:23:55 +01:00
darosior
f97d548582 pytest: test hsm_secret management with hsmtool 2019-11-12 05:29:19 +00:00
darosior
1171b5dba2 pytest: clean up hsm_secret_encryption test 2019-11-12 05:29:19 +00:00
darosior
95c6513ff3 pytest: test the 'rpc_command' hook 2019-11-11 22:30:01 +01:00
darosior
208123771e pytest: test shadow routing in pay plugin 2019-11-08 03:27:58 +00:00