Commit Graph

306 Commits

Author SHA1 Message Date
Christian Decker
87e924e270 docker: Remove Dockerfile for i386 builder
We aren't using it and it's broken with the specified upstream image, so
remove it outright.
2020-02-27 09:18:24 +10:30
Michael Schmoock
dd334df80d fix: makes testutils ln.pay assert status complete
`waitsendpay` in `lightningd/pay.c` can also return 'pending' and 'failed',
we should check that.
2020-02-18 10:10:17 +10:30
Michael Schmoock
3e2607bf2d chore: updates coincurve requirement to v13
Update to v13 as v12 started to raise undefined symbol exceptions agains latest
libsecp256k1.

Note: any version of `pip install coincurve` fails if no libsecp256k1 headers
are installed, should we point this out somewhere/somehow?

Changelog-None
2020-02-18 10:10:17 +10:30
Christian Decker
966ac95098 pytest: Add a test for plugin featurebits 2020-02-11 13:53:31 +10:30
Christian Decker
a13591400a pyln-client: Add facility to register featurebits from plugins
we have 4 venues in which we can add features, 3 of which are unilaterally
controlled (`init`, `node_announcement`, and `invoices`) the
`channel_announcement` is co-signed by both parties, so we can't add
featurebits without additional coordination overhead.

Each location is encoded as a key-value pair in a dict called `featurebits` in
the manifest (omitted if no custom featurebits are set).
2020-02-11 13:53:31 +10:30
Michael Schmoock
5f7cf307e7 pyln-testing: adds helper wait_for_htlcs
Changelog-Added: pyln-testing adds wait_for_htlcs helper to settle HTLCs
2020-02-10 12:14:19 +10:30
Christian Decker
6c19818314 pyln-proto: Add invoice utilities
We are about to disect a couple of invoices for features, so let's add a class
that can encode and decode invoices from bolt11 strings. This is pretty much
the lnaddr.py file created by @rustyrussell with some minor changes. I'm
planning to clean this up further which is why I'm only exporting the
`Invoice` class for now.
2020-02-01 16:50:58 +01:00
Christian Decker
24219b38f9 pyln: Add unit tests from pyln-* to make check-python 2020-02-01 16:50:58 +01:00
Christian Decker
28080b2e69 elementsd: Do not use generate for elementsd >= 0.17.0
The `generate` has been deprecated since 0.16 and has been removed in 0.18.0
so we better use `generatetoaddress` instead, which is already what we do with
`bitcoind`. So we remove the override here.
2020-01-28 20:03:55 +01:00
ZmnSCPxj jxPCSnmZ
67590fc6be lightningd/invoice.c: Add timeout parameter to waitanyinvoice.
Fixes: #3192

Changelog-Added: `waitanyinvoice` now supports a `timeout` parameter, which when set will cause the command to fail when the timeout is reached; can set this to 0 to fail immediately if no new invoice has been paid yet.
2020-01-28 14:07:52 +01:00
Christian Decker
d3f6ebf911 pyln-proto: Added a couple of utilities to manage onions and zbase32 2020-01-28 13:43:44 +10:30
lisa neigut
1763960c4a txprepare: don't crash on empty args
add test for empty args check
2020-01-27 22:59:41 +01:00
darosior
b7699cff9f pyln: actually specify jsonrpc in requests
As per the spec (https://www.jsonrpc.org/specification#request_object)

```
A rpc call is represented by sending a Request object to a Server. The Request object has the following members:

jsonrpc
    A String specifying the version of the JSON-RPC protocol. MUST be exactly "2.0".
```

Changelog-fixed: pyln now includes the "jsonrpc" field to jsonrpc2 requests
2020-01-27 12:38:40 +01:00
ZmnSCPxj
54cc735201 lightningd/peer_control.c: Implement waitblockheight.
This is needed to fully implement handling of blockheight disagreements
between us and payee.
If payee believes the blockheight is higher than ours, then `pay`
should wait for our node to achieve that blockheight.

Changelog-Add: Implement `waitblockheight` to wait for a specific blockheight.
2020-01-21 22:23:21 +01:00
Zoe Faltibà
205f9d814d pyln-client/LightningRpc: fixed logger in super()
* Changelog-None
2020-01-15 09:26:09 +08:00
Vasil Dimov
43cc2ba708 contrib: fix wrong number in comment
Changelog-None
2020-01-06 12:57:59 +01:00
Christian Decker
8cad3ffeac pyln: Work around the socket path length on Linux OSs
Some Linux OSs impose a length limit on the path a Unix socket may have. This
is not an issue in `lightningd` since we `chdir()` into that directory before
opening the socket, however in pyln this became a problem for some tests,
since we use absolute paths in the testing framework. It's also a rather
strange quirk to expose to users.

This patch introduces a `UnixSocket` abstraction that attempts to work around
these limitations by aliasing the directory containing the socket into
`/proc/self/fd` and then connecting using that alias.

It was inspired by Open vSwitch code here https://github.com/openvswitch/ovs/blob/master/python/ovs/socket_util.py

Signed-off-by: Christian Decker <@cdecker>
2020-01-02 16:05:52 +01:00
Christian Decker
de619b1754 pyln: Check to see if we have annotations on funcs before accessing
We were indiscriminately accessing the `__annotations__` which could cause
issues if the function had been wrapped by some functions such as
`functools.partial`. This just checks that the access is safe before doing it.

Suggested-by: jarret <@jarret>
Signed-off-by: Christian Decker <@cdecker>
2020-01-02 16:05:52 +01:00
Christian Decker
8c387932c0 pyln: Bump versions of pyln-client and pyln-testing 2019-12-31 16:43:04 +01:00
lisa neigut
689dd28ddd funding: enable push_msat
it's that time of year (merry xmas!)

enables the ability to push_msat on fundchannel

Changelog-Added: RPC: `fundchannel` and `fundchannel_start` can now accept an optional parameter, `push_msat`, which will gift that amount of satoshis to the peer at channel open.
2019-12-24 12:04:01 -06:00
darosior
051b5bd1bf Update bitcoin-core version
Bitcoin-core v0.18.X is now the more widely used.
2019-12-22 12:42:56 +01:00
Michael Schmoock
c77a085ca4 fix: update pip dependencies version in dockerfile 2019-12-19 15:17:25 +01:00
Michael Schmoock
dc5a1ad664 fix: normalize pip requirements
This will change some `requirements.txt` of pyln-testing in a way that
it does not require different package version i.e. to `tests/requirements.txt`.

The reason for this is that users are not forced to hassle with pyenv
or virtualenv and could just use `--user`.

```bash
pip install --user -r tests/requirements.txt -r contrib/pyln-testing/requirements.txt
```

Changelog-None
2019-12-19 15:17:25 +01:00
Michael Schmoock
43b5abeb22 fix: pip dependency version for psycopg2-binary
Trying to `pip install psycopg2-binary==2.8.3` raised an error for a
long time. Since version `2.8.4` is recent and also seem to work I
suggest updating the requirements so other users dont run into the
following error, where pip tries to use `pg_config` for the BINARY
package:

Collecting psycopg2-binary==2.8.3 (from -r contrib/pyln-testing/requirements.txt (line 6))
  Using cached 91911be018/psycopg2-binary-2.8.3.tar.gz
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-zapr0fhs/psycopg2-binary/setup.py'"'"'; __file__='"'"'/tmp/pip-install-zapr0fhs/psycopg2-binary/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
         cwd: /tmp/pip-install-zapr0fhs/psycopg2-binary/
    Complete output (23 lines):
    running egg_info
    creating pip-egg-info/psycopg2_binary.egg-info
    writing pip-egg-info/psycopg2_binary.egg-info/PKG-INFO
    writing dependency_links to pip-egg-info/psycopg2_binary.egg-info/dependency_links.txt
    writing top-level names to pip-egg-info/psycopg2_binary.egg-info/top_level.txt
    writing manifest file 'pip-egg-info/psycopg2_binary.egg-info/SOURCES.txt'

    Error: pg_config executable not found.

    pg_config is required to build psycopg2 from source.  Please add the directory
    containing pg_config to the $PATH or specify the full executable path with the
    option:

        python setup.py build_ext --pg-config /path/to/pg_config build ...

    or with the pg_config option in 'setup.cfg'.

    If you prefer to avoid building psycopg2 from source, please install the PyPI
    'psycopg2-binary' package instead.

    For further information please check the 'doc/src/install.rst' file (also at
    <http://initd.org/psycopg/docs/install.html>).

    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Changelog-None
2019-12-19 13:26:13 +01:00
Rusty Russell
839909d2cf Protocol: make var_onion, payment_secret and basic_mpp non-EXPERIMENTAL.
Thanks to @t-bast, who made this possible by interop testing with Eclair!

Changelog-Added: Protocol: can now send and receive TLV-style onion messages.
Changelog-Added: Protocol: can now send and receive BOLT11 payment_secrets.
Changelog-Added: Protocol: can now receive basic multi-part payments.
Changelog-Added: RPC: low-level commands sendpay and waitsendpay can now be used to manually send multi-part payments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:16:03 +01:00
Christian Decker
d449423983 pyln: Bump the version of pyln-client 2019-12-12 00:15:23 +01:00
Christian Decker
43dfb4dae7 pytest: Stabilize and parameterize closing_specified_destination
This test was doing a few contortions, so I disentangled it a bit and
parametrized it
2019-12-11 16:18:34 +01:00
Christian Decker
fc93787db9 pytest: Parametrize test_gossip_query_channel_range for elements
We were hardcoding the chainparams->chain_hash which caused the query to
return an empty result. By parametrizing the test we can make it work on
elements.
2019-12-11 16:18:34 +01:00
Christian Decker
15f04d5fff pytest: Give details about which node exited with a return code != 0 2019-12-11 16:18:34 +01:00
Christian Decker
518c43a9c2 pytest: Remember a node's return code so we can check it later 2019-12-11 16:18:34 +01:00
Michael Schmoock
bd369edd10 chore: reduce rpc load in is_channel_active helper 2019-12-07 21:27:28 +01:00
Michael Schmoock
44c23ede04 chore: adds announce_channel parameter to pyln fund_channel 2019-12-07 21:27:28 +01:00
Jarret Dyrbye
d712f732d8 pylightning - allow unicode symbols in the description to be passed as-is to the daemon
addresses issue #2753.

Formatting the JSON with the default parameters will escape the unicode
symbols in a way that c-lightning won't allow, leading to an exception.

Changelog-Fixed: `pylightning` now handles unicode characters in JSON-RPC requests and responses correctly.
2019-11-29 21:38:17 +01:00
Christian Decker
9e59740268 pyln-testing: Do not require a conftest.py to annotate tests
In the c-lightning tests we have `tests/conftest.py` which annotates test
function with the outcome. If we use pyln-testing outside of the c-lightning
tree we cannot rely on that annotation being there, so we assume it passed.
2019-11-29 15:06:39 +01:00
Christian Decker
3c5ed15b2c pyln: Add psycopg2-binary as a dependency
Using the psycopg2-binary package means that the apropriate compiled binary
for the user platform will be shipped alongside the python binaries. Otherwise
the python bindings and the C shims would be shipped which would then require
the postgres development packages as well.

This just makes things easier, since we don't require the build dependencies.
2019-11-29 15:06:39 +01:00
Rusty Russell
aab83e729b lightningd: change config-dir from plugin / wallet / hsm POV into <network> subdir
Changelog-changed: .lightningd plugins and files moved into <network>/ subdir
Changelog-changed: WARNING: If you don't have a config file, you now may need to specify the network to lightning-cli
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Christian Decker
1e89937a0a pytest: Skip mem-leak test when not running in developer mode
We were skipping it when running under valgrind, but not if not in developer,
which is required to have access to `dev-*` methods.
2019-11-22 03:48:54 +00:00
Rusty Russell
3437f7e25d devtools/gossipwith: change timeout to seconds.
I always get this wrong, then wonder why it's dying!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 11:01:20 +01:00
Rusty Russell
709c98f539 logging: remove spaces from subsystem names.
Spaces just make life a little harder for everyone.

(Plus, fix documentation: it's 'jsonrpc' not 'json' subsystem).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
4fc498f901 lightningd: enable io logging on subdaemons iff we're going to print it.
This simplifies our tests, too, since we don't need a magic option to
enable io logging in subdaemons.

Note that test_bad_onion still takes too long, due to a separate minor
bug, so that's marked and left dev-only for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
ef7a820ab1 log: make formatting more consistent.
1. Printed form is always "[<nodeid>-]<prefix>: <string>"
2. "jcon fd %i" becomes "jsonrpc #%i".
3. "jsonrpc" log is only used once, and is removed.
4. "database" log prefix is use for db accesses.
5. "lightningd(%i)" becomes simply "lightningd" without the pid.
6. The "lightningd_" prefix is stripped from subd log prefixes, and pid removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Logging: formatting made uniform: [NODEID-]SUBSYSTEM: MESSAGE
Changelog-removed: `lightning_` prefixes removed from subdaemon names, including in listpeers `owner` field.
2019-11-18 04:50:22 +00:00
Rusty Russell
eed654f684 connectd, gossipd: use per-peer logging.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
e433d4ddc1 lightningd: have logging include an optional node_id for each entry.
A log can have a default node_id, which can be overridden on a per-entry
basis.  This changes the format of logging, so some tests need rework.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
arowser
39fa486cb2 remove unnecessary software package from arm Dockerfile 2019-11-14 10:20:28 +01:00
lisa neigut
6f215a70e5 contrib: fixup documentation of l1/2-log 2019-11-13 03:31:20 +00:00
Christian Decker
9378be72a9 pyln: Allow users to override the LightningNode class
Quite a few of the things in the LightningNode class are tailored to their use
in the c-lightning tests, so I decided to split those customizations out into
a sub-class, and adding one more fixture that just serves the class. This
allows us to override the LightningNode implementation in our own tests, while
still having sane defaults for other users.
2019-11-12 21:23:55 +01:00
Christian Decker
bc9b1c405e pyln: Move RPC and daemon instantiation into the LightningNode
We were relying heavily on NodeFactory to do some magic before instantiating
the Node with rpc and daemon initialized, that meant that we'd have to replace
all 3 classes when customizing the node to our needs. Moving that
initialization into the node itself means that the LightningNode class now can
be swapped out and customized, without having to wire everything else through.
2019-11-12 21:23:55 +01:00
Christian Decker
53a2789547 pyln: Default to DEVELOPER=0 when running outside of tree
`DEVELOPER=1` assumes that the binary has been compiled with developer set to
true, which might not be the case for plugin developers. Setting this to 0 by
default has no effect in c-lightning since we always at least set it in
`config.vars` but may prevent some issues outside.
2019-11-12 21:23:55 +01:00
Christian Decker
b7e7e535c8 pytest: Migrate env to pyln-testing 2019-11-12 21:23:55 +01:00
Christian Decker
a2a3d33802 pyln-testing: Copy basic support infrastructure into pyln.testing
We'll rewrite the tests to use this infrastructure in the next commit.

Changelog-Added: The new `pyln-testing` package now contains the testing infrastructure so it can be reused to test against c-lighting in external projects
2019-11-12 21:23:55 +01:00