Commit Graph

7106 Commits

Author SHA1 Message Date
Christian Decker
518c43a9c2 pytest: Remember a node's return code so we can check it later 2019-12-11 16:18:34 +01:00
Vasil Dimov
cd30f76539 build: fix ./configure options' descriptions
The description of

--enable/disable-experimental-features was a bogus "Developer mode, good
for testing" and

--enable/disable-valgrind was "Valgrind binary to use for tests" which
gave the false impression that it should be set to something like
/usr/local/bin/valgrind whereas it is a boolean option.
2019-12-11 00:50:20 +01:00
darosior
fcbd11f0c5 plugins/libplugin: hook support
Changelog-Added: plugins: libplugin now supports writing plugins which register to hooks
2019-12-09 16:18:28 +01:00
darosior
3371f0cf78 plugins/libplugin: notifications support
Changelog-Added: plugins: libplugin now supports writing plugin which registers to notifications
2019-12-09 16:18:28 +01:00
Rusty Russell
f7ebbb2ec5 common: make sphinx code ignorant of payload format.
Now "raw_payload" is always the complete string (including realm or length
bytes at the front).

This has several effects:
1. We can receive an decrypt an onion which is grossly malformed.
2. We can still hand this to the htlc_accepted hook.
3. We then fail it unless the htlc_accepted accepts it manually.
4. The createonion API now takes the raw payload, and does not know
   anything about "style".

The only caveat is that the sphinx code needs to know the payload
length: we have a call for that, which simply tells it to copy the
entire onion (and treat us as the final node) if it's invalid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-09 14:33:31 +01:00
Rusty Russell
bb538a1862 common: don't crash on bad sphinx payload.
It's cleanest to eliminate the SPHINX_INVALID_PAYLOAD altogether.

lightning_channeld: FATAL SIGNAL (version v0.7.3-242-gb1583bb-modded)
0x55a8169eed08 send_backtrace
	common/daemon.c:41
0x55a8169fc3eb status_failed
	common/status.c:206
0x55a8169fc657 status_backtrace_exit
	common/subdaemon.c:25
0x55a8169eedbb crashdump
	common/daemon.c:57
0x7f0eaff8446f ???
	???:0
0x7f0eaff843eb ???
	???:0
0x7f0eaff63898 ???
	???:0
0x55a8169fb29f route_step_decode
	common/sphinx.c:759
0x55a8169fb60a process_onionpacket
	common/sphinx.c:834
0x55a8169d9b34 get_shared_secret
	channeld/channeld.c:605
0x55a8169d9d35 handle_peer_add_htlc
	channeld/channeld.c:649
0x55a8169dd88d peer_in
	channeld/channeld.c:1838
0x55a8169e11a8 main
	channeld/channeld.c:3233
0x7f0eaff651e2 ???

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-09 14:33:31 +01:00
Rusty Russell
f8d8348c9f doc: add 'id' to rpc_command
It's required, so at least hint about its presence!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-08 15:26:28 +01:00
Michael Schmoock
bd369edd10 chore: reduce rpc load in is_channel_active helper 2019-12-07 21:27:28 +01:00
Michael Schmoock
44c23ede04 chore: adds announce_channel parameter to pyln fund_channel 2019-12-07 21:27:28 +01:00
Rusty Russell
30c8db148d lightningd: make sure rpc_command replacement is well-formed.
In particular:
1. It must redirect to an existing command.
2. It must contain method, params and id.

And update the docs to show the id, which is vital.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-07 21:26:31 +01:00
Rusty Russell
e572b487d3 pytest: demonstrate failing rpc_command hook rewrite to plugin.
lightningd: lightningd/plugin.c:648: plugin_rpcmethod_dispatch: Assertion `idtok != NULL' failed.
lightningd: FATAL SIGNAL 6 (version v0.7.3-234-gf3aeb51-modded)
0x557652363eb1 send_backtrace
	common/daemon.c:41
0x557652363f5b crashdump
	common/daemon.c:54
0x7f5b065a346f ???
	???:0
0x7f5b065a33eb ???
	???:0
0x7f5b06582898 ???
	???:0
0x7f5b06582768 ???
	???:0
0x7f5b06594005 ???
	???:0
0x557652355901 plugin_rpcmethod_dispatch
	lightningd/plugin.c:648
0x557652331836 command_exec
	lightningd/jsonrpc.c:588
0x557652331bad rpc_command_hook_callback
	lightningd/jsonrpc.c:655
0x557652358745 plugin_hook_callback
	lightningd/plugin_hook.c:90
0x55765235486a plugin_response_handle
	lightningd/plugin.c:258
0x557652354a16 plugin_read_json_one
	lightningd/plugin.c:356
0x557652354b6d plugin_read_json
	lightningd/plugin.c:388
0x5576523b5e6c next_plan
	ccan/ccan/io/io.c:59
0x5576523b6a51 do_plan
	ccan/ccan/io/io.c:407
0x5576523b6a93 io_ready
	ccan/ccan/io/io.c:417
0x5576523b8d2b io_loop
	ccan/ccan/io/poll.c:445
0x55765232dc6f io_loop_with_timers
	lightningd/io_loop_with_timers.c:24
0x557652334a6e main
	lightningd/lightningd.c:848
0x7f5b065841e2 ???
	???:0
0x5576523193ed ???
	???:0
0xffffffffffffffff ???
	???:0
Log dumped in crash.log.20191206093336

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-07 21:26:31 +01:00
Rusty Russell
f6ed7f2e89 plugin: handle corner case where rpc_command is to stop the plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-06 16:52:16 +01:00
Rusty Russell
d8fc5332c3 pytest: test rpc_command hook when it handles command to stop itself.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-06 16:52:16 +01:00
lisa neigut
a30ee2b7cd fundchannel: attempt to connect before funding channel
Attempts to connect to peer before funding the channel
2019-12-04 15:32:31 -06:00
lisa neigut
f6ff5e5b19 connectd: make failure message more descriptive 2019-12-04 15:32:31 -06:00
darosior
ff412a6c9a plugins/fundchannel: try to connect to peer before funding
Changelog-Added: RPC: The 'fundchannel' command now tries to connect to the peer before funding the channel, no need to 'connect' before 'fundchannel' anymore !
2019-12-04 15:32:31 -06:00
Rusty Russell
4683d096dd doc: feerate is not obvious, refer to fundchannel documentation:
It has a whole *two paragraphs* on it:

*feerate* is an optional feerate used for the opening transaction and as
initial feerate for commitment and HTLC transactions. It can be one of
the strings *urgent* (aim for next block), *normal* (next 4 blocks or
so) or *slow* (next 100 blocks or so) to use lightningd’s internal
estimates: *normal* is the default.

Otherwise, *feerate* is a number, with an optional suffix: *perkw* means
the number is interpreted as satoshi-per-kilosipa (weight), and *perkb*
means it is interpreted bitcoind-style as satoshi-per-kilobyte. Omitting
the suffix is equivalent to *perkb*.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-04 08:35:50 -06:00
Saibato
2c16b410ab Add basic tests
Signed-off-by: Saibato <saibato.naga@pm.me>
2019-12-03 23:35:18 +01:00
Saibato
b18510a7e3 Reflect the changes in documentation
Signed-off-by: Saibato <saibato.naga@pm.me>
2019-12-03 23:35:18 +01:00
Saibato
f6006f43a9 Init commit to be able to create a tor static service on the fly.
We  want to have a static Tor service created from a blob bound to
our node on cmdline

Changelog-added: persistent Tor address support
Changelog-added: allow the Tor inbound service port differ from 9735

Signed-off-by: Saibato <saibato.naga@pm.me>

Add base64 encode/decode to common

We need this to encode the blob for the tor service

Signed-off-by: Saibato <saibato.naga@pm.me>
2019-12-03 23:35:18 +01:00
fiatjaf
99ff86f6fe update docs with payment_preimage.
Changelog-Added: JSON-RPC: `listinvoices` now displays the payment preimage if the invoice was paid.
2019-12-03 01:16:07 +00:00
fiatjaf
c0368a717a also return preimage on listinvoices. 2019-12-03 01:16:07 +00:00
fiatjaf
770789684c jsonrpc: Add json_add_preimage helper 2019-12-03 01:16:07 +00:00
Christian Decker
db92c2ac5e tlv: Remove unused TLV deserialization function 2019-12-03 00:37:15 +00:00
Christian Decker
626675c83c tlv: Migrate tlv serialization to typesafe function 2019-12-03 00:37:15 +00:00
Christian Decker
e12b5c3824 tlv: Add a typesafe serialization function for tlv namespaces
This is the counterpart to the typesafe deserialization function implemented
in an earlier commit.
2019-12-03 00:37:15 +00:00
Christian Decker
5a78671d9f wire: Remove unused fromwire_tlvs
We are now using the typesafe variant everywhere.
2019-12-03 00:37:15 +00:00
Christian Decker
838001af4c tlv: Migrate run-tlvstream to typesafe helpers 2019-12-03 00:37:15 +00:00
Christian Decker
69c17d2d31 wire: Let the TLV _is_valid function actually return validity
I got this one wrong myself, since the function name implied a boolean
result. So I changed it to take the optional err_index as argument.
2019-12-03 00:37:15 +00:00
darosior
d9aeeb4d14 pytest: skip test_sendonion if not DEVELOPER 2019-12-02 14:41:13 +01:00
darosior
4661dfa590 tests: update requirements
cheroot release(/"changes"?) notes:
    #218 via PR #219: Fix HTTP parser to return 400 on invalid major-only HTTP version in Request-Line.
    #198 via 9f7affe: Fix race condition when toggling stats counting in the middle of request processing.
    Improve post Python 3.9 compatibility checks.
    Fix support of abstract namespace sockets.
    #222 via 621f4ee: Fix socket.SO_PEERCRED constant fallback value under PowerPC
    Revisit PR #85 under PR #221. Now backports.functools_lru_cache is only required on Python 3.2 and earlier.
    CherryPy #1206 via PR #204: Fix race condition in threadpool shrink code.
    PR #224: Refactored “open URL” behavior in webtest to rely on retry_call. Callers can no longer pass raise_subcls or ssl_context positionally, but must pass them as keyword arguments.
    #231 via PR #232: Remove custom setup.cfg parser handling, allowing the project (including sdist) to build/run on setuptools 41.4. Now building cheroot requires setuptools 30.3 or later (for declarative config support) and preferably 34.4 or later (as indicated in pyproject.toml).
    Workers are now request-based, addressing the long-standing issue with keep-alive connections (#91 via PR #199).
    Deprecated use of negative timeouts as alias for infinite timeouts in ThreadPool.stop.
    CherryPy #1662 via PR #74: For OPTION requests, bypass URI as path if it does not appear absolute.
    CherryPy #1818: Restore support for None default argument to WebCase.getPage().

https://github.com/cherrypy/cheroot/blob/master/CHANGES.rst

flaky changes:
    Bugfixes - Reraise KeyboardInterrupt when running tests under pytest.

https://github.com/box/flaky/blob/v3.6.1/HISTORY.rst#361-2019-08-06

python-bitcoinlib:
    New RPC `generatetoaddress(self,numblocks,addr)`.
    Fixed Python 2.7 incompatibility.
    Various OpenSSL fixes, including a memory leak.

https://github.com/petertodd/python-bitcoinlib/blob/python-bitcoinlib-v0.10.2/release-notes.md#v0102

pytest release notes:
    A lot of misc fixes, see https://docs.pytest.org/en/latest/changelog.html.
2019-12-02 14:41:13 +01:00
Rusty Russell
7f36a4e3dd lightningd: fix reference to out-of-scope var.
cppcheck found this:

	[lightningd/options.c:1137] -> [lightningd/options.c:1120] -> [lightningd/options.c:1193]: (error) Using pointer to local variable 'buf' that is out of scope.

Indeed, answer can point into buf, which is no longer in scope at the end.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-02 09:44:39 +01:00
Christian Decker
0b61781746 json-rpc: Only show the amount_msat field if we know it in payments
If we initiated the payment using an externally generated onion we don't know
what the final hop gets, or even who it is, so we don't display the amount in
these cases. I chose to show `null` instead in order not to break dependees
that rely on the value being there.
2019-12-01 15:40:47 +01:00
Christian Decker
82255e2401 json-rpc: Add helper for an array of secrets
Suggested-by: Rusty Russell <@rustyrussell>
2019-12-01 15:40:47 +01:00
Christian Decker
21b5b59c8c json-rpc: Rename onion payload type to style
Suggested-by: Rusty Russell <@rustyrussell>
2019-12-01 15:40:47 +01:00
Christian Decker
55d8dcc907 param: Encapsulate hops parsing in a param_hops_array helper
Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <@cdecker>
2019-12-01 15:40:47 +01:00
Christian Decker
f569b52681 sphinx: Make the sphinx_hop struct public
We're about to create a param helper for sphinx hops and this struct seems
like the correct place to store the result.
2019-12-01 15:40:47 +01:00
Christian Decker
2080416a89 doc: Add manpages for createonion and sendonion
Changelog-Added: Added `createonion` and `sendonion` JSON-RPC methods allowing the implementation of custom protocol extensions that are not directly implemented in c-lightning itself.
2019-12-01 15:40:47 +01:00
Christian Decker
705e189f0a pay: Allow payments initiated with sendonion to be retried 2019-12-01 15:40:47 +01:00
Christian Decker
16b6c31010 json-rpc: Add the error onion if we stored it in the DB
If we can't decode the onion, because the onion got corrupted or we used
`sendonion` without specifying the `shared_secrets` used, the best we can do
is tell the caller instead.
2019-12-01 15:40:47 +01:00
Christian Decker
1cf3c12a81 cleanup: The failchannel is not tal-allocated when first assigned
This makes the copy on write redundant.
2019-12-01 15:40:47 +01:00
Christian Decker
27547ce8d4 pay: Allow sendonion callers to provide shared_secrets
This means that c-lightning can now internally decrypt an eventual error
message, and not force the caller to implement the decryption. The main
difficulty was that we now have a new state (channels and nodes not specified,
while shared_secrets are specified) which needed to be handled.
2019-12-01 15:40:47 +01:00
Christian Decker
7e41d6c9c4 pay: Make erring_channel optional in struct routing_failure
Same rationale as the previous commit: we may not have the channels in the
path so we don't try to infer the failing channel from the error.
2019-12-01 15:40:47 +01:00
Christian Decker
88961aa4b0 pay: Make erring_node optional in struct routing_failure
When using `sendonion` with `shared_secrets` we may be able to decode the
onioned error message but we cannot infer which node reported the failure
since we don't know which nodes where involved.
2019-12-01 15:40:47 +01:00
Christian Decker
43fb2a6ed0 json: Add helper to extract a secret from JSON 2019-12-01 15:40:47 +01:00
Christian Decker
b8ce175fd2 pay: Handle payment failures resulting from sendonion correctly
We are breaking with a couple of assumptions, namely that we have the
`path_secrets` to decode the error onion. If this happens we just want it to
error out.
2019-12-01 15:40:47 +01:00
Christian Decker
cb912c9ae8 pytest: Add a test for the sendonion method 2019-12-01 15:40:47 +01:00
Christian Decker
42277a7ad4 json-rpc: Add the sendonion RPC command 2019-12-01 15:40:47 +01:00
Christian Decker
57f13af1ec pay: Add param_route_hop helper to parse struct route_hop from JSON 2019-12-01 15:40:47 +01:00
Christian Decker
41221b6ecb pay: Make wallet_payment->destination optional
If we use `sendonion` we don't actually know the destination, so we
make the destination a pointer which is NULL if we don't know.
2019-12-01 15:40:47 +01:00