Commit Graph

5820 Commits

Author SHA1 Message Date
Christian Decker
f7bfe166a8 startup: Reorder HTLC wiring on startup after the topology init
Since the hook needs to pass information about the current blockheight to the
plugin we need to first initialize the topology.
2019-06-04 00:27:15 +00:00
Christian Decker
adb984dd45 hooks: Add the raw payload to the htlc_accepted call
Since we might soon be changing the payload it is a good idea to not just
expose the v0 payload, but also the raw payload for the plugin to
interpret. This might also include payloads that `lightningd` itself cannot
understand, but the plugin might.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Suggested-by: Corné Plooy <@bitonic-cjp>
2019-06-04 00:27:15 +00:00
Christian Decker
1561ffaea0 hooks: Add cltv_expiry_delta to the htlc_accepted hook
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Suggested-by: Corné Plooy <@bitonic-cjp>
2019-06-04 00:27:15 +00:00
Christian Decker
9fd8be6463 pytest: Add tests for htlc_accepted_hook
Two tests: one for failures and one for in-path resolution.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-04 00:27:15 +00:00
Christian Decker
2b81e02a2e plugin: Parse response for htlc_accepted hook
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-04 00:27:15 +00:00
Christian Decker
bf53821f1a plugin: Populate the request for the htlc_accepted hook
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-04 00:27:15 +00:00
Christian Decker
b54577041a pytest: Add a simple test for the hooks
This uses the `htlc_accepted` hook to delay payment acceptance.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-04 00:27:15 +00:00
Christian Decker
7499f7ddd4 plugin: Add the htlc_accepted hook
This is a rather simple hook that allows a plugin to take control over
HTLCs that were accepted, but weren't resolved as part of an invoice
or forwarded to the next hop yet.

The goal is to allow plugins to terminate a route early, perform
intermediate checks before the payment is accepted (check inventory or
service delivery before accepting in order to avoid a refund for
example) or handle an onion differently if it has a different
realm (cross-chain atomic swaps).

This doesn't implement serializing the payload or deserializing it,
instead just passes the full context along. The details for
serializing and deserializing will be implemented in a future commit.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-04 00:27:15 +00:00
Christian Decker
9a4cf7dda7 pytest: Mark test_option_upfront_shutdown_script as flaky
Signed-off-by: Christian Decker <@cdecker>
2019-06-04 00:27:15 +00:00
Christian Decker
80b3d8aef3 json: Add some more methods to add integers of various size to json
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-06-04 00:27:15 +00:00
Rusty Russell
763697eb4c gossipd: fix gossip_store calling delete.
Now we handle node_announcements properly, we have a failure case where we
try to move them when a channel is deleted while loading the store.

We're going to remove this soon, in favor of in-place delete, so
workaround this for now to avoid an assert() when we try to write to
the store while loading.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 11:04:25 -07:00
Rusty Russell
21fe518513 gossip_store: fix 'bad node_announcement' by allowing node_announcement on un-updated channel.
When we first receive a channel_update, we write both the
channel_announcement and that channel_update to the store: we need
that first update so we can set the channel_announcement timestamp.

However, the channel_update can be replaced later.  This means we can
have a channel_announcement, a node_update which relies on it, then
the channel_update later.

So move the "this applies to a pending announcement" check lower, where
gossip_store can use it too.  Has a nice side-effect of avoiding
one lookup of the node id.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 11:04:25 -07:00
Rusty Russell
048a650a6b pytest: more comprehensive tests for test_gossip_store_compact.
First, we should have a channel_update so we actually do some compaction!
(Reported-by @SimonVrouwe).  But we should also handle the cases where:

1. A channel_announcement is *not* directly followed by a
   channel_update (happens when the channel_update is replaced).
2. A node_announcement predates a channel_update for the peer
   (again, can happen once a channel_update is replaced).
3. A local/private channel_creation is not directly followed by an
   update.

In addition, we might as well check that we can *load* such a store,
before compaction.

This checks the corner cases which occur in real gossip stores.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 11:04:25 -07:00
Rusty Russell
1147e65602 pytest: make test_gossip_notices_close more reliable.
It's possible that it hasn't got the node_announcement messages;
it will still list the nodes, however (the channel_announcement tells
it the nodes exist).  Check for the alias field instead.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 11:04:25 -07:00
Rusty Russell
a40fe40ad2 pytest: catch more gossip-related errors.
Basically, any "Bad" message from gossipd is something we should look
at.  This covers failures loading the gossip_store, too!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 11:04:25 -07:00
Michael Schmoock
45645e3a85 test: fix flakyness test_no_fee_estimate 2019-06-03 19:30:19 +02:00
Michael Schmoock
107381aca2 test: fix flakyness of funding_reorg_private openchannel_hook 2019-06-03 19:30:19 +02:00
Rusty Russell
7961a815aa travis: add compilation to SOURCE_CHECK_ONLY.
Our "-O3" CI check wasn't doing what I thought.  But building with -flto
is hard with older gccs, so remove that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell
7092d215fa configure: detect modern gcc, and only suppress necessary errors.
Drive-by fix of CONFIGURATOR_CC setting when CC is overridden on cmdline
(not via env var).

Note that clang defines __GNUC__ (to 4) :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell
ed08ca6df3 configure: handle --reconfigure better.
`./configure CC=newcc --reconfigure` didn't set CC, because reconfigure
simply replaced all values: only make it replace undefined values.

Also, it didn't change COPTFLAGS or CWARNFLAGS, even if they were previously
the defaults (eg. --reconfigure --enable-developer).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell
5b7776cf13 wire/test: remove unused padding code.
Turns out the peer part of the spec no longer uses padding (it's used only
in the onion), and GCC-9 with -O3 warns we're padding NULL to memcpy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell
c233fc5063 gossipd: fix spurious unused error with gcc-9 -O3.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell
9820d0664b connectd: handle IPv6 correctly.
Untested, but the current code is clearly wrong (caught by gcc-4.8 -O3).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell
c091a4ee40 gossipd: fix spurious gcc warning.
It turns out that we don't look at type when we return 0, but gcc isn't
quite smart enough for that.  Initializing to -1 is good practice anyway
for the failure path.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
Rusty Russell
fae593c448 lightningd: fix uninitialized variable in DEVELOPER path
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
William Casarin
6f635b46fd gossipstore: fix uninitialized input fd
Initialize infd to STDIN_FILENO if the input file argument is missing.

Caught with gcc version: 7.4.0

devtools/create-gossipstore.c: In function ‘main’:
devtools/create-gossipstore.c:130:9: error: ‘infd’ may be used
uninitialized in this function [-Werror=maybe-uninitialized]
  while (read_all(infd, &be_inlen, sizeof(be_inlen))) {

Suggested-by: @ZmnSCPxj <https://github.com/ElementsProject/lightning/pull/2674#issuecomment-495617253>
Signed-off-by: William Casarin <jb55@jb55.com>
2019-06-03 00:07:11 +00:00
William Casarin
3f035cb3cc gossipd: fix uninitialized free on short_route in goto path
Fix a path where tal_free is called on an uninitialized variable

If the first `goto bad_total` executes, then that path has
uninitialized `short_route` but bad_total passes through to `out`
whose first call is tal_free(short_route).

This was noticed by a maybe-uninitialized heuristic on gcc 7.4.0:

gossipd/routing.c: In function ‘find_shorter_route’:
gossipd/routing.c:1096:2: error: ‘short_route’ may be used
uninitialized in this function [-Werror=maybe-uninitialized]
  tal_free(short_route);

Reported-by: @ZmnSCPxj <https://github.com/ElementsProject/lightning/pull/2674#issuecomment-495617253>
Signed-off-by: William Casarin <jb55@jb55.com>
2019-06-03 00:07:11 +00:00
darosior
4640d18ef0 Plugins: allow to specify plugin option type in libplugin.c, then specify it to autoclean options 2019-06-03 00:06:12 +00:00
darosior
3775a32659 Pylightning: allow to specify the option type at its creation 2019-06-03 00:06:12 +00:00
darosior
a6753fd812 Plugins: accept options of different types
A new struct containing the plugin options value as different types is created and appended to the plugin_option structure
2019-06-03 00:06:12 +00:00
darosior
fba050e891 CHANGELOG: Add some informations about the new category field in commands 2019-06-03 00:02:25 +00:00
darosior
78326a8d90 Pylightning: Allow a plugin to set the category of the RPC commands it creates 2019-06-03 00:02:25 +00:00
darosior
b1bbafb19c cli: Sort human_readable output by categories
A new field is available in the command json object : 'category' (corresponding to whether "bitcoin", "channel", "network", "payment", "plugin", "utility", "developer"). We use it to printf commands ordered by categories.

credits @rustyrussel
2019-06-03 00:02:25 +00:00
darosior
9d34121ec5 libplugin.c: Handle command categories then set them in pay and autoclean 2019-06-03 00:02:25 +00:00
darosior
5b0bf0ba1f jsonrpc: allow to set the command category plugin side 2019-06-03 00:02:25 +00:00
darosior
323adb467a jsonrpc: Add a category field to commands.
A new string field is added to the command structure and is specified at the creation of each native command, and in the JSON created by 'json_add_help_command()'.
2019-06-03 00:02:25 +00:00
Rusty Russell
0c189fe3a7 devtools/decode-iolog: tool to decode hexstrings from io logging.
Slow, but useful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-31 18:36:38 +02:00
Rusty Russell
695bec531c plugin: fix printing of bad plugin responses.
Before:
	Plugin for invoice_payment returned non-result response 

	"subscriptions": [], "hooks": ["invoice_payment"]}}

	�V

After:
	Plugin for invoice_payment returned non-result response {"jsonrpc": "2.0", "id": 6, "error": "Error while processing invoice_payment: ValueError(\"invalid literal for int() with base 10: '5.0'\")"}

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-31 18:36:38 +02:00
Rusty Russell
8f9c48254b plugins: do I/O logging.
I was trying to trace a problem with a plugin, and needed this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-31 18:36:38 +02:00
ZmnSCPxj
48df6c8566 lightningd/io_loop_with_timers.c: Move mainloop to its own source file, have chaintopology use it.
Fixes: #2687
2019-05-31 17:57:10 +02:00
Christian Decker
883d4b4e6e pytest: Add a test that reproduces #2687
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-05-31 17:57:10 +02:00
ZmnSCPxj
3466261b20 lightningd/channel.c: Transfer peer to connectd only if connectd alive.
Fixes: #2677
2019-05-31 15:01:58 +02:00
ZmnSCPxj
37440e9447 lightningd/subd.c: Return NULL from subd_shutdown.
And set pointers to shut down daemons as NULL in lightningd.
2019-05-31 15:01:58 +02:00
trueptolemy
11412435ce Channeld: init channel with remote announcement info when restart
1. Add remote_ann_node_sigs and remote_bitcoin_sigs fields in channel_init message;
2. Master add announcement signatures into channel_init message, and send this message to Channeld.
Channeld will initial the channel with this signatures when it reenables the channel.
2019-05-29 11:46:33 +02:00
trueptolemy
d93f61407a Channeld: Add new wire type:channel_got_announcement
Channeld sends announcement signatures to Master by this message.
When Channeld receive a new channel announcement msg, (After channel locking)it will sends announcement signatures to Master by this message.
2019-05-29 11:46:33 +02:00
trueptolemy
a645fbdecd run-wallet: Add the check of case that channel received ann_sigs 2019-05-29 11:46:33 +02:00
trueptolemy
b41d529b28 DB: Store the remote channel announcement signatures into DB
1. Add the fields of remote channel announcement signatures into TABLE channels
2. Add the related function
2019-05-29 11:46:33 +02:00
Rene Pickhardt
22600faed7 manually added a list-forward manpage because I have no idea how a2x works 2019-05-27 17:35:06 +02:00
Simon Vrouwe
52c84b2387 pytest: add test_funding_reorg_remote_lags
Nodes may disagree about short_channel_id before channel
announcement.
2019-05-27 13:29:32 +02:00
Simon Vrouwe
5b0c174251 pytest: add test_funding_reorg_private, reorg changes funding height
Reorg changes short_channel_id after lockin of private channel, while
one node restarts.

test that:
- peer->depth_togo in billboard decrements
- reorg and scid change is detected by running node and restarting node
- both `old` and `new` scids are in rtable

Also added a comment to test_blockchaintrack to clarify.
2019-05-27 13:29:32 +02:00