Commit Graph

2224 Commits

Author SHA1 Message Date
darosior
603e2f0ca9 plugins: split manifest_cb and plugin_config
This reduces error details for getmanifest response,
but avoids too much code duplication in the next commit.
2019-09-30 00:20:16 +00:00
darosior
6694dda8a3 plugins: return also on register error in add_plugin_dir() 2019-09-30 00:20:16 +00:00
darosior
fcc358c9bc plugins: return plugin in plugin_register, make conn initializers publics 2019-09-30 00:20:16 +00:00
Saibato
0f9ba53581 Add enable-autotor-v2 config variable
With enable-autotor-v2 defined in cmdline the default behavior to create
v3 onions with the tor service call, is set to v2 onions.

Signed-off-by: Saibato <saibato.naga@pm.me>
2019-09-28 00:31:02 +02:00
Rusty Russell
d24c850899 gossipd: restore a flag for fast pruning
I was seeing some accidental pruning under load / Travis, and in
particular we stopped accepting channel_updates because they were 103
seconds old.  But making it too long makes the prune test untenable,
so restore a separate flag that this test can use.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-27 00:01:34 +00:00
Christian Decker
07e6f43e1d master: Set our own rlimit for number of fds
We should never open more than 1024 file descriptors anyway, and under some
situations, namely running as root or in docker, would give us huge
allowances. This then results in a huge, unneeded, cleanup for subprocesses,
which we use a lot.

Fixes #2977
2019-09-25 22:49:41 +00:00
trueptolemy
12da10cd15 sendpay: Compatible with old parameters description 2019-09-25 12:29:53 +02:00
Rusty Russell
45381bba33 bitcoin: implement is_scid_depth_announceable helper.
The math is a bit tricky, so encapsulate it.

Includes the extra 'e' in 'announcable' as noted by @cdecker :)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-25 04:01:56 +00:00
Rusty Russell
2a74d53841 Move gossip_constants.h into common/
Turns out we weren't checking the BOLT comments before, so they
needed an overhaul.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-25 04:01:56 +00:00
Rusty Russell
6f9c5f2936 gossipd: get fed the blockheight from lightningd when we know it.
This will let gossipd be more intelligent about gossiping before we're
synced, and also it might know how far behind we are.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-25 04:01:56 +00:00
Rusty Russell
6955bf6b86 lightningd: initialize topology synclist earlier.
We were doing it in setup_topology, but that's too late if gossipd
wants to register earlier.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-25 04:01:56 +00:00
trueptolemy
d8dce6e61f cleanup: Use u32 as the type of max_hops in gossipd 2019-09-24 16:01:24 +02:00
Christian Decker
5953a5051c cli: Add command line option to specify the wallet location
Will be demuxed into starting the selected DB backend in one of the next
commits. Defaults to the old database location.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Rusty Russell
27790832a5 gossipd: gossip_queries_ex is not longer experimental.
The master spec has some typos which make it not parse, so I created
a PR and generated the CSV from that:

https://github.com/lightningnetwork/lightning-rfc/pull/673

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell
147eaced2e developer: consolidiate gossip timing options into one --dev-fast-gossip.
It's generally clearer to have simple hardcoded numbers with an
#if DEVELOPER around it, than apparent variables which aren't, really.

Interestingly, our pruning test was always kinda broken: we have to pass
two cycles, since l2 will refresh the channel once to avoid pruning.

Do the more obvious thing, and cut the network in half and check that
l1 and l3 time out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell
178baeba6c gossipd: get gossip_min_interval from lightningd.
Default is 5 x gossip interval == 5 minutes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell
9578fb4035 lightningd: fix crash in cancel_channel
This path is not atomic, so we can't assume channel stays around.  It
could be the peer closes, it could be we get a parallel
fund_channel_cancel.  test_funding_cancel_race caused this crash:

    FATAL SIGNAL 6 (version 5f0a18e)
    backtrace: common/daemon.c:45 (send_backtrace) 0x55c7c373a429
    backtrace: common/daemon.c:53 (crashdump) 0x55c7c373a479
    backtrace: (null):0 ((null)) 0x7f88ee6ddf5f
    backtrace: (null):0 ((null)) 0x7f88ee6dded7
    backtrace: (null):0 ((null)) 0x7f88ee6bf534
    backtrace: ccan/ccan/tal/tal.c:93 (call_error) 0x55c7c379427c
    backtrace: ccan/ccan/tal/tal.c:165 (check_bounds) 0x55c7c3794444
    backtrace: ccan/ccan/tal/tal.c:174 (to_tal_hdr) 0x55c7c3794483
    backtrace: ccan/ccan/tal/tal.c:186 (to_tal_hdr_or_null) 0x55c7c3794504
    backtrace: ccan/ccan/tal/tal.c:421 (tal_alloc_) 0x55c7c3794c10
    backtrace: ccan/ccan/tal/tal.c:466 (tal_alloc_arr_) 0x55c7c3794ded
    backtrace: ccan/ccan/tal/str/str.c:91 (tal_vfmt_) 0x55c7c3793560
    backtrace: common/wire_error.c:22 (towire_errorfmtv) 0x55c7c3747f7b
    backtrace: common/wire_error.c:39 (towire_errorfmt) 0x55c7c37480a1
    backtrace: lightningd/channel_control.c:635 (process_check_funding_broadcast) 0x55c7c37015bb
    backtrace: lightningd/bitcoind.c:558 (process_gettxout) 0x55c7c36f8e75
    backtrace: lightningd/bitcoind.c:227 (bcli_finished) 0x55c7c36f8090
    backtrace: ccan/ccan/io/poll.c:244 (destroy_conn) 0x55c7c37869fe
    backtrace: ccan/ccan/io/poll.c:250 (destroy_conn_close_fd) 0x55c7c3786a1e
    backtrace: ccan/ccan/tal/tal.c:235 (notify) 0x55c7c3794629
    backtrace: ccan/ccan/tal/tal.c:397 (del_tree) 0x55c7c3794b18
    backtrace: ccan/ccan/tal/tal.c:481 (tal_free) 0x55c7c3794ea4
    backtrace: ccan/ccan/io/io.c:450 (io_close) 0x55c7c378521d
    backtrace: ccan/ccan/io/poll.c:449 (io_loop) 0x55c7c3787139
    backtrace: lightningd/io_loop_with_timers.c:24 (io_loop_with_timers) 0x55c7c370b26d
    backtrace: lightningd/lightningd.c:837 (main) 0x55c7c3711661
    backtrace: (null):0 ((null)) 0x7f88ee6c0b6a
    backtrace: (null):0 ((null)) 0x55c7c36f70b9
    backtrace: (null):0 ((null)) 0xffffffffffffffff

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-19 18:41:12 +08:00
Rusty Russell
794bebcc81 lightningd: add find_channel_by_id() helper, use it.
This is overkill for the moment, but we'll need it in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-19 18:41:12 +08:00
trueptolemy
5361a5d059 JSON-API: getroute now also support exclude nodes 2019-09-16 12:22:06 +08:00
trueptolemy
090a43fd3d gossip: Add the struct exclude_entry and enum exclude_entry_type 2019-09-16 12:22:06 +08:00
Rusty Russell
a988ded3fa lightningd: fix case where config file overrides lightning-dir.
The simplest case is to explicitly load it when we see it's been
set.

This involves neatening the default config setup, to remove it from
opt_parse_from_config() and into the caller.  It also seems we don't
need to call it anymore before parsing early options: none of them
need ld->config set.

Closes: #3030
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-12 19:08:24 -05:00
Rusty Russell
a3273d4c84 developer: IFDEV() macro.
There are some more #if DEVELOPER one-liners coming, this makes them
clear, but still lets them stand out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-12 05:11:56 +00:00
Rusty Russell
91072f56b0 developer: add 'dev-gossip-set-time' call to manipulate gossipd's time.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-12 05:11:56 +00:00
lisa neigut
a35677fa0f fundchannel: use plugin; delete interior impl
Switch over to using the fundchannel plugin.
2019-09-11 23:56:27 +00:00
lisa neigut
b1f05795a8 rpc: have fundchannel_start also return scriptpubkey
Easier to pass it back than dig it out of the address, since we
have it. Needed for extracting fundchannel
2019-09-11 23:56:27 +00:00
lisa neigut
19e65068a9 json: move param node_id parser to common
Going to need this for a plugin that's external
2019-09-11 23:56:27 +00:00
trueptolemy
6c708b5854 API: fundchannel_cancel can cancel fundchannel process before funding broadcast 2019-09-11 17:04:14 -05:00
trueptolemy
1e5b619832 lightningd: Store commands that try to forget channel in struct channel 2019-09-11 17:04:14 -05:00
trueptolemy
db145f575b Fix: fundee can forget channel if it receives error during CHANNELD_AWAITING_LOCKIN 2019-09-11 17:04:14 -05:00
trueptolemy
f715e3ad84 pay: A cleanup for the comment about wait_payment()
json_waitsendpay_on_resolve() has been replaced by wait_payment(), so correct it here.
2019-09-11 00:57:39 +00:00
trueptolemy
086f096b86 Pay: Notify 'sendpay_success' and 'sendpay_failure' when sendpay succeeds and fails 2019-09-11 00:57:39 +00:00
trueptolemy
82e8db4ba4 plugin: Another new notification type, 'sendpay_failure'
(The json when sendpay successes is too different when sendpay fails, so
divide the sendpay result into two notifications: `sendpay_success` and
`sendpay_failure`)

`sendpay_failure`

A notification for topic `sendpay_failure` is sent every time a sendpay
success(with `failed` status). The json is same as the return value of
command `sendpay`/`waitsendpay` when this cammand fails.

```json
{
  "sendpay_failure": {
  "code": 204,
  "message": "failed: WIRE_UNKNOWN_NEXT_PEER (reply from remote)",
  "data": {
    "id": 2,
    "payment_hash": "9036e3bdbd2515f1e653cb9f22f8e4c49b73aa2c36e937c926f43e33b8db8851",
    "destination": "035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d",
    "msatoshi": 100000000,
    "amount_msat": "100000000msat",
    "msatoshi_sent": 100001001,
    "amount_sent_msat": "100001001msat",
    "created_at": 1561395134,
    "status": "failed",
    "erring_index": 1,
    "failcode": 16394,
    "failcodename": "WIRE_UNKNOWN_NEXT_PEER",
    "erring_node": "022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59",
    "erring_channel": "103x2x1",
    "erring_direction": 0
    }
  }
}
```
`sendpay` doesn't wait for the result of sendpay and `waitsendpay`
returns the result of sendpay in specified time or timeout, but
`sendpay_failure` will always return the result anytime when sendpay
fails if is was subscribed.
2019-09-11 00:57:39 +00:00
trueptolemy
9d8f46149a API: Add payment fields(if not NULL) into return value when sendpay fails
pPayment field includes the basic information of the payment, so the return valves of 'sendpay_success()' and 'sendpay_fail()' should include this field.
Note "immediate_routing_failure" is before payment creation, and for this case, return won't include payment fields.
2019-09-11 00:57:39 +00:00
trueptolemy
507f8d46df pay: Warp the json process of payment fail field
We will also call this warped function in the json process of the 'sendpay_failure' notification.
2019-09-11 00:57:39 +00:00
trueptolemy
07f85cbf72 plugin: A new notification type, 'sendpay_success'
`sendpay_success`

A notification for topic `sendpay_success` is sent every time a sendpay
success(with `complete` status). The json is same as the return value of
command `sendpay`/`waitsendpay` when these cammand succeeds.

```json
{
	"sendpay_success": {
  "id": 1,
  "payment_hash": "5c85bf402b87d4860f4a728e2e58a2418bda92cd7aea0ce494f11670cfbfb206",
  "destination": "035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d",
  "msatoshi": 100000000,
  "amount_msat": "100000000msat",
  "msatoshi_sent": 100001001,
  "amount_sent_msat": "100001001msat",
  "created_at": 1561390572,
  "status": "complete",
  "payment_preimage": "9540d98095fd7f37687ebb7759e733934234d4f934e34433d4998a37de3733ee"
  }
}
```
`sendpay` doesn't wait for the result of sendpay and `waitsendpay`
returns the result of sendpay in specified time or timeout, but
`sendpay_success` will always return the result anytime when sendpay
successes if is was subscribed.
2019-09-11 00:57:39 +00:00
Rusty Russell
226e2aee48 option_static_remotekey: update to latest draft.
531c8d7d9b

In this one, we always send my_current_per_commitment_point, though it's
ignored.  And we have our official feature numbers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell
0861279b65 channeld: don't exchange my_current_per_commitment_point if option_static_remotekey
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell
87f0ee6351 channeld: set option_static_remotekey when negotiated.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell
11ee089d4b db: store option_static_remotekey for each channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
darosior
9be28fe40f daemons tour: minor typos correction 2019-09-10 02:02:51 +00:00
trueptolemy
ffb2ee9b21 bitcoind: Remove unused type bitcoind_mode 2019-09-09 04:26:03 +00:00
trueptolemy
7ffa4ad0bc plugin: Register forward_event notification 2019-09-08 16:02:00 +02:00
trueptolemy
a3eb2942de plugin: Register channel_opened notification 2019-09-08 16:02:00 +02:00
trueptolemy
2907e430d5 plugin: Register invoice_payment notification 2019-09-08 16:02:00 +02:00
trueptolemy
55fe2b03a2 plugin: Register warning notification 2019-09-08 16:02:00 +02:00
trueptolemy
74d8969967 plugin: Register disconnect notification 2019-09-08 16:02:00 +02:00
trueptolemy
c289fa8f19 plugin: Register connect notification 2019-09-08 16:02:00 +02:00
trueptolemy
25d79c5c3f notification: Add registeration interface 2019-09-08 16:02:00 +02:00
Rusty Russell
b65cbc7dc3 plugins: fix false-positive memleak.
This moves field initialization into plugins_new(), and
adds a memleak helper to search the request map:

=================================== ERRORS ====================================
___________________ ERROR at teardown of test_plugin_command ___________________
[gw0] linux -- Python 3.7.1 /opt/python/3.7.1/bin/python3.7
>       lambda: ihook(item=item, **kwds),
        when=when,
    )
../../../.local/lib/python3.7/site-packages/flaky/flaky_pytest_plugin.py:306:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/fixtures.py:112: in node_factory
    ok = nf.killall([not n.may_fail for n in nf.nodes])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <utils.NodeFactory object at 0x7f873b245278>, expected_successes = [True]
    def killall(self, expected_successes):
        """Returns true if every node we expected to succeed actually succeeded""
        unexpected_fail = False
        for i in range(len(self.nodes)):
            leaks = None
            # leak detection upsets VALGRIND by reading uninitialized mem.
            # If it's dead, we'll catch it below.
            if not VALGRIND:
                try:
                    # This also puts leaks in log.
                    leaks = self.nodes[i].rpc.dev_memleak()['leaks']
                except Exception:
                    pass

            try:
                self.nodes[i].stop()
            except Exception:
                if expected_successes[i]:
                    unexpected_fail = True

            if leaks is not None and len(leaks) != 0:
                raise Exception("Node {} has memory leaks: {}".format(
                    self.nodes[i].daemon.lightning_dir,
>                   json.dumps(leaks, sort_keys=True, indent=4)
                ))
E               Exception: Node /tmp/ltests-qm87my20/test_plugin_command_1/lightnng-1/ has memory leaks: [
E                   {
E                       "backtrace": [
E                           "ccan/ccan/tal/tal.c:437 (tal_alloc_)",
E                           "lightningd/jsonrpc.c:1112 (jsonrpc_request_start_)",
E                           "lightningd/plugin.c:1041 (plugin_config)",
E                           "lightningd/plugin.c:1072 (plugins_config)",
E                           "lightningd/plugin.c:846 (plugin_manifest_cb)",
E                           "lightningd/plugin.c:252 (plugin_response_handle)",
E                           "lightningd/plugin.c:342 (plugin_read_json_one)",
E                           "lightningd/plugin.c:367 (plugin_read_json)",
E                           "ccan/ccan/io/io.c:59 (next_plan)",
E                           "ccan/ccan/io/io.c:407 (do_plan)",
E                           "ccan/ccan/io/io.c:417 (io_ready)",
E                           "ccan/ccan/io/poll.c:445 (io_loop)",
E                           "lightningd/io_loop_with_timers.c:24 (io_loop_with_tiers)",
E                           "lightningd/lightningd.c:840 (main)"
E                       ],
E                       "label": "lightningd/jsonrpc.c:1112:struct jsonrpc_reques",
E                       "parents": [
E                           "lightningd/plugin.c:66:struct plugin",
E                           "lightningd/lightningd.c:103:struct lightningd"
E                       ],
E                       "value": "0x55d6385e4088"
E                   },
E                   {
E                       "backtrace": [
E                           "ccan/ccan/tal/tal.c:437 (tal_alloc_)",
E                           "lightningd/jsonrpc.c:1112 (jsonrpc_request_start_)",
E                           "lightningd/plugin.c:1041 (plugin_config)",
E                           "lightningd/plugin.c:1072 (plugins_config)",
E                           "lightningd/plugin.c:846 (plugin_manifest_cb)",
E                           "lightningd/plugin.c:252 (plugin_response_handle)",
E                           "lightningd/plugin.c:342 (plugin_read_json_one)",
E                           "lightningd/plugin.c:367 (plugin_read_json)",
E                           "ccan/ccan/io/io.c:59 (next_plan)",
E                           "ccan/ccan/io/io.c:407 (do_plan)",
E                           "ccan/ccan/io/io.c:417 (io_ready)",
E                           "ccan/ccan/io/poll.c:445 (io_loop)",
E                           "lightningd/io_loop_with_timers.c:24 (io_loop_with_tiers)",
E                           "lightningd/lightningd.c:840 (main)"
E                       ],
E                       "label": "lightningd/jsonrpc.c:1112:struct jsonrpc_reques",
E                       "parents": [
E                           "lightningd/plugin.c:66:struct plugin",
E                           "lightningd/lightningd.c:103:struct lightningd"
E                       ],
E                       "value": "0x55d6386529d8"
E                   }
E               ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-07 16:54:03 +02:00
Rusty Russell
c99906a9a9 per-peer-daemons: tie in gossip filter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00