Commit Graph

2213 Commits

Author SHA1 Message Date
trueptolemy
d8dce6e61f cleanup: Use u32 as the type of max_hops in gossipd 2019-09-24 16:01:24 +02:00
Christian Decker
5953a5051c cli: Add command line option to specify the wallet location
Will be demuxed into starting the selected DB backend in one of the next
commits. Defaults to the old database location.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Rusty Russell
27790832a5 gossipd: gossip_queries_ex is not longer experimental.
The master spec has some typos which make it not parse, so I created
a PR and generated the CSV from that:

https://github.com/lightningnetwork/lightning-rfc/pull/673

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell
147eaced2e developer: consolidiate gossip timing options into one --dev-fast-gossip.
It's generally clearer to have simple hardcoded numbers with an
#if DEVELOPER around it, than apparent variables which aren't, really.

Interestingly, our pruning test was always kinda broken: we have to pass
two cycles, since l2 will refresh the channel once to avoid pruning.

Do the more obvious thing, and cut the network in half and check that
l1 and l3 time out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell
178baeba6c gossipd: get gossip_min_interval from lightningd.
Default is 5 x gossip interval == 5 minutes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell
9578fb4035 lightningd: fix crash in cancel_channel
This path is not atomic, so we can't assume channel stays around.  It
could be the peer closes, it could be we get a parallel
fund_channel_cancel.  test_funding_cancel_race caused this crash:

    FATAL SIGNAL 6 (version 5f0a18e)
    backtrace: common/daemon.c:45 (send_backtrace) 0x55c7c373a429
    backtrace: common/daemon.c:53 (crashdump) 0x55c7c373a479
    backtrace: (null):0 ((null)) 0x7f88ee6ddf5f
    backtrace: (null):0 ((null)) 0x7f88ee6dded7
    backtrace: (null):0 ((null)) 0x7f88ee6bf534
    backtrace: ccan/ccan/tal/tal.c:93 (call_error) 0x55c7c379427c
    backtrace: ccan/ccan/tal/tal.c:165 (check_bounds) 0x55c7c3794444
    backtrace: ccan/ccan/tal/tal.c:174 (to_tal_hdr) 0x55c7c3794483
    backtrace: ccan/ccan/tal/tal.c:186 (to_tal_hdr_or_null) 0x55c7c3794504
    backtrace: ccan/ccan/tal/tal.c:421 (tal_alloc_) 0x55c7c3794c10
    backtrace: ccan/ccan/tal/tal.c:466 (tal_alloc_arr_) 0x55c7c3794ded
    backtrace: ccan/ccan/tal/str/str.c:91 (tal_vfmt_) 0x55c7c3793560
    backtrace: common/wire_error.c:22 (towire_errorfmtv) 0x55c7c3747f7b
    backtrace: common/wire_error.c:39 (towire_errorfmt) 0x55c7c37480a1
    backtrace: lightningd/channel_control.c:635 (process_check_funding_broadcast) 0x55c7c37015bb
    backtrace: lightningd/bitcoind.c:558 (process_gettxout) 0x55c7c36f8e75
    backtrace: lightningd/bitcoind.c:227 (bcli_finished) 0x55c7c36f8090
    backtrace: ccan/ccan/io/poll.c:244 (destroy_conn) 0x55c7c37869fe
    backtrace: ccan/ccan/io/poll.c:250 (destroy_conn_close_fd) 0x55c7c3786a1e
    backtrace: ccan/ccan/tal/tal.c:235 (notify) 0x55c7c3794629
    backtrace: ccan/ccan/tal/tal.c:397 (del_tree) 0x55c7c3794b18
    backtrace: ccan/ccan/tal/tal.c:481 (tal_free) 0x55c7c3794ea4
    backtrace: ccan/ccan/io/io.c:450 (io_close) 0x55c7c378521d
    backtrace: ccan/ccan/io/poll.c:449 (io_loop) 0x55c7c3787139
    backtrace: lightningd/io_loop_with_timers.c:24 (io_loop_with_timers) 0x55c7c370b26d
    backtrace: lightningd/lightningd.c:837 (main) 0x55c7c3711661
    backtrace: (null):0 ((null)) 0x7f88ee6c0b6a
    backtrace: (null):0 ((null)) 0x55c7c36f70b9
    backtrace: (null):0 ((null)) 0xffffffffffffffff

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-19 18:41:12 +08:00
Rusty Russell
794bebcc81 lightningd: add find_channel_by_id() helper, use it.
This is overkill for the moment, but we'll need it in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-19 18:41:12 +08:00
trueptolemy
5361a5d059 JSON-API: getroute now also support exclude nodes 2019-09-16 12:22:06 +08:00
trueptolemy
090a43fd3d gossip: Add the struct exclude_entry and enum exclude_entry_type 2019-09-16 12:22:06 +08:00
Rusty Russell
a988ded3fa lightningd: fix case where config file overrides lightning-dir.
The simplest case is to explicitly load it when we see it's been
set.

This involves neatening the default config setup, to remove it from
opt_parse_from_config() and into the caller.  It also seems we don't
need to call it anymore before parsing early options: none of them
need ld->config set.

Closes: #3030
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-12 19:08:24 -05:00
Rusty Russell
a3273d4c84 developer: IFDEV() macro.
There are some more #if DEVELOPER one-liners coming, this makes them
clear, but still lets them stand out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-12 05:11:56 +00:00
Rusty Russell
91072f56b0 developer: add 'dev-gossip-set-time' call to manipulate gossipd's time.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-12 05:11:56 +00:00
lisa neigut
a35677fa0f fundchannel: use plugin; delete interior impl
Switch over to using the fundchannel plugin.
2019-09-11 23:56:27 +00:00
lisa neigut
b1f05795a8 rpc: have fundchannel_start also return scriptpubkey
Easier to pass it back than dig it out of the address, since we
have it. Needed for extracting fundchannel
2019-09-11 23:56:27 +00:00
lisa neigut
19e65068a9 json: move param node_id parser to common
Going to need this for a plugin that's external
2019-09-11 23:56:27 +00:00
trueptolemy
6c708b5854 API: fundchannel_cancel can cancel fundchannel process before funding broadcast 2019-09-11 17:04:14 -05:00
trueptolemy
1e5b619832 lightningd: Store commands that try to forget channel in struct channel 2019-09-11 17:04:14 -05:00
trueptolemy
db145f575b Fix: fundee can forget channel if it receives error during CHANNELD_AWAITING_LOCKIN 2019-09-11 17:04:14 -05:00
trueptolemy
f715e3ad84 pay: A cleanup for the comment about wait_payment()
json_waitsendpay_on_resolve() has been replaced by wait_payment(), so correct it here.
2019-09-11 00:57:39 +00:00
trueptolemy
086f096b86 Pay: Notify 'sendpay_success' and 'sendpay_failure' when sendpay succeeds and fails 2019-09-11 00:57:39 +00:00
trueptolemy
82e8db4ba4 plugin: Another new notification type, 'sendpay_failure'
(The json when sendpay successes is too different when sendpay fails, so
divide the sendpay result into two notifications: `sendpay_success` and
`sendpay_failure`)

`sendpay_failure`

A notification for topic `sendpay_failure` is sent every time a sendpay
success(with `failed` status). The json is same as the return value of
command `sendpay`/`waitsendpay` when this cammand fails.

```json
{
  "sendpay_failure": {
  "code": 204,
  "message": "failed: WIRE_UNKNOWN_NEXT_PEER (reply from remote)",
  "data": {
    "id": 2,
    "payment_hash": "9036e3bdbd2515f1e653cb9f22f8e4c49b73aa2c36e937c926f43e33b8db8851",
    "destination": "035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d",
    "msatoshi": 100000000,
    "amount_msat": "100000000msat",
    "msatoshi_sent": 100001001,
    "amount_sent_msat": "100001001msat",
    "created_at": 1561395134,
    "status": "failed",
    "erring_index": 1,
    "failcode": 16394,
    "failcodename": "WIRE_UNKNOWN_NEXT_PEER",
    "erring_node": "022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59",
    "erring_channel": "103x2x1",
    "erring_direction": 0
    }
  }
}
```
`sendpay` doesn't wait for the result of sendpay and `waitsendpay`
returns the result of sendpay in specified time or timeout, but
`sendpay_failure` will always return the result anytime when sendpay
fails if is was subscribed.
2019-09-11 00:57:39 +00:00
trueptolemy
9d8f46149a API: Add payment fields(if not NULL) into return value when sendpay fails
pPayment field includes the basic information of the payment, so the return valves of 'sendpay_success()' and 'sendpay_fail()' should include this field.
Note "immediate_routing_failure" is before payment creation, and for this case, return won't include payment fields.
2019-09-11 00:57:39 +00:00
trueptolemy
507f8d46df pay: Warp the json process of payment fail field
We will also call this warped function in the json process of the 'sendpay_failure' notification.
2019-09-11 00:57:39 +00:00
trueptolemy
07f85cbf72 plugin: A new notification type, 'sendpay_success'
`sendpay_success`

A notification for topic `sendpay_success` is sent every time a sendpay
success(with `complete` status). The json is same as the return value of
command `sendpay`/`waitsendpay` when these cammand succeeds.

```json
{
	"sendpay_success": {
  "id": 1,
  "payment_hash": "5c85bf402b87d4860f4a728e2e58a2418bda92cd7aea0ce494f11670cfbfb206",
  "destination": "035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d",
  "msatoshi": 100000000,
  "amount_msat": "100000000msat",
  "msatoshi_sent": 100001001,
  "amount_sent_msat": "100001001msat",
  "created_at": 1561390572,
  "status": "complete",
  "payment_preimage": "9540d98095fd7f37687ebb7759e733934234d4f934e34433d4998a37de3733ee"
  }
}
```
`sendpay` doesn't wait for the result of sendpay and `waitsendpay`
returns the result of sendpay in specified time or timeout, but
`sendpay_success` will always return the result anytime when sendpay
successes if is was subscribed.
2019-09-11 00:57:39 +00:00
Rusty Russell
226e2aee48 option_static_remotekey: update to latest draft.
531c8d7d9b

In this one, we always send my_current_per_commitment_point, though it's
ignored.  And we have our official feature numbers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell
0861279b65 channeld: don't exchange my_current_per_commitment_point if option_static_remotekey
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell
87f0ee6351 channeld: set option_static_remotekey when negotiated.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell
11ee089d4b db: store option_static_remotekey for each channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
darosior
9be28fe40f daemons tour: minor typos correction 2019-09-10 02:02:51 +00:00
trueptolemy
ffb2ee9b21 bitcoind: Remove unused type bitcoind_mode 2019-09-09 04:26:03 +00:00
trueptolemy
7ffa4ad0bc plugin: Register forward_event notification 2019-09-08 16:02:00 +02:00
trueptolemy
a3eb2942de plugin: Register channel_opened notification 2019-09-08 16:02:00 +02:00
trueptolemy
2907e430d5 plugin: Register invoice_payment notification 2019-09-08 16:02:00 +02:00
trueptolemy
55fe2b03a2 plugin: Register warning notification 2019-09-08 16:02:00 +02:00
trueptolemy
74d8969967 plugin: Register disconnect notification 2019-09-08 16:02:00 +02:00
trueptolemy
c289fa8f19 plugin: Register connect notification 2019-09-08 16:02:00 +02:00
trueptolemy
25d79c5c3f notification: Add registeration interface 2019-09-08 16:02:00 +02:00
Rusty Russell
b65cbc7dc3 plugins: fix false-positive memleak.
This moves field initialization into plugins_new(), and
adds a memleak helper to search the request map:

=================================== ERRORS ====================================
___________________ ERROR at teardown of test_plugin_command ___________________
[gw0] linux -- Python 3.7.1 /opt/python/3.7.1/bin/python3.7
>       lambda: ihook(item=item, **kwds),
        when=when,
    )
../../../.local/lib/python3.7/site-packages/flaky/flaky_pytest_plugin.py:306:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/fixtures.py:112: in node_factory
    ok = nf.killall([not n.may_fail for n in nf.nodes])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <utils.NodeFactory object at 0x7f873b245278>, expected_successes = [True]
    def killall(self, expected_successes):
        """Returns true if every node we expected to succeed actually succeeded""
        unexpected_fail = False
        for i in range(len(self.nodes)):
            leaks = None
            # leak detection upsets VALGRIND by reading uninitialized mem.
            # If it's dead, we'll catch it below.
            if not VALGRIND:
                try:
                    # This also puts leaks in log.
                    leaks = self.nodes[i].rpc.dev_memleak()['leaks']
                except Exception:
                    pass

            try:
                self.nodes[i].stop()
            except Exception:
                if expected_successes[i]:
                    unexpected_fail = True

            if leaks is not None and len(leaks) != 0:
                raise Exception("Node {} has memory leaks: {}".format(
                    self.nodes[i].daemon.lightning_dir,
>                   json.dumps(leaks, sort_keys=True, indent=4)
                ))
E               Exception: Node /tmp/ltests-qm87my20/test_plugin_command_1/lightnng-1/ has memory leaks: [
E                   {
E                       "backtrace": [
E                           "ccan/ccan/tal/tal.c:437 (tal_alloc_)",
E                           "lightningd/jsonrpc.c:1112 (jsonrpc_request_start_)",
E                           "lightningd/plugin.c:1041 (plugin_config)",
E                           "lightningd/plugin.c:1072 (plugins_config)",
E                           "lightningd/plugin.c:846 (plugin_manifest_cb)",
E                           "lightningd/plugin.c:252 (plugin_response_handle)",
E                           "lightningd/plugin.c:342 (plugin_read_json_one)",
E                           "lightningd/plugin.c:367 (plugin_read_json)",
E                           "ccan/ccan/io/io.c:59 (next_plan)",
E                           "ccan/ccan/io/io.c:407 (do_plan)",
E                           "ccan/ccan/io/io.c:417 (io_ready)",
E                           "ccan/ccan/io/poll.c:445 (io_loop)",
E                           "lightningd/io_loop_with_timers.c:24 (io_loop_with_tiers)",
E                           "lightningd/lightningd.c:840 (main)"
E                       ],
E                       "label": "lightningd/jsonrpc.c:1112:struct jsonrpc_reques",
E                       "parents": [
E                           "lightningd/plugin.c:66:struct plugin",
E                           "lightningd/lightningd.c:103:struct lightningd"
E                       ],
E                       "value": "0x55d6385e4088"
E                   },
E                   {
E                       "backtrace": [
E                           "ccan/ccan/tal/tal.c:437 (tal_alloc_)",
E                           "lightningd/jsonrpc.c:1112 (jsonrpc_request_start_)",
E                           "lightningd/plugin.c:1041 (plugin_config)",
E                           "lightningd/plugin.c:1072 (plugins_config)",
E                           "lightningd/plugin.c:846 (plugin_manifest_cb)",
E                           "lightningd/plugin.c:252 (plugin_response_handle)",
E                           "lightningd/plugin.c:342 (plugin_read_json_one)",
E                           "lightningd/plugin.c:367 (plugin_read_json)",
E                           "ccan/ccan/io/io.c:59 (next_plan)",
E                           "ccan/ccan/io/io.c:407 (do_plan)",
E                           "ccan/ccan/io/io.c:417 (io_ready)",
E                           "ccan/ccan/io/poll.c:445 (io_loop)",
E                           "lightningd/io_loop_with_timers.c:24 (io_loop_with_tiers)",
E                           "lightningd/lightningd.c:840 (main)"
E                       ],
E                       "label": "lightningd/jsonrpc.c:1112:struct jsonrpc_reques",
E                       "parents": [
E                           "lightningd/plugin.c:66:struct plugin",
E                           "lightningd/lightningd.c:103:struct lightningd"
E                       ],
E                       "value": "0x55d6386529d8"
E                   }
E               ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-07 16:54:03 +02:00
Rusty Russell
c99906a9a9 per-peer-daemons: tie in gossip filter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Rusty Russell
aca2e4f722 common/memleak: add dynamic hooks for assisting memleak.
Rather than reaching into data structures, let them register their own
callbacks.  This avoids us having to expose "memleak_remove_xxx"
functions, and call them manually.

Under the hood, this is done by having a specially-named tal child of
the thing we want to assist, containing the callback.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Rusty Russell
077ba88b88 JSON: remove listpayments.
You either want listpays (high level) or listsendpays.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:19:14 +02:00
Rusty Russell
884f4fa6d0 JSON: Remove description fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:19:14 +02:00
Rusty Russell
acf3952acc JSON: remove handling of pre-Adelaide (B:T:N) short_channel_ids.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:19:14 +02:00
Rusty Russell
a134062f98 bolt11: handle 9 fields for new features.
This implements https://github.com/lightningnetwork/lightning-rfc/pull/656

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-05 23:39:05 -05:00
Christian Decker
b6d583c26a db: Move tracking of pending statements into the struct db
We now have a much stronger consistency check from the combination of
transaction wrapping, tal memory leak detection. Tramsaction wrapping ensures
that each statement is executed before the transaction is committed. The
commit is also driven by the `io_loop`, which means that it is no longer
possible for us to have statements outside of transactions and transactions
are guaranteed to commit at the round's end.

By adding the tal-awareness we can also get a much better indication as to
whether we have un-freed statements flying around, which we can test at the
end of the round as well.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker
803007ecdf db: Make the db struct private and provide accessors instead
We will soon generalize the DB, so directly reaching into the `struct db`
instance to talk to the sqlite3 connection is bad anyway. This increases
flexibility and allows us to tailor the actual implementation to the
underlying DB.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
darosior
4672795e1a lightningd: don't join the config_dir to the pid_file path 2019-09-05 19:10:33 +02:00
Rusty Russell
049529542a lightningd: delay reprocessing of incoming htlcs at startup until plugins ready.
Fixes: #2923
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-01 16:55:32 +02:00
Rusty Russell
6901732ee0 lightningd: create --list-features-only which lists what features we support.
This allows the lightning-rfc protocol tests to automatically query what
features we support.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
d943d8abbc lightningd: expose full onion error when we have it.
Mainly useful for testing.  In particular, we don't save it to the db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00