Commit Graph

2122 Commits

Author SHA1 Message Date
Christian Decker
820b52207e lightningd: Defer creating the PID until we actually want to start
This was causing `--help` to fail if we already had a `lightningd` running
with the same `--lightning-dir`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-06 13:10:32 +08:00
darosior
a2c00d42f2 Remove the 'signal_startup' member of the plugin struct
It's set but not used
2019-08-05 23:06:55 -05:00
Rusty Russell
e78a80495b log: make --log-file an early arg (since we move to dir early now).
Otherwise we bleed plugin log messages to stderr, as they're initted before
log-file.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-05 17:09:57 +08:00
Rusty Russell
e808aaa1bb lightningd: clean up pidfile test, crashlog.
1. Now checking the pid file really does precede touching the db and
   starting plugins, which is far safer.
2. Crashlog is now activated just after daemon parent release, and just
   before the main loop, which means no "crash" on startup if we call fatal().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Rusty Russell
979fbeb3b0 lightningd: simplify --daemon.
Dumb programs which have a --daemon option call fork() early.  This is
terrible UX since startup errors get lost: the program exits with
"success" immediately then you discover via the logs that it didn't
start at all.

However, forking late introduced a heap of problems with changing
pids.  Instead, fork early but keep stderr and the parent around: if
we fail early on, the parent fails with us.  We release our parent
with an explicit action just before the main loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-04 21:29:03 +02:00
Christian Decker
8ed77753ef json-rpc: Add size and cumulative size to dev-memdump
Since we are walking the entire allocation tree anyway, and access the tal
metadata anyway, we can just as well also track the size of the memory
allocations to simplify debugging of memory use.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-04 20:54:53 +02:00
Rusty Russell
4c9bfa351a lightningd: handle --version before trying to move to lightning-dir.
Otherwise it creates the lightning-dir.  This can't be helped for --help
(at least, if plugins are present), but --version simply prints and exits.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell
b460590278 plugins: detect and fixup old relative paths.
Note that we move adding the plugin to the plugins list to the end, otherwise
the hook from logging can examine the (uninitialized) plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell
fd63b8bf53 lightningd: chdir as soon as we know lightning dir.
This is easy since we did the option parsing cleanup, but it has the
effect that plugins are launched from the lightning-dir.  Now
we have dynamic plugins, this means startup and post-startup plugins
experience the same environment.

This is absolutely a desirable thing: they can just drop files in
their cwd rather than having to move (including, I might note, core
files!).

We also highlight the change in various places (and a drive-up update
of PLUGINS.md which says you have to use --plugin).

The next patch adds a backwards compatibility wedge for old users of
relative plugin paths.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-03 09:10:11 +00:00
Rusty Russell
913a1a9b59 bolt: update to 8b2cf0054660bece9e1004f42a500c6a1a77efd3
This contains only typo fixes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell
9b88fd4c60 bolt: update to 950b2f5481c2a4b57ef1102e2374543e81c4aa88
Just a simple field renaming which only alters comments,
though I updated variable names too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell
2b3003f25b channeld: delay sending channel_announcement by 60 seconds.
We currently send channel_announcement as soon as we and our
peer agree it's 6 blocks deep.  In theory, our other peers might
not have seen that block yet though, so delay a little.

This is mitigated by two factors:
1. lnd will stash any "not ready yet" channel_announcements anyway.
2. c-lightning doesn't enforce the 6 depth minimum at all.

We should not rely on other nodes' generosity or laxity, however!

Next release, we can start enforcing the depth limit, and maybe stashing
ones which don't quite make it (or simply enforce depth 5, not 6).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 16:50:45 +02:00
Rusty Russell
359433f374 lightningd: convert the compiler-wanted-init FIXME.
I'm sure there are others, but this was the only one which showed up
in grep.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell
02609773c0 lightningd: suppress gcc-7.4.0 error
In file included from wallet/test/run-wallet.c:15:0:
./lightningd/peer_htlcs.c: In function ‘htlcs_reconnect’:
./lightningd/peer_htlcs.c:2060:15: error: ‘failcode’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
   }  else if (failcode) {
               ^~~~~~~~
./lightningd/peer_htlcs.c:2056:19: error: ‘failcode’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
          failcode != 0
          ~~~~~~~~~^~~~

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
trueptolemy
31e2e70f17 invoice: a cleanup for the json of struct sha256
Here should't be accessed directly to the underlying of struct sha256.
2019-08-01 18:49:25 +08:00
trueptolemy
e75d8e061b Plugin: New notification type, forward_event
`forward_event`

A notification for topic `forward_event` is sent every time the status
of a forward payment is set. The json format is same as the API
`listforwards`.

```json
{
  "forward_event": {
  "payment_hash": "f5a6a059a25d1e329d9b094aeeec8c2191ca037d3f5b0662e21ae850debe8ea2",
  "in_channel": "103x2x1",
  "out_channel": "103x1x1",
  "in_msatoshi": 100001001,
  "in_msat": "100001001msat",
  "out_msatoshi": 100000000,
  "out_msat": "100000000msat",
  "fee": 1001,
  "fee_msat": "1001msat",
  "status": "settled",
  "received_time": 1560696342.368,
  "resolved_time": 1560696342.556
  }
}
```
or

```json
{
  "forward_event": {
  "payment_hash": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
  "in_channel": "103x2x1",
  "out_channel": "110x1x0",
  "in_msatoshi": 100001001,
  "in_msat": "100001001msat",
  "out_msatoshi": 100000000,
  "out_msat": "100000000msat",
  "fee": 1001,
  "fee_msat": "1001msat",
  "status": "local_failed",
  "failcode": 16392,
  "failreason": "WIRE_PERMANENT_CHANNEL_FAILURE",
  "received_time": 1560696343.052
  }
}

```
 - The status includes `offered`, `settled`, `failed` and `local_failed`,
   and they are all string type in json.
   - When the forward payment is valid for us, we'll set `offered`
     and send the forward payment to next hop to resolve;
   - When the payment forwarded by us gets paid eventually, the forward
     payment will change the status from `offered` to `settled`;
   - If payment fails locally(like failing to resolve locally) or the
     corresponding htlc with next hop fails(like htlc timeout), we will
     set the status as `local_failed`. `local_failed` may be set before
     setting `offered` or after setting `offered`. In fact, from the
     time we receive the htlc of the previous hop, all we can know the
     cause of the failure is treated as `local_failed`. `local_failed`
     only occuors locally or happens in the htlc between us and next hop;
     - If `local_failed` is set before `offered`, this
       means we just received htlc from the previous hop and haven't
       generate htlc for next hop. In this case, the json of `forward_event`
       sets the fields of `out_msatoshi`, `out_msat`,`fee` and `out_channel`
       as 0;
       - Note: In fact, for this case we may be not sure if this incoming
         htlc represents a pay to us or a payment we need to forward.
         We just simply treat all incoming failed to resolve as
         `local_failed`.
     - Only in `local_failed` case, json includes `failcode` and
       `failreason` fields;
   - `failed` means the payment forwarded by us fails in the
     latter hops, and the failure isn't related to us, so we aren't
     accessed to the fail reason. `failed` must be set after
     `offered`.
     - `failed` case doesn't include `failcode` and `failreason`
       fields;
 - `received_time` means when we received the htlc of this payment from
   the previous peer. It will be contained into all status case;
 - `resolved_time` means when the htlc of this payment between us and the
   next peer was resolved. The resolved result may success or fail, so
   only `settled` and `failed` case contain `resolved_time`;
 - The `failcode` and `failreason` are defined in [BOLT 4][bolt4-failure-codes].
2019-08-01 18:49:25 +08:00
trueptolemy
a449a91ae2 JSON: Warp the process of forward payment json object
Warp this process as a new function: 'void json_format_forwarding_object()'. This function will be used in 'forward_event' next, and can ensure the consistent json object structure for forward_payment between 'listforwards' API and 'forward_event' notification.
2019-08-01 18:49:25 +08:00
trueptolemy
bcec6bb6cc API: 'listforwards' now include 'payment_hash' field
'payment_hash' can help users learn more about the forward payment.
2019-08-01 18:49:25 +08:00
Rusty Russell
2255dd4dda
lightningd: avoid thundering herd on restart.
The reason lnd was sending sync error was that we were taking more than
30 seconds to send the channel_reestablish after connect.  That's
understandable on my test node under valgrind, but shouldn't happen normally.

However, it seems it has at least once,
(see https://github.com/ElementsProject/lightning/issues/2847)
: space out startup so it's less likely to happen.

Suggested-by: @cfromknecht
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-01 01:10:11 +00:00
Rusty Russell
cc70b9c4ec wire: use common/bigsize routines
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Rusty Russell
3fa375881a bigsize: make it a proper first-class type.
It doesn't belong in bitcoin, and should not be confused with varint_t.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Simon Vrouwe
1ae8b73f0e doc: invoice manpage: add some details about route hint selection
Also add more detail to the warnings returned by the invoice command.
2019-07-31 23:25:37 +00:00
Christian Decker
5dff67900e tx: Add chainparams when deserializing transactions from wire msgs
This is the other origin, besides `bitcoin_tx`, where we create `bitcoin_tx`
instances, so add the context as soon as possible. Sadly I can't weave the
chainparams into the deserialization code since that'd need to change all the
generated wire code as well.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
Christian Decker
d14bd286ba bitcoin: Add chainparams to transactions from blocks
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
Christian Decker
9288a7906b tx: Add chainparams to struct bitcoin_tx as context
The way we build transactions, serialize them, and compute fees depends on the
chain we are working on, so let's add some context to the transactions.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
Christian Decker
2537bd5afa closingd: Tell closingd which chain we are working on
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
Christian Decker
c5ac6c7b1f hsmd: Tell hsmd which chain we are working on
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
Christian Decker
15e73e3dc3 onchaind: Pass genesis hash to onchaind so it knows the chainparams
It'll be creating quite a few transactions and we will have to know which
params to use.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
trueptolemy
130bf20516 lightningd: check bitcoind version when setup_topology 2019-07-30 17:38:54 +08:00
Christian Decker
b83d15ea4a sphinx: Remove standalone v0 payload in favor of the unionized one
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
6831db62f7 sphinx: Clean up after migrating to the sphinx_path struct
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
a0a1a1f752 sphinx: Add function to add a new v0 hop to a sphinx_path
This is just taking the existing serialization code and repackaging it in a
more useful form.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Rusty Russell
79d32ec2f2 plugin: notice when plugin has *started* configuring.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 08:44:01 +08:00
Rusty Russell
a477c69d46 lightningd: fix crash on dynamic plugin startup
2019-07-29T04:25:31.503Z DEBUG plugin-manager started(25413) /home/rusty/text/webinar/lightning-2019-07/helloworld.py
2019-07-29T04:25:39.168Z UNUSUAL lightningd(24972): Unable to estimate CONSERVATIVE/2 fee
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): FATAL SIGNAL 6 (version v0.7.1-144-g6bb8525)
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: common/daemon.c:45 (send_backtrace) 0x5581b809d7d6
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: common/daemon.c:53 (crashdump) 0x5581b809d823
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: (null):0 ((null)) 0x7f4ecd46af5f
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: (null):0 ((null)) 0x7f4ecd46aed7
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: (null):0 ((null)) 0x7f4ecd44c534
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:93 (call_error) 0x5581b80da741
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:165 (check_bounds) 0x5581b80da81d
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:174 (to_tal_hdr) 0x5581b80da845
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:186 (to_tal_hdr_or_null) 0x5581b80da88e
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:632 (tal_bytelen) 0x5581b80db59c
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/plugin.c:1056 (plugin_subscriptions_contains) 0x5581b8095aca
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/plugin.c:1071 (plugins_notify) 0x5581b80972f4
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/notification.c:63 (notify_warning) 0x5581b808615a
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/log.c:269 (logv) 0x5581b8085393
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/log.c:329 (log_) 0x5581b8085233
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/bitcoind.c:372 (process_estimatefee) 0x5581b80759d7
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/bitcoind.c:226 (bcli_finished) 0x5581b8075cfb
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/io/poll.c:244 (destroy_conn) 0x5581b80d2413
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/io/poll.c:250 (destroy_conn_close_fd) 0x5581b80d2435
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:235 (notify) 0x5581b80dabb0
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:397 (del_tree) 0x5581b80dac7a
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:481 (tal_free) 0x5581b80db1c5
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/io/io.c:450 (io_close) 0x5581b80d1147
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/io/poll.c:449 (io_loop) 0x5581b80d29ff
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/io_loop_with_timers.c:24 (io_loop_with_timers) 0x5581b808058d
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/lightningd.c:830 (main) 0x5581b8084396
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: (null):0 ((null)) 0x7f4ecd44db6a
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: (null):0 ((null)) 0x5581b8075059
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: (null):0 ((null)) 0xffffffffffffffff

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 08:44:01 +08:00
Rusty Russell
310d806dd3 plugins: document and extend the ~/.lightning/plugins/ dir.
Load any plugins directly as well as subdirs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-29 14:32:07 +02:00
darosior
dda154612c doc: add the new init and getmanifest fields 2019-07-28 07:24:04 +00:00
darosior
307fb0708e lightningd/plugin_control: don't control non-dynamic plugins 2019-07-28 07:24:04 +00:00
darosior
12e28c2554 lightningd/plugin: Add a 'dynamic' field to getmanifest and a 'startup' field to init
This lets a plugin specify whether it can be restarted, and to know if it is started at lightningd startup
2019-07-28 07:24:04 +00:00
darosior
2864b4de01 lightningd/plugin_control: add a 'plugin' command
This adds a new pair of files : lightningd/plugin_control, along with a new RPC
command : 'plugin'. This command can be used to manage plugins without restarting lightningd:

lightning-cli plugin start helloworld.py
lightning-cli plugin stop helloworld.py
2019-07-28 07:24:04 +00:00
darosior
d299420fbe lightningd/plugin_hook: make it possible to unregister a hook
This adds 'plugin_unregister_hook' and 'plugin_unregister_hook_all'
functions to unregister a given hook a plugin registered, or all hooks a
plugin registered for. Since hooks can only be registered once, it's
useful in the case a new plugin is added which would be prefered for
hook registration over an already loaded plugin.
2019-07-28 07:24:04 +00:00
darosior
ce12a37a2b lightningd/plugin: Add a 'configured' member to the plugin struct, split 'plugins_init'
This adds a 'configured' boolean member to the plugin struct so that we can add plugins to ld->plugins' list and differenciate fresh plugins.
This also adds 'plugins_start' so that new plugins can be started without calling 'plugins_init' and running an io loop
2019-07-28 07:24:04 +00:00
darosior
2e25c87bd4 lightningd/plugin: Move structs to header, make 'paths_match' and 'plugin_kill' non static 2019-07-28 07:24:04 +00:00
Rusty Russell
6da420a65b lightning-cli: change default printing in response to "format-hint": "simple".
And set it for 'help <command>'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-28 06:09:56 +00:00
Rusty Russell
b03369ea2d common: generalize json_tok_remove.
It assumes the head of the array is the object/array we want to remove from,
but that's not true if we're trying to remove from a sub-object.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-28 06:09:56 +00:00
darosior
736651ba43 lightningd/notification: add 'channel_opened' notification
This notification is sent when a peer succesfully opens a channel to us
2019-07-27 12:18:25 +02:00
darosior
f55d29ee49 lightningd/notification: use notification name in 'jsonrpc_notification_start()'
That way 'notification_topics' array modification would not lead to unwanted renaming
2019-07-27 12:18:25 +02:00
Rusty Russell
39e8e98b49 options: make option registration a single function.
No code changes, just move.

Put all the dev options into the one function, and register (and
comment on) the early args first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-27 05:16:22 +00:00
Rusty Russell
f68c9fa9c9 opt: make sure early cmdline options override config file.
I noticed that --network=regtest didn't override 'network=bitcoin' in
the config file.

Normally we parse the config file first, then the commandline (so the cmdline
wins).  But for early options, we do cmdline first so we can find the config
file.  That was fine when the only early option was the location of the
config file, but now it includes plugins and the network setting.

So do a boutique cmdline parse *just* to find the config file, then parse
the config file early options, then the cmdline early options.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-27 05:16:22 +00:00
Rusty Russell
0b5b1faff5 common/configdir: simply supply defaults, leave parsing to programs.
We're going to get tricky with lightingd's parsing next, so split it out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-27 05:16:22 +00:00
Rusty Russell
875854f71c invoice: fix up internal help msg to match updated expiry default.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-27 05:16:22 +00:00