doc: improve/update lightning-plugin, PLUGINS.md and lightning-listconfigs

doc/PLUGINS.md

@@ -7,7 +7,7 @@ variety of ways:
 
  - **Command line option passthrough** allows plugins to register their
    own command line options that are exposed through `lightningd` so
-   that only the main process needs to be configured.
+   that only the main process needs to be configured[^options].
  - **JSON-RPC command passthrough** adds a way for plugins to add their
    own commands to the JSON-RPC interface.
  - **Event stream subscriptions** provide plugins with a push-based
@@ -22,6 +22,15 @@ used as protocol on top of the two streams, with the plugin acting as
 server and `lightningd` acting as client. The plugin file needs to be
 executable (e.g. use `chmod a+x plugin_name`)
 
+A `helloworld.py` example plugin based on [pyln-client][pyln-client]
+can be found [here](../contrib/plugins/helloworld.py). There is also a
+[repository](https://github.com/lightningd/plugins) with a collection of
+actively maintained plugins and finally, `lightningd`'s own internal
+[tests](../tests) can be a useful (and most reliable) resource.
+
+[^options]:  Only for plugins that start when `lightningd` starts, option
+    values are not remembered when a plugin is stopped or killed.
+
 ### Warning
 
 As noted, `lightningd` uses `stdin` as an intake mechanism.  This can
@@ -47,7 +56,8 @@ plugin dirs, usually `/usr/local/libexec/c-lightning/plugins` and
 lightningd --plugin=/path/to/plugin1 --plugin=path/to/plugin2
 ```
 
-`lightningd` will run your plugins from the `--lightning-dir`/networkname, then
+`lightningd` will run your plugins from the `--lightning-dir`/networkname
+as working directory and env variables "LIGHTNINGD_PLUGIN" and "LIGHTNINGD_VERSION" set, then
 will write JSON-RPC requests to the plugin's `stdin` and
 will read replies from its `stdout`. To initialize the plugin two RPC
 methods are required:
@@ -67,6 +77,13 @@ through incoming JSON-RPC commands that were registered and the plugin
 may interact with `lightningd` using the JSON-RPC over Unix-Socket
 interface.
 
+Above is generally valid for plugins that start when `lightningd` starts.
+For dynamic plugins that start via the [plugin][lightning-plugin] JSON-RPC command there
+is some difference, mainly in options passthrough (see note in [Types of Options](#types-of-options)).
+
+ - `shutdown` (optional): if subscribed to "shutdown" notification, a plugin can
+   exit cleanly when `lightningd` is shutting down or when stopped via `plugin stop`.
+
 ### The `getmanifest` method
 
 The `getmanifest` method is required for all plugins and will be
@@ -123,8 +140,8 @@ example:
 }
 ```
 
-The `options` will be added to the list of command line options that
+During startup the `options` will be added to the list of command line options that
-`lightningd` accepts. The above will add a `--greeting` option with a
+`lightningd` accepts. If any `options` "name" is already taken startup will abort. The above will add a `--greeting` option with a
 default value of `World` and the specified description. *Notice that
 currently string, integers, bool, and flag options are supported.*
 
@@ -142,8 +159,8 @@ you plan on removing them: this will disable them if the user sets
 right?).
 
 The `dynamic` indicates if the plugin can be managed after `lightningd`
-has been started. Critical plugins that should not be stopped should set it
+has been started using the [plugin][lightning-plugin] JSON-RPC command. Critical plugins that should not be stopped should set it
-to false.
+to false. Plugin `options` can be passed to dynamic plugins as argument to the `plugin` command .
 
 If a `disable` member exists, the plugin will be disabled and the contents
 of this member is the reason why.  This allows plugins to disable themselves
@@ -175,7 +192,7 @@ Plugins are free to register any `name` for their `rpcmethod` as long
 as the name was not previously registered. This includes both built-in
 methods, such as `help` and `getinfo`, as well as methods registered
 by other plugins. If there is a conflict then `lightningd` will report
-an error and exit.
+an error and kill the plugin, this aborts startup if the plugin is *important*.
 
 #### Types of Options
 
@@ -230,6 +247,12 @@ Here's an example option set, as sent in response to `getmanifest`
   ],
 ```
 
+**Note**: `lightningd` command line options are only parsed during startup and their
+values are not remembered when the plugin is stopped or killed.
+For dynamic plugins started with `plugin start`, options can be
+passed as extra arguments to that [command][lightning-plugin].
+
+
 #### Custom notifications
 
 The plugins may emit custom notifications for topics they have
@@ -328,6 +351,12 @@ of this member is the reason why.
 The `startup` field allows a plugin to detect if it was started at
 `lightningd` startup (true), or at runtime (false).
 
+### Timeouts
+During startup ("startup" is true), the plugin has 60 seconds to
+return `getmanifest` and another 60 seconds to return `init`, or gets killed.
+When started dynamically via the [plugin][lightning-plugin] JSON-RPC command, both `getmanifest`
+and `init` should be completed within 60 seconds.
+
 ## JSON-RPC passthrough
 
 Plugins may register their own JSON-RPC methods that are exposed
@@ -1488,7 +1517,7 @@ handled the remaining plugins will be skipped.
 ### `rpc_command`
 
 The `rpc_command` hook allows a plugin to take over any RPC command. It sends
-the received JSON-RPC request to the registered plugin,
+the received JSON-RPC request (for any method!) to the registered plugin,
 
 ```json
 {
@@ -1716,3 +1745,5 @@ The plugin must broadcast it and respond with the following fields:
 [oddok]: https://github.com/lightningnetwork/lightning-rfc/blob/master/00-introduction.md#its-ok-to-be-odd
 [spec]: [https://github.com/lightningnetwork/lightning-rfc]
 [bolt9]: https://github.com/lightningnetwork/lightning-rfc/blob/master/09-features.md
+[lightning-plugin]: lightning-plugin.7.md
+[pyln-client]: ../contrib/pyln-client

doc/lightning-listconfigs.7

@@ -7,7 +7,10 @@ lightning-listconfigs - Command to list all configuration options\.
 
 .SH DESCRIPTION
 
-The \fBlistconfigs\fR RPC command to list all configuration options, or with \fIconfig\fR, just that one\.
+\fIconfig\fR (optional) is a configuration option name, or "plugin" to show plugin options
+
+
+The \fBlistconfigs\fR RPC command to list all configuration options, or with \fIconfig\fR only a selection\.
 
 
 The returned values reflect the current configuration, including
@@ -283,4 +286,4 @@ Vincenzo Palazzo \fI<vincenzo.palazzo@protonmail.com\fR> wrote the initial versi
 
 Main web site: \fIhttps://github.com/ElementsProject/lightning\fR
 
-\" SHA256STAMP:c55d5a93c5917bbab0be1633f0f9f06196cfdde6b79434ef012e9dfc2fbbcca9
+\" SHA256STAMP:67e5f100671ed8ef0e490caa46d57dc73e7443caa86a85f32806a0e8183cd1b8

doc/lightning-listconfigs.7.md

@@ -9,7 +9,9 @@ SYNOPSIS
 DESCRIPTION
 -----------
 
-The **listconfigs** RPC command to list all configuration options, or with *config*, just that one.
+*config* (optional) is a configuration option name, or "plugin" to show plugin options
+
+The **listconfigs** RPC command to list all configuration options, or with *config* only a selection.
 
 The returned values reflect the current configuration, including
 showing default values (`dev-` options are not shown).

doc/lightning-plugin.7.md

@@ -4,35 +4,46 @@ lightning-plugin -- Manage plugins with RPC
 SYNOPSIS
 --------
 
-**plugin** command [*parameter*] [*second\_parameter*]
+**plugin** *subcommand* [plugin|directory] [*options*] ...
+
 
 DESCRIPTION
 -----------
 
-The **plugin** RPC command allows to manage plugins without having to
+The **plugin** RPC command command can be used to control dynamic plugins,
-restart lightningd. It takes 1 to 3 parameters: a command
+i.e. plugins that declared themself "dynamic" (in getmanifest).
-(start/stop/startdir/rescan/list) which describes the action to take and
-optionally one or two parameters which describes the plugin on which the
-action has to be taken.
 
-The *start* command takes a path as the first parameter and will load
+*subcommand* can be **start**, **stop**, **startdir**, **rescan** or **list** and
-the plugin available from this path. The path can be a full path to a
+determines what action is taken
-plugin or a relative path to a plugin that is located in or below the
-default plugins directory. Any additional parameters are passed to the
-plugin. It will wait for the plugin to complete the handshake with
-`lightningd` for 20 seconds at the most.
 
-The *stop* command takes a plugin name as parameter. It will kill and
+*plugin* is the *path* or *name* of a plugin executable to start or stop
-unload the specified plugin.
 
-The *startdir* command takes a directory path as first parameter and will
+*directory* is the *path* of a directory containing plugins
-load all plugins this directory contains. It will wait for each plugin to
-complete the handshake with `lightningd` for 20 seconds at the most.
 
-The *rescan* command starts all not-already-loaded plugins from the
+*options* are optional *keyword=value* options passed to plugin, can be repeated
-default plugins directory (by default *~/.lightning/plugins*).
 
-The *list* command will return all the active plugins.
+*subcommand* **start** takes a *path* to an executable as argument and starts it as plugin.
+*path* may be an absolute path or a path relative to the plugins directory (default *~/.lightning/plugins*).
+If the plugin is already running and the executable (checksum) has changed, the plugin is
+killed and restarted except if its an important (or builtin) plugin.
+If the plugin doesn't complete the "getmanifest" and "init" handshakes within 60 seconds,
+the command will timeout and kill the plugin.
+Additional *options* may be passed to the plugin, but requires all parameters to
+be passed as keyword=value pairs, for example:
+ `lightning-cli -k plugin subcommand=start plugin=helloworld.py greeting='A crazy'`
+(using the `-k|--keyword` option is recommended)
+
+*subcommand* **stop** takes a plugin executable *path* or *name* as argument and stops the plugin.
+If the plugin subscribed to "shutdown", it may take up to 30 seconds before this
+command returns. If the plugin is important and dynamic, this will shutdown `lightningd`.
+
+*subcommand* **startdir** starts all executables it can find in *directory* (excl. subdirectories)
+as plugins. Checksum and timeout behavior as in **start** applies.
+
+*subcommand* **rescan** starts all plugins in the default plugins directory (default *~/.lightning/plugins*)
+that are not already running. Checksum and timeout behavior as in **start** applies.
+
+*subcommand* **list** lists all running plugins (incl. non-dynamic)
 
 RETURN VALUE
 ------------
@@ -44,7 +55,7 @@ On success, an object is returned, containing:
 If **command** is "start", "startdir", "rescan" or "list":
   - **plugins** (array of objects):
     - **name** (string): full pathname of the plugin
-    - **active** (boolean): status; since plugins are configured asynchronously, a freshly started plugin may not appear immediately.
+    - **active** (boolean): status; plugin completed init and is operational, plugins are configured asynchronously.
 
 If **command** is "stop":
   - **result** (string): A message saying it successfully stopped
@@ -54,6 +65,10 @@ If **command** is "stop":
 On error, the reason why the action could not be taken upon the
 plugin is returned.
 
+SEE ALSO
+--------
+lightning-cli(1), lightning-listconfigs(1), [writing plugins][writing plugins]
+
 AUTHOR
 ------
 
@@ -64,4 +79,5 @@ RESOURCES
 
 Main web site: <https://github.com/ElementsProject/lightning>
 
-[comment]: # ( SHA256STAMP:a07c71d232c39c0b959d07b9391d107413841753b67443d5f3698e1afd9cd2e4)
+[writing plugins]: PLUGINS.md
+[comment]: # ( SHA256STAMP:ee9c974be30d7870cb6a7785956548700b95d2d6b3fe4f18f7757afdfa015553)

doc/schemas/plugin.schema.json

@@ -57,7 +57,7 @@
                 },
                 "active": {
                   "type": "boolean",
-                  "description": "status; since plugins are configured asynchronously, a freshly started plugin may not appear immediately."
+                  "description": "status; plugin completed init and is operational, plugins are configured asynchronously."
                 }
               }
             }