Without this the RPC will fail to continue buffering if the response does not
fit in the first read, and if we don't switch over to the non-compat
mode. This was introduced by our mitigation of the UTF-8 misalignment, but I
missed this path.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
These weren't checked by CI yet, and they are really short so I just added
them to the check-python target.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This indicates that the method or hook will accepts a request
parameter, and will use that to return the result or raise an
exception instead of returning the return value. This allows the hook
or method to stash the incomplete request or pass it around, without
blocking the JSON-RPC interface.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This isn't a problem for now since we don't support multithreading,
and only allow synchronous calls, but eventually this'll become
important.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
We well need this in the next commit to be able to return from an
asynchronous call. We also guard stdout access with a reentrant lock
since we are no longer guaranteed that all communication happens on
the same thread.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Sending around unnamed tuples is bound to cause some issues sooner or
later, so we just create a quick class that holds all the information
about a plugin method.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Little point having users handle the postfixes manually, this
translates them, and also allows Millisatoshi to be used wherever an
'int' would be previously.
There are also helpers to create the formatting in a way c-lightning's
JSONRPC will accept.
All standard arithmetic operations with integers work.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We read a JSON message from the buffer, after converting it from raw bytes to
UTF-8, and returning the remainder of the byte array back to the
caller. However the return value of `raw_decode` refers to symbols in the
UTF-8 decoded string, not the raw bytes underlying byte-array, which means
that if we have multi-byte encoded UTF-8 symbols in the byte-array we end up
with a misaligned offset and will return part of the message as
remainder. This would then end up being interpreted as the result of the next
call.
This could not be exploited currently since we use a socket only for a single
JSON-RPC call and will close the connection afterwards, but since we want to
eventually recycle connections for multiple calls, this could have been very
dangerous.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Corné Plooy <@bitonic-cjp>
The next patch wants to decorate the methods with a compulsory
'usage' option, which doesn't make sense for init. So I wanted
to change the init to its own decoration.
Made-to-work-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Logging an empty line (without newline character) would raise an
Exception due to out of bounds check.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Instead of creating a new map I opted to re-use the Plugin.methods
map, since the semantics are really similar and we don't allow
duplicates. The only difference is in how they are announced to
lightningd, so we use an enum to differentiate rpcmethods from hooks,
since only the former will get added to the JSON-RPC dispatch table in
lightningd.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
After this code change people can use `plugin.rpc` from anywhere in
their plugin code this is much nicer than going this way:
```
@plugin.method("init")
def init(options, configuration, plugin):
global rpc
basedir = plugin.lightning_dir
rpc_filename = plugin.rpc_filename
path = os.path.join(basedir, rpc_filename)
rpc = LightningRpc(path)
```
or similarly that way:
```
@plugin.method("init")
def init(options, configuration, plugin):
global rpc
basedir = configuration['lightning-dir']
rpc_filename = configuration['rpc-file']
path = os.path.join(basedir, rpc_filename)
rpc = LightningRpc(path)
```
Also the imports have been sorted alphabetically
Co-authored-by: Rene Pickhardt <rene@rene-pickhardt.de>
Co-authored-by: Christian Decker <decker.christian@gmail.com>
If the `request` or `plugin` parameter that are injected by the
framework where before or inbetween positional arguments we'd be
injecting them incorrectly, i.e., we'd be providing them both as
`args` (and mismapping another argument) as well as `kwargs`.
This is a better way to map arguments, which takes advantage of the
fact that JSON-RPC calls are either all positional or named arguments.
I also included a test for various scenarios, that hopefull cover the
most common cases.
Reported-by: Rene Pickhardt <@renepickhardt>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Just like we added the RPC methods, the notification handlers can also
be registered using a function decorator, and we auto-subscribe when
asked for a manifest.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This was causing `listchannels` to be incredibly slow. The response is
several megabyte in size, and we were only buffering 1Kb on each
iteration.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
It's flask inspired with the Plugin instance and decorators to add
methods to the plugin description.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
We inadvertently broke the compatibility between the python library
and the binary when switching to \n\n-delimiters. This reintroduces
the old inefficient parsing, and dynamically upgrades to the faster
version if it detects the \n\n-delimiter.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This doesn't make a performance difference, but even better, it
simplifies the code.
We hacked test_multirpc to send 200x as many commands, and timed the
pytest over 20 runs:
Before:
=================== 1 passed, 136 deselected in 8.550000-9.400000(9.0045+/-0.2) seconds ===================
After:
=================== 1 passed, 136 deselected in 8.540000-9.370000(8.97286+/-0.16) seconds ===================
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need to keep the remaining buffer, and we need to try to parse it
before we read the next. I first tried keeping it in the object, but
its lifetime is that of the *socket*, which we actually reopen for
every command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
