1. We need to read in as a byte string, then decode into utf8 once we
have a marker. Otherwise we seem to mangle it horribly, and we
might have a bad utf8 string anyway.
2. We need to suppress the JSON \u escapes on output.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rather than using LightningJSONDecoder's implicit "field name and
value ends in msat, try converting to Millisatoshi", we do it to
parameters using type annotations.
If you had a parameter which was an array or dict itself, we don't
delve into that, but that's probably OK.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I originally converted input JSON naively into Millisatoshi, and the
result was a strange failure in Millisatoshi.__eq__.
It seems this is because inspect._empty.__eq__(Millisatoshi) raises
NotImplemented, and so it tries Millisatoshi.__eq__(inspect._empty)
which doesn't like it.
'is' is the correct test here, AFAICT, and doesn't suffer from these
problems.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Some JSON functions want a *class*, not just a hook, so provide one.
To make it clear that we want an encoding *class* and a decoding *object*,
rename the UnixDomainSocketRpc encode parameter to encode_cls.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If we can't marshall an object into JSON, the exception causes a deadlock
and we don't get any results.
Instead of deadlocking, our failure now is:
lightning.lightning.RpcError: RPC call failed: method: echo, payload: {'msat': 17msat}, error: Error while processing echo: TypeError("Object of type 'Millisatoshi' is not JSON serializable",)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I tried annotating the plugin-millisatoshis.py plugin, and it failed like so:
plugin-millisatoshis.py Killing plugin: "getmanifest" result is not an object: {"jsonrpc": "2.0", "id": 1, "error": "Error while processing getmanifest: ValueError(\'Function has keyword-only parameters or annotations, use getfullargspec() API which can support them\',)"}'
So, let's do that!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is the same deprecation, but one level up. For the moment, we
still support invoices with a `h` field (where description will be
necessary) but that will be removed once this option is removed.
Note that I just changed pylightning without backwards compatibility,
since the field was unlikely to be used, but we could do something
more complex here?
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Without this the RPC will fail to continue buffering if the response does not
fit in the first read, and if we don't switch over to the non-compat
mode. This was introduced by our mitigation of the UTF-8 misalignment, but I
missed this path.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
These weren't checked by CI yet, and they are really short so I just added
them to the check-python target.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This indicates that the method or hook will accepts a request
parameter, and will use that to return the result or raise an
exception instead of returning the return value. This allows the hook
or method to stash the incomplete request or pass it around, without
blocking the JSON-RPC interface.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This isn't a problem for now since we don't support multithreading,
and only allow synchronous calls, but eventually this'll become
important.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
We well need this in the next commit to be able to return from an
asynchronous call. We also guard stdout access with a reentrant lock
since we are no longer guaranteed that all communication happens on
the same thread.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Sending around unnamed tuples is bound to cause some issues sooner or
later, so we just create a quick class that holds all the information
about a plugin method.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Little point having users handle the postfixes manually, this
translates them, and also allows Millisatoshi to be used wherever an
'int' would be previously.
There are also helpers to create the formatting in a way c-lightning's
JSONRPC will accept.
All standard arithmetic operations with integers work.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We read a JSON message from the buffer, after converting it from raw bytes to
UTF-8, and returning the remainder of the byte array back to the
caller. However the return value of `raw_decode` refers to symbols in the
UTF-8 decoded string, not the raw bytes underlying byte-array, which means
that if we have multi-byte encoded UTF-8 symbols in the byte-array we end up
with a misaligned offset and will return part of the message as
remainder. This would then end up being interpreted as the result of the next
call.
This could not be exploited currently since we use a socket only for a single
JSON-RPC call and will close the connection afterwards, but since we want to
eventually recycle connections for multiple calls, this could have been very
dangerous.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Corné Plooy <@bitonic-cjp>
The next patch wants to decorate the methods with a compulsory
'usage' option, which doesn't make sense for init. So I wanted
to change the init to its own decoration.
Made-to-work-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Logging an empty line (without newline character) would raise an
Exception due to out of bounds check.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Instead of creating a new map I opted to re-use the Plugin.methods
map, since the semantics are really similar and we don't allow
duplicates. The only difference is in how they are announced to
lightningd, so we use an enum to differentiate rpcmethods from hooks,
since only the former will get added to the JSON-RPC dispatch table in
lightningd.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
After this code change people can use `plugin.rpc` from anywhere in
their plugin code this is much nicer than going this way:
```
@plugin.method("init")
def init(options, configuration, plugin):
global rpc
basedir = plugin.lightning_dir
rpc_filename = plugin.rpc_filename
path = os.path.join(basedir, rpc_filename)
rpc = LightningRpc(path)
```
or similarly that way:
```
@plugin.method("init")
def init(options, configuration, plugin):
global rpc
basedir = configuration['lightning-dir']
rpc_filename = configuration['rpc-file']
path = os.path.join(basedir, rpc_filename)
rpc = LightningRpc(path)
```
Also the imports have been sorted alphabetically
Co-authored-by: Rene Pickhardt <rene@rene-pickhardt.de>
Co-authored-by: Christian Decker <decker.christian@gmail.com>
If the `request` or `plugin` parameter that are injected by the
framework where before or inbetween positional arguments we'd be
injecting them incorrectly, i.e., we'd be providing them both as
`args` (and mismapping another argument) as well as `kwargs`.
This is a better way to map arguments, which takes advantage of the
fact that JSON-RPC calls are either all positional or named arguments.
I also included a test for various scenarios, that hopefull cover the
most common cases.
Reported-by: Rene Pickhardt <@renepickhardt>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Just like we added the RPC methods, the notification handlers can also
be registered using a function decorator, and we auto-subscribe when
asked for a manifest.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This was causing `listchannels` to be incredibly slow. The response is
several megabyte in size, and we were only buffering 1Kb on each
iteration.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
It's flask inspired with the Plugin instance and decorators to add
methods to the plugin description.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
We inadvertently broke the compatibility between the python library
and the binary when switching to \n\n-delimiters. This reintroduces
the old inefficient parsing, and dynamically upgrades to the faster
version if it detects the \n\n-delimiter.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This doesn't make a performance difference, but even better, it
simplifies the code.
We hacked test_multirpc to send 200x as many commands, and timed the
pytest over 20 runs:
Before:
=================== 1 passed, 136 deselected in 8.550000-9.400000(9.0045+/-0.2) seconds ===================
After:
=================== 1 passed, 136 deselected in 8.540000-9.370000(8.97286+/-0.16) seconds ===================
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need to keep the remaining buffer, and we need to try to parse it
before we read the next. I first tried keeping it in the object, but
its lifetime is that of the *socket*, which we actually reopen for
every command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And there's a difference between no description and "" as a description:
for no description, listpayments doesn't show the field at all. So fix
that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And, reluctantly, default to bitcoind style.
"It's wrong to be right too soon."
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
No semantical change but when using the python lib for the rpc-api it is confusing that my developing environment suggests that I should fund a channel and pass a channel_id when in fact I want to pass a node_id
This is useful mainly in the case where bitcoind is not giving estimates,
but can also be used to bias results if you want.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Manipulate fees via fake-bitcoin-cli. It's not quite the same, as
these are pre-smoothing, so we need a restart to override that where
we really need an exact change. Or we can wait until it reaches a
certain value in cases we don't care about exact amounts.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
They're much more useful being programatically-accessible, AFAICT.
The string stays the same so they're backwards compatible.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
`getinfo` has been providing the blockheight for a good while and doesn't
require the `DEVELOPER=1` flag during compilation, so it should be the preferred
method to retrieve the blockchain height.
Also report tx and txid, and whether we closed unilaterally or
bilaterally, if we could close the channel.
Also make a manpage.
Fixes: #1207Fixes: #714Fixes: #622
We can have more than one; eg we might offer both bech32 and a p2sh
address, and in future we might offer v1 segwit, etc.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
* Modifies invoice command to have the following format
invoice <msatoshi> <label> <desc> <?expiry> <?fallbackaddr>
* Adds support for Segwit bcrt1 addresses for withdraw
* Add test case for fallback address in invoice creation
* Create a common json_tok_address_scriptpubkey to be used
by invoice and withdraw commands.
* Fix dev_setfees to set slow and normal fees correctly.
Due to a bug def_setfees(100, slow=100) would instead set immediate and
normal fees to 100. This behavior has been updated to set fees to
correct values, make the values truly optional as per documentation and
unit test this behavior.
* Fix pay() to set msatoshi, description and risk factor properly
Due to a bug pay(invoice, description='1000') resulted in payment of
1000 msatoshi instead. This was fixed and covered with tests.
* Fix named args in listpayments, listpeers and connect
* Do not pass None to methods where it is default value
* Make description on invoice and pay match.
Suggested-by: @ZmnSCPxj
* Fix dev_setfees to set slow and normal fees correctly.
Due to a bug def_setfees(100, slow=100) would instead set immediate and
normal fees to 100. This behavior has been updated to set fees to
correct values, make the values truly optional as per documentation and
unit test this behavior.
* Fix pay() to set msatoshi, description and risk factor properly
Due to a bug pay(invoice, description='1000') resulted in payment of
1000 msatoshi instead. This was fixed and covered with tests.
* Fix named args in listpayments, listpeers and connect
* Do not pass None to methods where it is default value
* Make description on invoice and pay match.
Suggested-by: @ZmnSCPxj
'rhash' is the old terminology, but 'payment_preimage' and
'payment_hash' were decided on for the BOLTs, so we should fix that here.
We still use rhash internally, but that's much easier to fix.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
