This was introduced to allow creating a shared secret, but it's better to use
makesecret which creates unique secrets. getsharedsecret being a generic ECDH
function allows the caller to initiate conversations as if it was us; this
is generally OK, since we don't allow untrusted API access, but the commando
plugin had to blacklist this for read-only runes explicitly.
Since @ZmnSCPxj never ended up using this after introducing it, simply
remove it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSONRPC: `getsharedsecret` API: use `makesecret`
routing.c now flags rate-limited gossip as it enters the gossip_store but
makes use of it in updating the routing graph. Flagged gossip is not
rebroadcast to gossip peers.
Changelog-Changed: gossipd: now accepts spam gossip, but squelches it for
peers.
This will be used to decouple internal use of gossip from what is
passed to gossip peers. Updates GOSSIP_STORE_VERION to 10.
Changelog-Changed: gossip_store updated to version 10.
This will eventually enable us to specify 0 for zeroconf channels.
Changelog-Added: JSON-RPC: Added `mindepth` argument to specify the number of confirmations we require for `fundchannel` and `multifundchannel`
Over time, it has cost us more developer cycles than it has gained.
It has hidden intermittant bugs, and allowed cruft to accumulate:
when we eventually tried to figure out what was going wrong, the
actual change which caused it was now stale and forgotten.
This was a particular bane during the connectd rewrite, and I
worked through some issues which had occurred before, but were not
more likely.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
So this was quite a journey:
- We want relative depdendencies (using the `path` argument) whenever
developing locally. Otherwise we would have to install each
dependency every time we change a single character, which
undoubtedly would cause us to waste time trying to debug an issue
just because we forgot to install.
- When publishing however we want to rely on the version number,
since the repo context gets lost upon publishing, and path
dependencies cause failures.
The solution then it seems is to use `dev-dependencies` (not that
surprising once you find it) with relative paths, so that `poetry
install` uses these over the normal dependencies (no idea how they
dedup them) and use `dependencies` when publishing. The paths are
still in there when publishing, but `pip install` ignores them.
I checked that `poetry install` from an unrelated project doesn't
accidentally use the path dependencies, even when adding them as
dev-dependencies. This should hopefully also allow installing them
as a repo link, though I can't test that right now.
This prepares for when they start being u64, not strings with msat appended.
This has a strange side effect on our schema: despite the name,
decodepay's `fee_base_msat` is actually a u64, which we now convert to
msat on decode.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is consistent with our output changes, and increases consistency.
It also keeps future sanity checks happy, that we only use JSON msat
helpers with '_msat' fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `invoice`, `sendonion`, `sendpay`, `pay`, `keysend`, `fetchinvoice`, `sendinvoice`: `msatoshi` argument is now called `amount_msat` to match other fields.
Changelog-Deprecated: JSON-RPC: `invoice`, `sendonion`, `sendpay`, `pay`, `keysend`, `fetchinvoice`, `sendinvoice` `msatoshi` (use `amount_msat`)
Mostly comments and docs: some places are actually paths, which
I have avoided changing. We may migrate them slowly, particularly
when they're user-visible.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `pay` has `description` parameter, will be required if bolt11 only has a hash.
Changelog-Deprecated: JSON-RPC: `pay` for a bolt11 which uses a `description_hash`, without setting `description`.
This is what LND does, and it's better for upper layers than trying to
twist our maxfeepercent / exemptfee heuristics to suit.
(I don't remember who complained about this, sorry!)
I'm doing this now because I want to add YA parameter next!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `pay` has new parameter `maxfee` for setting absolute fee (instead of using `maxfeepercent` and/or `exemptfee`)
Make it always a number; this makes the JSON request specification
simpler. We allowed a number since v0.10.1.
(reserve=True is the default anyway, so usually it can be omitted:
reserve=False becomes reserve=0).
Changelog-Deprecated: JSON-RPC: `fundpsbt`/`utxopsbt` `reserve` must be a number, not bool (for `true` use 72/don't specify, for `false` use 0). Numbers have been allowed since v0.10.1.
Means that field is now optional in JSON output.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `delinvoice` has a new parameter `desconly` to remove description.
LNURL wants this so they can include images etc in descriptions.
Replaces: #4892
Changelog-Added: JSON-RPC: `invoice` has a new parameter `deschashonly` to put hash of description in bolt11.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Turns out that the pyln-proto dependency in the bolt packages is only
needed for testing, not for production. Making it a dev-dependency
means it isn't considered in resolution anymore.
Since the bolt, testing and client packages are to be used outside
from the project we can't use relative dependencies either, so make
then dependent on the version on PyPI. This also means we had to push
a couple of updated to PyPI.
Changelog-None
Suggested by @m-schmook, I realized that if we append it later I'll
never get it right: I expect parameters min and max, not max and min!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: you can now alter the `htlc_minimum_msat` and `htlc_maximum_msat` your node advertizes.
Based on setchannelfee, but expanded to allow setting max htlc amount (and others
in future?).
The main differences:
1. It doesn't change values which are not specified (that would be hard to
add fields to!)
2. It says exactly what all values are in any potentially changed channels.
Changelog-Added: JSON-RPC: new `setchannel` command generalizes `setchannelfee`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We must use `None` for default arguments since otherwise they aren't
filtered out when serializing the request. In addition default
arguments to functions are initialized once and, if mutable, could
persist internal changes across function calls.
Changelog-None
This was measured as a 95th percentile in our rough testing, thanks to
all the volunteers who monitored my channels.
Fixes: #4761
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `setchannelfee` gives a grace period (`enforcedelay`) before rejecting old-fee payments: default 10 minutes.
This affects the range we offer even without quick-close, but it's
more critical for quick-close.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSONRPC: `close` now takes a `feerange` parameter to set min/max fee rates for mutual close.
This only happens when a deletion is added by a running gossipd, so
we put a deletion at the end of the store to test it.
mypy noticed that this code was nonsensical, so clearly untested.
The testing noticed that making a nodeid from a string was also buggy.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Mainly fixing type annotations, but some real fixes:
1. GossmapHalfchannel.from_str() should be a classmethod.
2. update_channel had weird, unusable default values (fields can't be NULL,
since we use it below).
[ There was one more occurence where isinstance should be used above
type() == xyz comparison. -- MS ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Do not mix bytes and GossmapNodeId when accessing Gossmap.nodes dicts.
Therefore the definion got GossmapNodeId also needed to be pulled to the
beginning of the file.
This reads the `gossip_store_channel_amount` that always follows the
`channel_announcement` messages. Therefore it uses an internal variable
_last_scid to know what channel has been added last time.
This is more efficient than converting them all to Pubkeys: about 3.8
seconds vs 5.4 seconds. Usually treating them as raw bytes is what we
want anyway.
[ Fixup by Michael Schmoock <michael@schmoock.net> ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It doesn't do much work, but it does parse the gossmap file and extract
nodes and channels.
[ Fixup by Michael Schmoock <michael@schmoock.net> ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
As suggested in this issue https://github.com/python/mypy/issues/7484#issuecomment-529363083 we skip following import becuase with the recent version of mypy the __init__.py file make confusione inside the analysis (in the python issue it is unclear the main motivation of this issue. At list unclear to me).
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
The issue is that the new keyword `force_lease_closed` was being set
even if the user didn't specify it, which results in breakage if this
new pyln version talks to older c-lightning nodes that don't have this
keyword yet. By setting the default to `None` it gets filtered out if
the user has not explicitly set it, but still retains the `True` /
`False` values if they did.
Changelog-None Issue is not present in released versions
We need to know what the lease we're expecting is. To do this
we pass around the hex encoded portion of the wire format.
We can use this passed in expected lease rates to confirm that the peer
is, in fact, using the same rates as what we have currently.
Changelog-Added: JSON-RPC: fundchannel, multifundchannel, and openchannel_init now accept a 'compact_lease' for any requested funds
By default, we won't close a channel that we leased to a peer.
You can override this with the `force_lease_closed` flag.
Changelog-Added: JSON-RPC: close now has parameter to force close a leased channel (option_will_fund)
Using a 'feestep' is more restrictive than you'd want, instead we
enforce that the next feerate must be at least 1/64th more than the
last, but put no upper limit on it
Includes update to lnprototest changes
Contributed-By: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-EXPERIMENTAL: Protocol: Replaces init_rbf's `fee_step` for RBF of v2 opens with `funding_feerate_perkw`, breaking change
It sometimes fails because the two race, and sometimes because there's
randomness, but it generally works (and doesn't fail systemically).
We remove this before the final merge.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It sometimes fails because the two race, and sometimes because there's
randomness, but it generally works (and doesn't fail systemically).
We remove this before the final merge.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It sometimes fails because the two race, and sometimes because there's
randomness, but it generally works (and doesn't fail systemically).
We remove this before the final merge.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This adds our first (basic) schema, and sews support into pyln-testing
so it will load schemas for any method for doc/schemas/{method}.schema.json.
All JSON responses in a test run are checked against the schema (if any).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Not an API break: reserve=true|false still works for fundpsbt and utxopsbt,
but we also allow a raw number in there.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We will eventually start emitting and dispatching custom notifications
from plugins just like we dispatch internal notifications. In order to
get reasonable error messages we need to make sure that the topics
plugins are asking for were correctly registered. When doing this we
don't really care about whether the plugin that registered the
notification is still alive or not (it might have died, but
subscribers should stay up and running), so we keep a list of all
topics attached to the `struct plugins` which gathers global plugin
information.
Allows us to clean up an in-progress open that we won't be completing
Changelog-Added: EXPERIMENTAL JSON-RPC: Permit user-initiated aborting of in-progress opens. Only valid for not-yet-committed opens and RBF-attempts
The semantics don't change, since `lightningd` will use false as
default as well, however setting it to something other than `None`
causes the RPC library to include the parameter in the query, and
since the parameter was introduced only in 0.9.3 and pyln may be used
with older versions this then results in an error about an unknown
parameter.
Setting this to `None` makes sure pyln filters out the argument before
calling.
Changelog-Fixed: pyln: Fixed an error when calling `listfunds` with an older c-lightning version causing an error about an unknown `spent` parameter
I had way too much fun with this and got a bit carried away with the
letter writing. The idea is to be helpful when users start the plugin
from the command line, rather than run it under the control of
lightningd. We also print detailed information about the user-visible
things such as the methods and options exposed by the plugin.
Changelog-Added: pyln: Plugins that are run from the command line print helpful information on how to configure c-lightning to include them and print metadata about what RPC methods and options are exposed.
Suggested-by: Rusty Russell <@rustyrussell>
In the case where you want a PSBT and also want the output to be added
as a change address, use `excess_as_change` = true.
Generates a change address to use. If you want to pay the excess
elsewhere, you will have to add separately.
Changelog-Added: JSON-RPC: Add new parameter `excess_as_change` to fundpsbt+utxopsbt
We were getting a number of incompatibility warning due to the
dependencies being expressed too rigidly. This losens the requirement
definitions to being compatible with a known good version, and while
we're at it we also bump all outdated requirements.
This is required if we want to create a "bouncer" plugin (in my copious free time!)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `invoice` now takes an optional `cltv` parameter.
A fractional satoshi value isn't really useful; rounding up loses
precision but that's why you called "whole satoshi", wasn't it?
Changelog-Changed: pyln-client: Millisatoshi has new method, `to_whole_satoshi`; *rounds value up* to the nearest whole satoshi
We need to use it for the 'df_accepter' plugin, so we get the feerate
correct.
Changelog-Added: pyln-client: `fundpsbt`/`utxopsbt` now support `min_witness_weight` param
Allows caller to set code and exact message to be returned.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln-client: plugins can now raise RpcException for finer control over error returns.
"multi" means that specifying a parameter twice will append, not override.
Multi args are always given as a JSON array, even if only one.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: new "multi" field allows an option to be specified multiple times.
This only adds a test that currently makes sure its not possible
to init a Millisatoshi by a floating number.
Discussion:
As @gallizoltan points out, initialization with a float should be possible:
https://github.com/ElementsProject/lightning/pull/4273#discussion_r540369093
> Millisatoshi(5) / 2 currently works, and removes the half msat.
> So, I think Millisatoshi(5 / 2) should be the same.
Before this patch this fails: `Millisatoshi(42) / Millisatoshi(2)`
This is an operation that should return the ratio between the two
operands as a float number. Same goes for __floordiv__ operator `//`.
Changelog-None
We promised to deprecate the old "destination satoshi feerate utxos"
variant some time on mid-2020 so let's do just that. `txprepare`
doesn't parse it either anymore.
Changelog-Changed: pyln: `txprepare` no longer supports the deprecated `destination satoshi feerate utxos` call format.
We also sanity check that response id matches our request.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln: pyln.client handles and can send progress notifications.
`openchannel_signed` and `openchannel_update` which allow a user to
continue a openchannel or kick off the completion of a openchannel.
`openchannel_update` should be called until it returns with
`commitments_secured`.
Finally, extends the 'close_to' functionality up to the flagship 'open a
channel' command.
Changelog-Added: JSON-API `fundchannel` now accepts an optional 'close_to' param, a bitcoin address that the channel funding should be sent to on close. Requires `opt_upfront_shutdownscript`
We had a couple of instances where a plugin would be killed by `lightningd`
because we were returning a result of an exception twice, and it was hard to
trace down the logic error in the user plugin that caused that. This patch
adds a traceback the first time we return a result/exception, and raise an
exception with a stacktrace of the first termination when a second one comes
in.
This can still terminate the plugin, but the programmer gets a clear
indication where the result was set, and can potentially even recover from it.
Changelog-Added: pyln: Plugin method and hook requests prevent the plugin developer from accidentally setting the result multiple times, and will raise an exception detailing where the result was first set.
It is often pretty usefuk to use the builtin logging module to debug things,
including libraries that a plugin may use. This adds a simple
`PluginLogHandler` that maps the python logging levels to the `lightningd`
logging levels, and formats the record in a way that it doesn't clutter up the
`lightningd` logs (no duplicate timestamps and levels).
This allow us to tweak the log level that is reported to `lightningd` simply
using the following
```python3
import logging
logging.basicConfig(level=logging.DEBUG)
```
Notice that in order for the logs to be displayed on the terminal or the
logfile, both the logging level in the plugin _and_ the `--log-level`
`lightningd` is running need to be adjusted (the python logging level only
controls which messages get forwarded to `lightningd`, it does not have the
power to overrule `lightningd` about what to actually display).
I chose `logging.INFO` as the default, since libraries have a tendency to spew
out everything in `logging.DEBUG` mode
Changelog-Added: pyln: Plugins have been integrated with the `logging` module for easier debugging and error reporting.
Several times we had issues with plugins not being able to re-encode an RPC
result because they forgot to use the custom encoder class. This allows us to
patch the JSONEncoder when we start the RPC or the plugin and automagically
support classes that provide a `to_json` method.
Hooks do not tolerate failures at all. If we return a JSON-RPC error to a hook
call the only thing the main daemon can really do is to crash. This commit
adds a mapping of error to a safe fallback result, including a warning to the
node operator that this should be addressed in the plugin. The warning is
reported as a `**BROKEN**` message, and should therefore fail any testing done
on the plugin.
Changelog-Fixed: pyln: Fixed HTLCs hanging indefinitely if the hook function raises an exception. A safe fallback result is now returned instead.
Fixes: #2679
Changelog-Added: JSON-RPC: New `multiwithdraw` command to batch multiple onchain sends in a single transaction. Note it shuffles inputs and outputs, does not use BIP69.
This is an extra safety check for dual funding, where we only want to sign
the inputs we provided!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `signpsbt` takes an optional `signonly` array to limit what inputs to sign.
Required for dual funding where the opener sets it.
Changelog-Added: JSON-RPC: `fundpsbt` takes a new `locktime` parameter
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
fundpsbt forces the caller to manually add their weight * feerate
to the satoshis they ask for. That means no named feerates.
Instead, create a startweight parameter and do the calc for them
internally, and return the feerate we used (and, while we're at it,
the estimated final weight).
This API change is best done now, as it would otherwise have to
be appended as a parameter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: new call `signpsbt` which will add the wallet's signatures to a provided psbt
Changelog-Added: JSON-RPC: new call `sendpsbt` which will finalize and send a signed PSBT
Reserve and unreserve wallet UTXOs using a PSBT which includes those
inputs.
Note that currently we unreserve inputs everytime the node restarts.
This will be addressed in a future commit.
Changelog-Added: JSON-RPC: Adds two new rpc methods, `reserveinputs` and `unreserveinputs`, which allow for reserving or unreserving wallet UTXOs
I quickly ran through the comments in lightning.py and saw a few small inconsistencies:
- upper/lower case for the "B" in "Bitcoin" unified (see https://github.com/lnbook/lnbook/pull/98)
- added missing "." after a complete sentence
- removed unnecessary double spaces
