When core lightning is asking the information about
the blockchain with `getchaininfo` command lightningd
know already the information about the min and max block height.
the problem is when we have a smarter Bitcoin backend that is able
to switch between different clients in some cases is helpful
give the information about current known height by lightningd and
pass it down to the plugin.
In this way, the plugin knows what is the correct known height from lightnind, and can
try to fix some problems if any exit.
This is particularly useful when you are syncing a new backend from scratch
like https://github.com/cloudhead/nakamoto and we avoid returning the
lower height from the known, and avoid the crash of core lightning.
With this information, the plugin can start to sync the chain and return
the answer back only when the chain is in sync with the current status of
lightningd.
Another reason to add this field and not wait the correct block in core
lightning itself is because Bitcoin Core is extremely slow to sync up,
so the question here is, how long should we wait? The time depends
on various factors.
With this approach of informing the plugin about the height, in some cases,
you can start the syncing but move the execution to another backend until
the previous one is ready.
The problem I want to solve is that I don't want to be left in the dark when
we run `getchaininfo`, and I want to have the opportunity to wait for
the blockchain sync or decide to dispatch the request elsewhere.
Changelog-Added: Pass the current known block height down to the getchaininfo call.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Previously any attempt would result in "is not an integer field"; we
now recognize valid JSON integers as integer fields.
Changelog-Fixed: Plugins: `commando` runes can now compare integer parameters using '<' and '>' as expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `listconfigs` direct fields, use `configs` sub-object and `set`, `value_bool`, `value_str`, `value_int`, or `value_msat` fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This integrates them with configvars properly: they almost "just work"
in listconfigs now, and we don't put them in a special sub-object
under their plugin.
Unfortunately, this means `listconfigs` now has a loose schema: any
plugin can add something to it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: reloaded plugins get passed any vars from configuration files.
Changelog-Deprecated: Config: boolean plugin options set to `1` or `0` (use `true` and `false` like non-plugin options).
We use multi-specifiable options elsewhere, this is just another.
Otherwise you can't add, you can only set them all.
Changelog-Added: Config: `accept-htlc-tlv-type` (replaces awkward-to-use `accept-htlc-tlv-types`)
Changelog-Deprecated: Config: `accept-htlc-tlv-types` (use `accept-htlc-tlv-type` multiple times)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
listconfigs is convenient, but it doesn't handle multi-options well: it
outputs an object with duplicate fields in this case (e.g. log-file), nor
is it extensible to show more than raw values.
However, listconfigs doesn't do what other list commands do (use a
sub-object "configs") so we can put the new values under that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listconfigs` now has `configs` subobject with more information about each config option.
Now we wire in the code which gathers configvars and parses from there;
lightningd keeps the array of configuration variables for future use.
Note that lightning-cli also needs to read the config, but it has its
own options (including short ones!) and doesn't want to use this
configvar mechanism, so we have a different API for that now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This adds:
1. ability to search for an option by name.
2. allowance to set our own bits when registering options.
3. show callbacks which can say "don't show", and variable length.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
These are only likely to confuse users, by silently changing behavior.
Changelog-Deprecated: Config: bind-addr=xxx.onion and addr=xxx.onion, use announce=xxx.onion (which was always equivalent).
Changelog-Deprecated: Config: addr=/socketpath, use listen=/socketpath (which was always equivalent).
This obsoletes the use of --announce-addr-dns which I know Michael
didn't really like either.
Changelog-Deprecated: Config: `announce-addr-dns`; use `--bind-addr=dns:ADDR` for finer control.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I never really liked this hack: websockets are useful, advertizing
them not so much.
Note that we never actually documented that we would advertize these!
Changelog-EXPERIMENTAL: Protocol: Removed support for advertizing websocket addresses in gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
CI hit this issue where it would get a tx_abort in fundchannel. This
happens when the dualopend we use to query the feerates has not exited
yet (it waits for the tx_abort reply), and we mistakenly reuse it.
With multi-channel support, this is wrong: just run another one and it
all Just Works.
This means we need to rework our dual_open_control.c logic, since it
would previously create an unsaved channel then not clean up if
something went wrong.
Most people will never try to negotiate opening multiple channels to
the same peer at the same time (vs. having an established channel and
opening a new one), so this case is a bit weird.
```
rates = l1.rpc.dev_queryrates(l2.info['id'], amount, amount)
# l1 leases a channel from l2
l1.rpc.fundchannel(l2.info['id'], amount, request_amt=amount,
feerate='{}perkw'.format(feerate),
> compact_lease=rates['compact_lease'])
tests/test_opening.py:1611:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-client/pyln/client/lightning.py:833: in fundchannel
return self.call("fundchannel", payload)
contrib/pyln-testing/pyln/testing/utils.py:721: in call
res = LightningRpc.call(self, method, payload, cmdprefix, filter)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pyln.testing.utils.PrettyPrintingLightningRpc object at 0x7f6cbcd97950>
method = 'fundchannel'
payload = {'amount': 500000, 'announce': True, 'compact_lease': '029a00640064000000644c4b40', 'feerate': '2000perkw', ...}
cmdprefix = None, filter = None
def call(self, method, payload=None, cmdprefix=None, filter=None):
"""Generic call API: you can set cmdprefix here, or set self.cmdprefix
...
if not isinstance(resp, dict):
raise ValueError("Malformed response, response is not a dictionary %s." % resp)
elif "error" in resp:
> raise RpcError(method, payload, resp['error'])
E pyln.client.lightning.RpcError: RPC call failed: method: fundchannel, payload: {'id': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 'amount': 500000, 'feerate': '2000perkw', 'announce': True, 'request_amt': 500000, 'compact_lease': '029a00640064000000644c4b40'}, error: {'code': -1, 'message': 'Abort requested', 'data': {'id': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 'method': 'openchannel_init'}}
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Failure under CI:
```
> bitcoind.generate_block(1000)
tests/test_closing.py:853:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-testing/pyln/testing/utils.py:496: in generate_block
return self.rpc.generatetoaddress(numblocks, to_addr)
contrib/pyln-testing/pyln/testing/utils.py:374: in f
res = proxy._call(name, *args)
../../../.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.7/lib/python3.7/site-packages/bitcoin/rpc.py:246: in _call
response = self._get_response()
../../../.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.7/lib/python3.7/site-packages/bitcoin/rpc.py:276: in _get_response
http_response = self.__conn.getresponse()
/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/http/client.py:1373: in getresponse
response.begin()
/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/http/client.py:319: in begin
version, status, reason = self._read_status()
/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/http/client.py:280: in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <socket.SocketIO object at 0x7fa21aa5d710>
b = <memory at 0x7fa21b771390>
def readinto(self, b):
"""Read up to len(b) bytes into the writable buffer *b* and return
the number of bytes read. If the socket is non-blocking and no bytes
are available, None is returned.
If *b* is non-empty, a 0 return value indicates that the connection
was shutdown at the other end.
"""
self._checkClosed()
self._checkReadable()
if self._timeout_occurred:
raise OSError("cannot read from timed out object")
while True:
try:
> return self._sock.recv_into(b)
E socket.timeout: timed out
/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/socket.py:589: timeout
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
07413c20b9 et al reworked how onchaind
does broadcasts, meaning tests needed to be updated to the new helpers
rather than searching logs themselves.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were passing a max_output_len that was too small, so every call to
bech32_encode was failing. Now we set max_output_len to the full size of
bech32_str.
This currently means anchors tests are disabled, awaiting the
PR which implements zero-fee-htlc anchors to reenable them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And no longer insist on opt_quiesce.
Changelog-EXPERIMENTAL: Config: `--experimental-upgrade-protocol` enables simple channel upgrades.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need to check that the key is valid for two reasons:
1) towire_ext_key() aborts if the key is invalid
2) fromwire_ext_key() doesn't check the parsed key for validity
Since bip32_key_get_fingerprint() fails if the key is invalid, we can
call it first to guarantee the key is valid before serializing.
When we release too fast, the plugin crashes:
```
thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: SendError(())', plugins/examples/cln-plugin-reentrant.rs:31:27
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
```
This happens with CI under VALGRIND!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Avoids failing the test with the pip warning:
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
reported by: @ksedgwic
Changelog-None
This was pointed out by Daywalker [1]: we are synchronously processing
events from `lightningd` which means that if processing one of the
hooks or requests was slow or delayed, we would not get notifications,
hooks, or RPC requests, massively impacting the flexbility.
This highlights the issue with a failing test (it times out), and in
the next commit we will isolate event processing into their own task,
so to free the event loop from having to wait for an eventual
response.
[1] https://community.corelightning.org/c/developers/hold-invoice-plugin#comment_wrapper_16754493
This was previously the role of connectd, but it's actually more
efficient for us to do it: connectd has to sweep through the entire
gossip_store, but we have datastructures for this already.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were in fact feeding l1 its own gossip, which it doesn't ratelimit (this was
a bit fuzzy before, but definitely is the case now!).
So make this node actually l3, so we test what we expected to test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This test makes l2 save db, make a payment, then rollback.
*Sometimes* under CI (but not here) we don't shutdown fast enough
after the payment, so the moves.json from the coin_movements.py
records it. Sure enough, the result is the node ends up with
a -10000msat balance.
Validated by putting a time.sleep(5) between:
```
l2.rpc.pay(inv['bolt11'])
import time
time.sleep(5)
# stop both nodes, roll back l2's database
```
The answer, of course, is to save and rollback *both* the db and
moves.json file.
Here's the error:
```
def test_penalty_htlc_tx_timeout(node_factory, bitcoind, chainparams):
...
assert account_balance(l3, channel_id) == 0
> assert account_balance(l2, channel_id) == 0
tests/test_closing.py:1527:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/utils.py:184: in account_balance
m_sum -= Millisatoshi(m['debit_msat'])
contrib/pyln-client/pyln/client/lightning.py:197: in __sub__
return Millisatoshi(int(self) - int(other))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = -10000msat, v = -10000
...
if self.millisatoshis < 0:
> raise ValueError("Millisatoshi must be >= 0")
E ValueError: Millisatoshi must be >= 0
contrib/pyln-client/pyln/client/lightning.py:82: ValueError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need to wait until we're sure bcli has handed results to lightningd:
```
> assert feerates['perkw']['mutual_close'] == 5000
E assert 6250 == 5000
tests/test_misc.py:1617: AssertionError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
You still need to actually make a rune when lightningd starts, as
commando (for safety) won't work unless you actually generate a rune
(that it knows of!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: hsmtool: `makerune` command to make a master rune for a node.
