It only works on BOLT11, and has long been replaced by the more
generic "decode".
Removing it will stop the confusion!
(Note: documentation claims it was introduced in 23.08, but that was
wrong, as it's been in CLN since the beginning).
[ Fixup from: niftynei <niftynei@gmail.com> ]
Fixes: https://github.com/ElementsProject/lightning/issues/6419
Changelog-Deprecated: JSON-RPC: `decodepay`: use `decode`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since wait_for_onchaind_tx doesn't actually wait for the call to bitcoind to
return, we have a race in checking if the txid is in the mempool. Fix this
by making wait_for_onchaind_tx actually wait for the response (except for delayed txs!).
```
2024-11-15T07:15:22.0836959Z def test_htlc_in_timeout(node_factory, bitcoind, executor):
2024-11-15T07:15:22.0837722Z """Test that we drop onchain if the peer doesn't accept fulfilled HTLC"""
2024-11-15T07:15:22.0838208Z
2024-11-15T07:15:22.0838585Z # HTLC 1->2, 1 fails after 2 has sent committed the fulfill
2024-11-15T07:15:22.0839137Z disconnects = ['-WIRE_REVOKE_AND_ACK*2']
2024-11-15T07:15:22.0839741Z # Feerates identical so we don't get gratuitous commit to update them
2024-11-15T07:15:22.0840304Z l1 = node_factory.get_node(disconnect=disconnects,
2024-11-15T07:15:22.0840839Z options={'dev-no-reconnect': None},
2024-11-15T07:15:22.0841285Z feerates=(7500, 7500, 7500, 7500))
2024-11-15T07:15:22.0841673Z l2 = node_factory.get_node()
2024-11-15T07:15:22.0842278Z # Give it some sats for anchor spend!
2024-11-15T07:15:22.0842679Z l2.fundwallet(25000, mine_block=False)
2024-11-15T07:15:22.0843013Z
2024-11-15T07:15:22.0843342Z l1.rpc.connect(l2.info['id'], 'localhost', l2.port)
2024-11-15T07:15:22.0843753Z chanid, _ = l1.fundchannel(l2, 10**6)
2024-11-15T07:15:22.0844058Z
2024-11-15T07:15:22.0844291Z sync_blockheight(bitcoind, [l1, l2])
2024-11-15T07:15:22.0844606Z
2024-11-15T07:15:22.0844958Z amt = 200000000
2024-11-15T07:15:22.0845713Z inv = l2.rpc.invoice(amt, 'test_htlc_in_timeout', 'desc')['bolt11']
2024-11-15T07:15:22.0846612Z assert only_one(l2.rpc.listinvoices('test_htlc_in_timeout')['invoices'])['status'] == 'unpaid'
2024-11-15T07:15:22.0847141Z
2024-11-15T07:15:22.0847430Z executor.submit(l1.dev_pay, inv, dev_use_shadow=False)
2024-11-15T07:15:22.0847805Z
2024-11-15T07:15:22.0848041Z # l1 will disconnect and not reconnect.
2024-11-15T07:15:22.0848660Z l1.daemon.wait_for_log('dev_disconnect: -WIRE_REVOKE_AND_ACK')
2024-11-15T07:15:22.0850393Z
2024-11-15T07:15:22.0851297Z # Deadline HTLC expiry minus 1/2 cltv-expiry delta (rounded up) (== cltv - 3). cltv is 5+1.
2024-11-15T07:15:22.0852146Z # shadow route can add extra blocks!
2024-11-15T07:15:22.0852622Z status = only_one(l1.rpc.call('paystatus')['pay'])
2024-11-15T07:15:22.0853044Z if 'shadow' in status:
2024-11-15T07:15:22.0853861Z shadowlen = 6 * status['shadow'].count('Added 6 cltv delay for shadow')
2024-11-15T07:15:22.0854325Z else:
2024-11-15T07:15:22.0854547Z shadowlen = 0
2024-11-15T07:15:22.0854845Z bitcoind.generate_block(2 + shadowlen)
2024-11-15T07:15:22.0855292Z assert not l2.daemon.is_in_log('hit deadline')
2024-11-15T07:15:22.0855669Z bitcoind.generate_block(1)
2024-11-15T07:15:22.0855950Z
2024-11-15T07:15:22.0856406Z l2.daemon.wait_for_log('Fulfilled HTLC 0 SENT_REMOVE_COMMIT cltv .* hit deadline')
2024-11-15T07:15:22.0856997Z l2.daemon.wait_for_log('sendrawtx exit 0')
2024-11-15T07:15:22.0857360Z l2.bitcoin.generate_block(1)
2024-11-15T07:15:22.0857741Z l2.daemon.wait_for_log(' to ONCHAIN')
2024-11-15T07:15:22.0858137Z l1.daemon.wait_for_log(' to ONCHAIN')
2024-11-15T07:15:22.0858644Z
2024-11-15T07:15:22.0859068Z # L2 will collect HTLC (iff no shadow route)
2024-11-15T07:15:22.0859741Z _, txid, blocks = l2.wait_for_onchaind_tx('OUR_HTLC_SUCCESS_TX',
2024-11-15T07:15:22.0860287Z 'OUR_UNILATERAL/THEIR_HTLC')
2024-11-15T07:15:22.0860662Z assert blocks == 0
2024-11-15T07:15:22.0860908Z
2024-11-15T07:15:22.0861262Z # If we try to reuse the same output as we used for the anchor spend, then
2024-11-15T07:15:22.0861951Z # bitcoind can reject it. In that case we'll try again after we get change
2024-11-15T07:15:22.0862433Z # from anchor spend.
2024-11-15T07:15:22.0862768Z if txid not in bitcoind.rpc.getrawmempool():
2024-11-15T07:15:22.0863354Z bitcoind.generate_block(1)
2024-11-15T07:15:22.0863735Z > bitcoind.generate_block(1, wait_for_mempool=1)
2024-11-15T07:15:22.0864019Z
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Bitcoind's web server has a default of 4 threads, with queue depth 16 and it fails rather than queue beyond that.
Increasing thread count to 20 for running >10 lightning test nodes simultaneously.
v4.0 no longer works for me (see below, and widely reported elsewhere).
v5.0 doesn't understand f strings, and creates a flood of complaints.
v6.0 requires python >= 3.8.1, so we need to update that.
v7.0 is the latest, but why push it.
```
make check-python-flake8
Traceback (most recent call last):
File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/bin/flake8", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/main/cli.py", line 22, in main
app.run(argv)
File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/main/application.py", line 375, in run
self._run(argv)
File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/main/application.py", line 363, in _run
self.initialize(argv)
File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/main/application.py", line 343, in initialize
self.find_plugins(config_finder)
File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/main/application.py", line 157, in find_plugins
self.check_plugins = plugin_manager.Checkers(local_plugins.extension)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/plugins/manager.py", line 363, in __init__
self.manager = PluginManager(
^^^^^^^^^^^^^^
File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/plugins/manager.py", line 243, in __init__
self._load_entrypoint_plugins()
File "/home/rusty/.cache/pypoetry/virtualenvs/cln-meta-project-BgKQHyxC-py3.12/lib/python3.12/site-packages/flake8/plugins/manager.py", line 261, in _load_entrypoint_plugins
eps = importlib_metadata.entry_points().get(self.namespace, ())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'EntryPoints' object has no attribute 'get'
make: *** [Makefile:535: check-python-flake8] Error 1
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means we do have to set the network correctly though,
and also we can get query messages from lightningd which we have to filter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rather than `allow_broken_log`, we have `broken_log` which is a regex
indicating what log lines are expected. This tightens our tests
significantly, as it will catch *unexpected* brokenness.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We do this for DEBUG_SUBD already, but I wanted to debug the main lightningd.
(We rename --debugger to the more accurate --dev-debug-self)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We still want to test non-anchor channels, as we still support them, but
we've made it non-experimental. To test non-anchor channels, we
use dev-force-features: -23.
Changelog-Added: Protocol: `option_anchors_zero_fee_htlc_tx` enabled, no longer experimental.
Changelog-Changed: Config: `experimental-anchors` now does nothing (it's enabled by default).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Header from folded patch 'fixup!_options__make_anchors_enabled_by_default,_ignore_experimental-anchors.patch':
fixup! options: make anchors enabled by default, ignore experimental-anchors.
Now _msat fields are all integers (last conversion 23.08) we can simply
leave them alone, rather than trying to convert them.
And for turning Millisatoshi into JSON, we simply globally replace the
default encoding function to try ".to_json()" on items, which allows
anything to be marshalled.
The global replacement was interfering with other uses of JSON, such
as the clnrest plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: pyln-client: no longer autoconverts _msat field to Millisatoshi class (leaves as ints).
This breaks our tests a bit, which assumed we can always see ourselves
even if we don't have a proper channel.
Usually we would deprecate this first, but it's unlikely to break
anyone since it's a bit obscure that this worked at all.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `listnodes` no longer shows private (peer) nodes: use listpeers
Put an assertion inside db.c, and run every command we do (in testing) through
a `check` variant.
I inserted a deliberate bug (made addpsbtoutput call wallet_get_newindex()
before returning when running `check`, and indeed, backtrace as expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
`[False]` is considered True in Python, so this flake fix didn't work.
```
>>> if [False]:
... print('x')
...
x
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We truncate the file on stop(), but don't re-created it on start().
We didn't notice it before, but the net
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This makes the pay helper function being able to route
a payment using the optional `route` paramter that defaults to `False`.
I added this, as some plugins maintained their own version of `pay` that
needed routed payment helper.
Changelog-None
Update the lightningd <-> channeld interface with lots of new commands to needed to facilitate spicing.
Implement the channeld splicing protocol leveraging the interactivetx protocol.
Implement lightningd’s channel_control to support channeld in its splicing efforts.
Changelog-Added: Added the features to enable splicing & resizing of active channels.
We also document how the grpc test mode works, and why it currently
lacks coverage.
Changelog-Changed: pyln-testing: The grpc dependencies are now optional.
Having the grpc bindings in the pyln-testing package was always a bit
strange, however it came with additional issues. Due to the way that
protos are handled by protobuf, any name clash, independently of where
we import the protos, would cause an import error. This usually
happens in diamond-pattern dependencies, and so pull out the generated
files into their own package that everyone else can rely on.
Changelog-None
We use parameterization here. The old `anchor_expected()` was for
non-zero-fee anchors, and have bitrotted so there are some other
changes as well.
Unfortunately, all the anchor accounting seems to be broken, but I
cannot understand these tests at all. I had to simply disable them
for now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now we've set everything up, the replacement code is quite simple.
Some tests now have to deal with RBF though, and our rbf tests need work
since they look for the old onchaind messages.
In particular, when we can't afford the fee we want, we back off to
the next blockcount estimate, rather than spending all on fees
(necessarily). So test_penalty_rbf_burn no longer applies.
Changelog-Changed: Protocol: spending unilateral close transactions now use dynamic fees based on deadlines (and RBF), instead of fixed fees.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rather than have specific-purpose levels, have an array of
[blockcount, feerate], and rebuild the specific-purpose levels
for now on top.
We also keep a *separate* smoothed feerate, so you can ask for that
explicitly.
Since all the plugins used the same formula to derive the different
named fee levels, we apply the reverse to return to the underlying
estimates: updating the interface comes next.
This is ugly for now, but various specific-purpose levels will be
going away, as we shift to deadline-driven fees.
This temporarily breaks the floor calculation, so that test is
disabled.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Using single tuples in Python is ugly, so:
1. Rename wait_for_onchaind_tx to wait_for_onchaind_txs.
2. Make it take tuples explicitly.
3. Make wait_for_onchaind_tx a simpler wrapper/unwrapper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We can no longer grab the tx in one line as we did with
wait_for_onchaind_broadcast, we need to track the broadcast from
lightningd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
CI seems to block; Christian suggests the throttler may be to blame somehow?
Since trying to fix it made it worse, let's just remove it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
