Running `git describe` in local setup correctly identifies the tag available on the tag. But `docker buildx` via actions does not identify it without `--tags` param.
Reference Issue: https://github.com/ElementsProject/lightning/issues/7626
Changelog-Fixed: Docker image created via github actions correctly reads the tag available on the HEAD.
Python was installing `pyln-client` directly from the server for testplugpass plugin. This commit is updating the requirements.txt file to install pyln-client with absolute local path.
Commit a1fdeee76b "Makefile: clean up install path handling."
broke the ability to configure with one path and then run in a
different path. Turns out people actually do this! So, we have
to use relative paths, compared to our existing binary.
And we can't use path_rel, because that requires that the path
exist (thanks @Lagrang3!).
Fixes: https://github.com/ElementsProject/lightning/issues/7595
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Of course, we still will, since spendable is for a single HTLC, but
this also shows why we should treat *minimum* as the incorrect answer
if they cross, too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/7563
It seems we didn't handle it correctly: we need to cap the first
segment as well as the others, as far as I can tell.
Also, it can be less than the maximum capacity.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Conversion is lossy, and we don't want to spend more than the channel,
so it's conservative to round down here.
This doesn't actually help our test though!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Based on great test case from https://github.com/daywalker90
```
E AssertionError: assert {'107x2x0/1': 'Path total 285720859 > spendable 285718000', '108x1x0/1': 'Path total 384721849 > spendable 384718000'} == {}
E Left contains 2 more items:
E {'107x2x0/1': 'Path total 285720859 > spendable 285718000',
E '108x1x0/1': 'Path total 384721849 > spendable 384718000'}
E Full diff:
E {
E - ,
E + '107x2x0/1': 'Path total 285720859 > spendable 285718000',
E + '108x1x0/1': 'Path total 384721849 > spendable 384718000',
E }
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
- **Poetry Issue:** The Docker build for `linux/arm/v7` failed in recent RC releases on the Poetry installation step in the `builder-python` stage. This issue occurred because the `builder-python` stage builds on target's arch but poetry was unable to install on arm/v7 without rust >= v1.56.1.
- **Solution:** Instead of installing poetry on the `builder-python` stage, we leveraged the existing multi-arch `builder` stage, which already had Poetry. Now, we export the dependencies from `pyproject.toml` to `requirements.txt` within the `builder` stage and then copy `requirements.txt` to the `builder-python` stage for pip installation.
- **Cryptography installation Issue:** python installations for `pyln-proto` started failing due to Cryptography upgrade from v41 to v42 (#7475). It is because now Cryptography needs cargo/rust also.
- **Solution:** Installing cargo in `builder-python` stage also.
- **Configure Prefix Issue:** Previously, we used `RUN ./configure --prefix=/tmp/lightning_install --enable-static` in the `builder` image and then copied `/tmp/lightning_install` from the `builder` stage to `/usr/local` in the `final` stage. However, this approach is now causing errors due to missing binaries/plugins at their default locations.
- **Solution:** We are now configuring the installation to use the default location (`/usr/local`). To prevent the local image size from increasing by up to 87MB, instead of copying the entire `/usr/local/` directory, we are explicitly copying only the core lightning binaries.
Changelog-Fixed: Fixes failing Docker build for `arm32` arch.
- Temporarily adding `rc` tag trigger for testing `Build and push multi-platform docker images` action flow before the final release.
- Added some variable inputs for testing like repo, platforms, etc.
- Added more logs for future debugging.
You need to know it to make an onion, and in theory if we decided to
fuzz it could be different for different paths.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Channel filter must apply to the modified gossmap+localmods,
otherwise we disable local channels with htlcmax=0.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Add a little bit of uncertainty to the local channels to avoid
consuming precisely all spendable_msat on our side, which leads to
temporary channel failure if the spendable_msat changes during the
course of the payment.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Refresh gossmap once before we read the routehints.
Make sure that if channels in the routehints are public,
we retrieve their capacities from gossip.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
- add more checks
- add more error messages
- compute probabilities without fees during MCF
- compute probabilities with fees during get_routes
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
In theory we should not have htlc_total<=known_max.
But for some strange race condition we do sometimes.
Until we find a solution to ensure the correct state
of the uncertainty network we remove the assertion.
Thanks to signed arithmetic and MIN guards, the rest
of the code in linearize_channel can handle the
weird cases with known_max<htlc_total.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
The probability for a success forward of x amount on a channel is now
computed as:
P_success(x) = b * P_rene(x)
where P_rene(x) is the probability distribution proposed by Rene
Pickhardt with a uniform distribution on the known liquidity of the
channel and 'b' is a new parameter we add in this PR, by the name of
"base probability". The "base probability" represents the probability
for a channel in the network choosen at random to be able to forward at
least 1msat, ie. of being alive, non-depleted and with at least one HTLC
slot. We don't know the value of 'b', but for the moment we assume that
it is 0.98 and use that as default.
As a consequence the probability cost becomes non-linear and non-convex
because of the additional constant term:
Cost(x) = - log P_success(x)
= - log b - - log P_rene(x)
= - log b + Cost_rene(x)
We currently don't handle well base fees and neither this additional
"base probability" but we can as a first approximation linearize the
cost function in this way:
Cost_linear(x) = (- log b)*x + Cost_rene_linear(x)
Changelog-Added: renepay: Add a dev parameter to renepay that represents
a constant probability of availability for all channels in the network.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Commit 901342b50d ("pyln-testing: require
explicit pattern for BROKEN messages.") changed to a manual scan of
logs rather than using is_in_log, so it needs to manually refresh,
otherwise we miss final log messages.
This causes us to often miss memleak messages, which are printed
only in the exit path!
Reported-by: Lagrang3
Fixes: https://github.com/ElementsProject/lightning/issues/7565
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Mainly, it just causes complaints if we're running a lightningd which is installed
(in which case, my_path is not stolen, unlike running locally installed).
Reported-by: daywalker90
Fixes: https://github.com/ElementsProject/lightning/issues/7569
Changelog-None: regression this release
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Our hash tables allow duplicate keys, and we use that in a few places.
However, the get() function only returns the first, so it's not a good
idea with such hash tables.
Another patch fixes this at a deeper level (using different hash table
types depending on whether this table can have duplicates), but this
is the minimal fix for existing code.
This may be the cause behind us occasionally missing onchain events:
Fixes: https://github.com/ElementsProject/lightning/issues/7460
Fixes: https://github.com/ElementsProject/lightning/issues/7377
Fixes: https://github.com/ElementsProject/lightning/issues/7118
Fixes: https://github.com/ElementsProject/lightning/issues/6951
This fixes them in future: fixing them now will require something else.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: lightningd: occasionally we could miss transaction outputs (not telling gossipd, or even onchaind)
msggen cannot handle the complex type in renepay-exclude,
therefore I added a rule override for it, just like pay-exclude.
```
msggen cln-grpc/proto/node.proto
Traceback (most recent call last):
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/__main__.py", line 131, in <module>
main()
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/__main__.py", line 115, in main
run()
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/__main__.py", line 72, in run
service = load_jsonrpc_service(
^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/utils/utils.py", line 241, in load_jsonrpc_service
methods = [load_jsonrpc_method(name) for name in grpc_method_names]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/utils/utils.py", line 209, in load_jsonrpc_method
request = CompositeField.from_js(schema["methods"][rpc_name]['request'], path=name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/model.py", line 297, in from_js
field = ArrayField.from_js(fpath, ftype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/model.py", line 464, in from_js
itemtype = UnionField.from_js(child_js, path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rusty/devel/cvs/lightning/contrib/msggen/msggen/model.py", line 393, in from_js
itemtype = PrimitiveField(
^^^^^^^^^^^^^^^
TypeError: PrimitiveField.__init__() missing 2 required positional arguments: 'added' and 'deprecated'
```
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Changelog-EXPERIMENTAL: renepay: add cli option "exclude" to manually disable channels and nodes.
We add a channel filtering paymod that disables channels that have very
low max_htlc. It can be expanded to consider other properties as well,
for instance high base fee, low capacity or high latency.
Changelog-Added: renepay: prune the network by disabling channels we
don't like, eg. very low max_htlc.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Expecting to have more than just a bunch of disabled channels if we
prune the lightning network heavily I am changing the internal data
structure of disabledmap from a simple array to a hashtable.
Have a finer control over disabled channels by targeting
short_channel_id_dir instead of short_channel_id,
ie. we can disable a single direction of a channel.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
