Commit graph

15684 commits

Author SHA1 Message Date
Lagrang3
2b3fd67dfb askrene: don't skip fee_fallback test
The fee_fallback test would fail after fixing the computation of the
median. Now by we can restore it by making the probability cost factor
1000x higher than the ratio of the median. This shows how hard it is to
combine fee and probability costs and why is the current approach so
fragile.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
9fdcc26d1d askrene: bugfix queue overflow
Changelog-none

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
9966969d4c askrene: remove allocation checks
Rusty: "allocations don't fail"

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
460a28bb32 askrene: add compiler flag ASKRENE_UNITTEST
Rusty: "We don't generally use NDEBUG in our code"

Instead use a compile time flag ASKRENE_UNITTEST to make checks on unit
tests that we don't normally need on release code.

Changelog-none

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
b1cd26373b askrene: small fixes suggested by Rusty Russell
- use graph_max_num_arcs/nodes instead of tal_count in bound checks,
- don't use ccan/lqueue, use instead a minimalistic queue
  implementation with an array,
- add missing const qualifiers to temporary tal allocators,
- check preconditions with assert,
- remove inline specifier for static functions,

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
44c9609f3a askrene: add arbitrary precision flow unit
Changelog-none: askrene: add arbitrary precision flow unit

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
225939a5e3 add ratio ceil and floor operators on amount_msat
Changelog-none: add ratio ceil and floor operators on amount_msat

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
4dc1a44cd9 askrene: fix the median
The calculation of the median values of probability and fee cost in the
linear approximation had a bug by counting on non-existing arcs.

Changelog-none: askrene: fix the median

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
ee623616d2 askrene: fix CI
check the return value of scanf in askrene unit tests,

Changelog-none: askrene: fix CI

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
937cf7a554 askrene: use the new MCF solver
Changelog-none: askrene: use the new MCF solver

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
84a9476311 askrene: add mcf_refinement to the public API
Changelog-none: askrene: add mcf_refinement to the public API

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
2142094e43 askrene: fix bug, not all arcs exists
We use an arc "array" in the graph structure, but not all arc indexes
correspond to real topological arcs. We must be careful when iterating
through all arcs, and check if they are enabled before making operations
on them.

Changelog-None: askrene: fix bug, not all arcs exists

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
e655fe7bbd askrene: add bigger test for MCF
Using zlib to read big test case file.

Changelog-None: askrene: add bigger test for MCF

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
2ea8e49683 askrene: add a MCF refinement
Add a new function to compute a MCF using a more general description of
the problem. I call it mcf_refinement because it can start with a
feasible flow (though this is not necessary) and adapt it to achieve
optimality.

Changelog-None: askrene: add a MCF refinement

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
1dfa562cd9 askrene algorithm add helper for flow conservation
Changelog-None: askrene algorithm add helper for flow conservation

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
42d075cc97 askrene: add a simple MCF solver
Changelog-EXPERIMENTAL: askrene: add a simple MCF solver

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
8558299dec askrene: add algorithm to compute feasible flow
Changelog-EXPERIMENTAL: askrene: add algorithm to compute feasible flow

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
f4f2985bdf askrene: add dijkstra algorithm
Changelog-EXPERIMENTAL: askrene: add dijkstra algorithm

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
507153a1cd askrene: add graph algorithms module
Changelog-EXPERIMENTAL: askrene: add graph algorithms module

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
59ce410699 askrene: add priorityqueue
It is just a copy-paste of "dijkstra" but the name
implies what it actually is. Not an implementation of minimum cost path
Dijkstra algorithm, but a helper data structure.
I keep the old "dijkstra.h/c" files for the moment to avoid breaking the
current code.

Changelog-EXPERIMENTAL: askrene: add priorityqueue

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Lagrang3
32548cf02b askrene: add a new graph abstraction
Changelog-EXPERIMENTAL: askrene new graph abstraction

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-11-21 16:17:52 +10:30
Rusty Russell
799acc90e6 lightningd: tell gossipd channel is closed if it tells us about our channel and is wrong.
While we have (I hope!) fixed the underlying sync problem, this can still happen with
older gossip.  So now we tell it the channel is dead, so it won't happen more than once.

Fixes: #7703
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-21 15:42:06 +10:30
Alex Myers
ead5dbf6a2 pytest: allow additional gossip filters
in test_gossip_force_broadcast_channel_msgs now that the seeker
is asking for periodic full gossip syncs.
2024-11-21 14:23:57 +10:30
Alex Myers
84b0dace31 gossipd: seeker: rotate worst gossiper every 30 minutes
This will allow all gossipers to be rotated in ~5 hours similar to
how it operated with half as many gossip streaming peers.
2024-11-21 14:23:57 +10:30
Alex Myers
323f23cf8f gossipd: seeker: rotate out the worst performing gossiper
Previously they were chosen at random.  We should instead
prefer to strop streaming from the peer who has provided
the least novel gossip.
2024-11-21 14:23:57 +10:30
Alex Myers
04180c1cad gossipd: add separate counter for unsolicted gossip
This will be used to drop underperforming gossipers instead
of choosing at random.
2024-11-21 14:23:57 +10:30
Alex Myers
995648911e gossipd: seeker: choose a new node when resyncing 2024-11-21 14:23:57 +10:30
Alex Myers
be694cba3a gossipd: seeker: add hourly full gossip resync from a random peer
This can help us backfill any missing gossip if our current
peers haven't been the most reliable.

Changelog-Changed: Gossipd requests a full sync from a random peer every hour.
2024-11-21 14:23:57 +10:30
Alex Myers
80bde554a4 gossipd: Increase gossiping peers to 10
Based on gossip sync data from random network peers, listening to only 5
peers will not reliably catch all gossip traffic.  For now, add extra
redundancy.
2024-11-21 14:23:57 +10:30
Dusty Daemon
d04e64670d splice: tx_abort no longer reestablishes
As per eclair implementation we skip `channel_reestablish` and go straight into the channel for `tx_abort` events.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
6d63e68e99 splice: Update messages to spec
Changelog-Changed: Splicing moved from test numbers to spec numbers.
2024-11-21 14:15:36 +10:30
Dusty Daemon
dcdf7db65f splice: Enable remote funding key rotation
Allows our peer to change their funding pub key during a splice.

Changelog-Changed: Support added for peers that wish to rotate their funding pubkey during a splice.
2024-11-21 14:15:36 +10:30
Dusty Daemon
03d7d8f45a splice: Update funding pubkey on splice lock
Set the remote funding pubkey on both lightningd and channeld when mutual splice lock is achieved.

This will be needed once rotating funding keys is enabled during splicing

Changelog-None.
2024-11-21 14:15:36 +10:30
Dusty Daemon
5f330b3cd6 channeld: tx_abort should skip reestablish
`tx_abort` should not send reestablish message and instead go into ‘reconnect only mode’

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
d04478f99e splice: Enable shared tx on interactivetx
By placing the funding tx into `interactivetx`, the message will be compressed by only sending the txid via tlvs.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
90c786f61b channeld: Move tx lookup function up
This function needs to be used earlier in the file so it is moved vertically up.

Chanelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
9b06a85f54 splice: Update find_channel_output for rotating funding key
We need to differentiate the funding pubkey since we allow the peer to rotate it now.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
2bb5d8ac67 splice: Update commit sigs to use dynamic remote funding pubkey.
Update the sending and receiving of commit sigs to use dyniamic funding pubkeys incase our remote peer rotates theirs during a splice.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
7a8e377b81 splice: Update commitment_signed_tlvs
As per eclair spec proposal.
1) A renaming to `funding_txid`
2) Adding of `batch_size` to indicate how many commitment_signed msgs are expected.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
2b41ada941 splice: Add new funding output balance
The prior spec left this value at 0 to be calculted later but the current spec requires we fill it in in advance.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
2b3cb8b8a8 splice: Update splice signature msg type
Update to use spec signature type.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
d077fd59c9 splice: Remove blockhash from peer msg
This is no longer used.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
aea35536c0 channeld: Update commitsigs to support remote_funding
Since funding keys can be rotated during splice, commit sig routines must be able to handle a dynamic value for the funding keys.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
a21ae33b2d channeld: Add dynamic funding_pubkeys to channel_txs
In anticipation of adding support for rotating funding pubkeys during a splice, `channel_txs` is updated to support specifying these manually instead of using the channel’s funding pubkeys.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
42440e3bee splice: Pass remote_funding between ld and channeld
Update lightningd and channeld interface to pass the remote funding pubkey back and forth to both daemons.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
17d2b24ebb channeld: Add remote_funding to inflight
Channeld stores its own cache of `inflight` and that needs to have a copy of `remote_funding` as well.

Since copying a secp256k1 pubkey isn’t documented and `copy_inflight` isn’t used anyway — we’re dropping `copy_inflight`.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
ba3bd750fa channeld: Store remote_funding for splice
Instead of assuming the remote funding pubkey does not change during splice, we store the new pubkey in the splice object.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
2e92fdc507 interactivetx: Add support for shared prevtx
It is possible for prevtx to be larger than max packet size, so for shared outputs (currently only the funding tx) we add support for sending the `txid` only across the wire and filling in the prevtx locally.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
560ca00a2b splice: Add remote_funding to database
Enable storing the remote funding pubkey in DB if the channel peer decides to change it during splicing. It needs to be in DB incase of restarts mid-splice.

Changelog-None
2024-11-21 14:15:36 +10:30
Dusty Daemon
7d1a43d5d5 channeld: Fix tx_abort encoding
Switch to using same message format for `tx_abort` that wire_error and wire_warning use.

Changelog-None
2024-11-21 14:15:36 +10:30