Commit graph

7716 commits

Author SHA1 Message Date
Matt Corallo
61153f10eb
Merge pull request #3245 from valentinewallace/2024-07-blinded-pay-path-refactor
Move `BlindedPayInfo` into `BlindedPaymentPath`
2024-08-19 23:24:15 +00:00
Valentine Wallace
4ef83a0566
Test blinded PaymentParameter round trip serialization. 2024-08-19 16:42:23 -04:00
Valentine Wallace
db94d930f0
Move BlindedPayInfo into blinded_path::payment module. 2024-08-19 16:42:22 -04:00
Valentine Wallace
7038906890
Move BlindedPayInfo into BlindedPaymentPath.
Also removes the implementation of Writeable for BlindedPaymentPath, to ensure
callsites are explicit about what they're writing.
2024-08-19 16:28:30 -04:00
Matt Corallo
fb4403f8ae
Merge pull request #3204 from valentinewallace/2024-07-rb-test-vectors
Implement route blinding test vectors
2024-08-19 19:57:04 +00:00
Matt Corallo
05ed0db67f
Merge pull request #3188 from TheBlueMatt/2023-12-cache-scoring-points
Better Encapsulate Scoring and Cache Point Count
2024-08-19 19:02:05 +00:00
Matt Corallo
37154109c1
Merge pull request #3249 from TheBlueMatt/2024-08-feature-cleanup-1
bitcoin 0.32 upgrade followups
2024-08-19 18:50:52 +00:00
Matt Corallo
bad5e32694
Merge pull request #3222 from tnull/2024-08-rustfmt-lightning-persister
`rustfmt`: Run on `lightning-persister`
2024-08-19 17:18:04 +00:00
Valentine Wallace
6f6115f177
Avoid && in public API for onion peeling/decoding.
No reason to take a reference to a Deref.
2024-08-19 12:47:42 -04:00
Valentine Wallace
8739b06620
Implement route blinding spec test vectors. 2024-08-19 12:47:42 -04:00
Valentine Wallace
9dba36a441
Fix blinded hop feature serialization.
We were writing a length redundantly...
2024-08-19 12:47:42 -04:00
Valentine Wallace
f68b1249a9
Support next_blinding_override in blinded payment paths.
This allow us to forward blinded payments where the blinded path that we are
forwarding within was concatenated to another blinded path that starts at the
next hop.

Also allows constructing blinded paths using this override.
2024-08-19 12:47:40 -04:00
Matt Corallo
8049f99ebc Remove the std feature implications from the lightning crate
Now that we don't have to have everything in our entire ecosystem
use the same `std`/`no-std` feature combinations we should start by
untangling our own features a bit.

This takes one last step, removing the implications of the `std`
feature from the `lightning` crate.
2024-08-19 15:09:40 +00:00
Matt Corallo
3fe4ef9640 Drop the _test_vectors feature in favor of a cfg flag
This exists just for tests, so there's no reason for it to be
publicly visible.
2024-08-19 15:09:40 +00:00
Matt Corallo
e212d74298 Somewhat clean up ci-tests.sh and sort by RUSTFLAGS 2024-08-19 15:09:40 +00:00
Matt Corallo
51f5bc881f Drop the no-std feature from lightning-rapid-gossip-sync
Now that we don't have to have everything in our entire ecosystem
use the same `std`/`no-std` feature combinations we should start by
untangling our own features a bit.

This takes another step by removing the `no-std` feature entirely
from the `lightning-rapid-gossip-sync` crate and removing all
feature implications on dependencies from the remaining `std`
feature.
2024-08-19 15:09:40 +00:00
Matt Corallo
31a9cd2d3d Drop the no-std feature from lightning-invoice
Now that we don't have to have everything in our entire ecosystem
use the same `std`/`no-std` feature combinations we should start by
untangling our own features a bit.

This takes another step by removing the `no-std` feature entirely
from the `lightning-invoice` crate and removing all feature
implications on dependencies from the remaining `std` feature.
2024-08-19 15:09:40 +00:00
Matt Corallo
5e37074246 Drop the no-std feature from BP and drop feature implications
Now that we don't have to have everything in our entire ecosystem
use the same `std`/`no-std` feature combinations we should start by
untangling our own features a bit.

This takes the first step by removing the `no-std` feature entirely
from the `lightning-background-processor` crate and removing most
feature implications on dependencies from the remaining `std`
feature.

It also addresses a CI oversight where we were not testing
`lightning-background-processor` without the `std` feature in CI at
all.
2024-08-19 15:09:40 +00:00
Matt Corallo
9ca38c59f4 Ensure we always pass a path for in-workspace dependencies
In order to ensure our crates depend on the workspace copies of
each other in test builds we need to override the crates.io
dependency with a local `path`.

We can do this in one of two ways - either specify the `path` in
the dependency listing in each crate's `Cargo.toml` or use the
workspace `Cargo.toml` to `patch` all dependencies. The first is
tedious while the second lets us have it all in one place. However,
the second option does break `cargo *` in individual crate
directories (forcing the use of `cargo -p crate *` instead) and
makes it rather difficult to depend on local versions of workspace
crates.

Thus, here we drop the `patch.crates-io` from our top-level
`Cargo.toml` entirely.

Still, we do update the `ci/ci-tests.sh` script here to use
`cargo -p crate` instead of switching to each crate's directory as
it allows `cargo` to use a shared `target` and may speed up tests.
2024-08-19 15:09:38 +00:00
Matt Corallo
f8b7ad242d Optimize BufReader somewhat for the only used case
`rust-bitcoin` doesn't ever actually *use* its `BufRead`
requirement when deserializing objects, and forcing it is somewhat
inefficient, so we optimize the only (actual) case here by passing
reads straight through to the backing stream.
2024-08-19 14:55:28 +00:00
Matt Corallo
aadf233d6c Fix bug in BufReader::fill_buf when reaching EOF
When we reach EOF we may return a full buffer when we should return
an empty one.
2024-08-19 14:55:28 +00:00
Matt Corallo
dd37077c35
Merge pull request #3240 from tnull/2024-08-add-ci-flag-for-bitcoind
Disallow skipping tx-sync tests in CI
2024-08-19 14:53:36 +00:00
Matt Corallo
4fd07ecada Tweak alignment and memory order in the ProbabilisticScorer
During routing, the majority of our time is spent in the scorer.
Given the scorer isn't actually doing all that much computation,
this means we're quite sensitive to memory latency. Thus, the cache
lines our data sits on is incredibly important.

Here, we manually lay out the `ChannelLiquidity` and
`HistoricalLiquidityTracker` structs to ensure that we can do the
non-historical scoring and skip historical scoring for channels
with insufficient data by just looking at the same cache line the
channel's SCID is on.

Sadly, to do the full historical scoring we need to load a second
128-byte cache line pair, but we have some time to get there. We
might consider issuing a preload instruction in the future.

This improves performance a few percent.
2024-08-19 14:25:58 +00:00
Matt Corallo
8904073a15 Cache the total points tracked in our historical liquidity data
When we go to score a channel using the historical liquidity data,
the first thing we do is step through all the valid bucket
combinations, multiply the min and max bucket, and then add them
together to calculate the total number of points tracked. This
isn't a free operation, and for scorers without much data it
represents a large part of the total time spent scoring during
routefinding.

Thus, here we cache this value, updating it every time the buckets
are updated.
2024-08-19 14:25:58 +00:00
Matt Corallo
c0cc30e874 Change the directed history tracker's storage of its direction
Rather than storing the two direction's buckets in
`HistoricalMinMaxBuckets` (renamed
`DirectedHistoricalLiquidityTracker`), we store a single reference
to the `HistoricalLiquidityTracker` as well as the direction bool.

This will allow us in the next commit to reference fields in the
`HistoricalLiquidityTracker` aside from the two directions.
2024-08-19 14:25:58 +00:00
Matt Corallo
3bd8140301 Make the historical bucket data private to bucketed_history
In a comming commit we'll cache some additional data in the
historical bucket tracker. In order to do so, here we isolate the
buckets themselves into the `bucketed_history` module, reducing
the possibility of accidentally updating them directly without
updating caches.
2024-08-19 14:25:58 +00:00
Matt Corallo
c62e346b5d Store both min and max historical buckets in one, new, struct
In the coming commits we'll isolate historical bucket logic slightly
further, allowing us to cache some state. This is the first step
towards that, storing the historical liquidity information in a new
`HistoricalLiquidityTracker` rather than in the general
`ChannelLiquidity`.
2024-08-19 14:25:48 +00:00
Matt Corallo
dcc84dee4d Format log_approx.rs per rustfmt 2024-08-19 14:24:18 +00:00
Matt Corallo
1c84d3b14c Move log approximation from scoring.rs to its own file 2024-08-19 14:24:18 +00:00
valentinewallace
ccce9d9602
Merge pull request #3242 from jkczyz/2024-08-payment-id-auth-api
Add `PaymentId` authentication to public API
2024-08-19 10:14:37 -04:00
Elias Rohrer
e393893a34
rustfmt: Drop lightning-persister from exclusion list 2024-08-17 09:23:37 +02:00
Elias Rohrer
f87eaccae8
rustfmt: Run on lightning-persister 2024-08-17 09:23:36 +02:00
Jeffrey Czyz
5bee20e237
Add PaymentId authentication to public API
When receiving an InvoiceError message, it should be authenticated
before using it to abandon the payment. Add methods to PaymentId's
public API for constructing and verifying an HMAC for use in
OffersContext::OutboundPayment. This allows other implementations of
OffersMessageHandler to construct the HMAC and authenticate the message.
2024-08-16 15:43:43 -05:00
Matt Corallo
43dcf2f3d8
Merge pull request #3239 from arik-so/bitcoin-0.32.2-upgrade
Bitcoin 0.32.2 upgrade
2024-08-16 20:13:26 +00:00
Arik Sosman
36b484a720
Fix fuzz warnings.
Version 0.32.2 of `rust-bitcoin` deprecates a number of methods that
are commonly used in this project, most visibly `txid()`, which is
now called `compute_txid()`. This resulted in a lot of warnings, and
this commit is part of a series that seeks to address that.
2024-08-16 10:31:45 -07:00
Arik Sosman
9914fd4f8d
Fix lightning-transaction sync warnings.
Version 0.32.2 of `rust-bitcoin` deprecates a number of methods that
are commonly used in this project, most visibly `txid()`, which is
now called `compute_txid()`. This resulted in a lot of warnings, and
this commit is part of a series that seeks to address that.
2024-08-16 10:31:45 -07:00
Arik Sosman
92aac2a2ae
Fix lightning-block-sync warnings.
Version 0.32.2 of `rust-bitcoin` deprecates a number of methods that
are commonly used in this project, most visibly `txid()`, which is
now called `compute_txid()`. This resulted in a lot of warnings, and
this commit is part of a series that seeks to address that.
2024-08-16 10:31:45 -07:00
Arik Sosman
813202f554
Fix lightning-background-processor warnings.
Version 0.32.2 of `rust-bitcoin` deprecates a number of methods that
are commonly used in this project, most visibly `txid()`, which is
now called `compute_txid()`. This resulted in a lot of warnings, and
this commit is part of a series that seeks to address that.
2024-08-16 10:31:45 -07:00
Arik Sosman
d85a1a9a8d
Fix lightning warnings.
Version 0.32.2 of `rust-bitcoin` deprecates a number of methods that
are commonly used in this project, most visibly `txid()`, which is
now called `compute_txid()`. This resulted in a lot of warnings, and
this commit is part of a series that seeks to address that.
2024-08-16 10:31:45 -07:00
Arik Sosman
60ca255e5e
Suppress Taproot warnings.
Taproot is not yet being actively used, and this commit suppresses
the noisy warning generation.
2024-08-16 10:31:45 -07:00
Arik Sosman
0cc9378afc
Remove secp256k1 dependency. 2024-08-16 10:31:45 -07:00
Arik Sosman
176d2ad599
Upgrade rust-bitcoin to 0.32.2. 2024-08-16 10:31:45 -07:00
Matt Corallo
8fe3a56d4b
Merge pull request #3182 from valentinewallace/2024-07-expose-advance-bp
Expose blinded paths' `advance_path_by_one`
2024-08-16 17:11:21 +00:00
Jeffrey Czyz
fa6c0587e7
Return a Result from verify_payment_id 2024-08-16 11:31:31 -05:00
Arik Sosman
40b04000aa
Reimplement io::Cursor.
The `bitcoin::io` Cursor will miss a bunch of functionality that we were
using that we will reimplement here, most critically `set_position`.
2024-08-16 09:23:09 -07:00
Arik Sosman
b05113ab72
Introduce io module.
The rust-bitcoin upgrade will introduce `bitcoin::io` module, which
will be missing a necessary subset of traits.

To accommodate those traits' future implementations, we move the
`lightning::io` module to its own file, where we will be able to
implement the missing trait subset in the next commit.
2024-08-16 09:23:09 -07:00
Arik Sosman
fb818e09a0
Drop Seek and SeekFrom usage.
In lieu of using `Seek` and `SeekFrom`, we will instead rely
exclusively on the `io::Cursor` type.
2024-08-16 09:23:09 -07:00
Arik Sosman
d1767cdedf
Simplify Readable mutability.
In anticipation of the rust-bitcoin upgrade, which incorporates its
own `io::Read` implementation, we need to make our usage compatible
with dropping `std::io` and `core2::io`.

Notably, in version 0.32.2, `bitcoin::io`'s `Read` is no longer
implemented for `&mut R where R: Read + ?Sized`, which results in
errors anytime `&mut &mut Readable` is passed instead of
`&mut Readable`.

This commit fixes those instances.
2024-08-16 09:23:09 -07:00
Elias Rohrer
43bc78ce3d
Disallow skipping tx-sync tests in CI
Previously, we'd always skip tx-sync tests if the
`BITCOIND_EXE`/`ELECTRS_EXE` environment variables would be unset. While
this is especially fine for local testing, we still want to enforce
tests failing if somehow the `bitcoind`/`electrs` downloading or caching
in CI stops working. Here, we therefore add a `CI_ENV` variable that
indicates we're indeed running in CI, and only skip if it's unset.
2024-08-16 17:41:16 +02:00
Valentine Wallace
b204a6a11d
Make advance_path_by_one an associated method. 2024-08-16 10:30:42 -04:00