Commit graph

18060 commits

Author SHA1 Message Date
Andras Banki-Horvath
5e3ef3ec0c
invoices+sql: use the stored AmtPaid value instead of recalculating
Previously we'd recalculate the paid amount by summing amounts of
settled HTLCs. This approach while correct would stop the SQL migration
process as some KV invoices may have incorrectly stored paid amounts.
2025-01-23 09:11:02 +01:00
Andras Banki-Horvath
0839d4ba7b
itest: remove obsolete itest 2025-01-23 09:11:02 +01:00
Andras Banki-Horvath
a29f2430c1
itest: add integration test for invoice migration 2025-01-23 09:11:01 +01:00
Andras Banki-Horvath
8d20e2a23b
lnd: run invoice migration on startup
This commit runs the invoice migration if the user has a KV SQL backend
configured.
2025-01-23 09:11:01 +01:00
Andras Banki-Horvath
94e2724a34
sqldb+invoices: Optimize invoice fetching when the reference is only a hash
The current sqlc GetInvoice query experiences incremental slowdowns during
the migration of large invoice databases, primarily due to its complex
predicate set. For this specific use case, a streamlined GetInvoiceByHash
function provides a more efficient solution, maintaining near-constant
lookup times even with extensive table sizes.
2025-01-23 09:11:01 +01:00
Andras Banki-Horvath
b92f57e0ae
invoices: add migration code that runs a full invoice DB SQL migration 2025-01-23 09:11:01 +01:00
Andras Banki-Horvath
708bed517d
invoices: add migration code for a single invoice 2025-01-23 09:11:01 +01:00
Andras Banki-Horvath
43797d6be7
invoices: add method to create payment hash index
Certain invoices may not have a deterministic payment hash. For such
invoices we still store the payment hashes in our KV database, but we do
not have a sufficient index to retrieve them. This PR adds such index to
the SQL database that will be used during migration to retrieve payment
hashes.
2025-01-23 09:11:00 +01:00
Andras Banki-Horvath
be18f55ca1
invoices: extract method to create invoice insertion params 2025-01-23 09:11:00 +01:00
Andras Banki-Horvath
d65b630568
sqldb: remove unused preimage query parameter 2025-01-23 09:11:00 +01:00
Andras Banki-Horvath
b7d743929d
sqldb: add a temporary index to store KV invoice hash to ID mapping 2025-01-23 09:11:00 +01:00
Andras Banki-Horvath
3820497d7f
sqldb: set settled_at and settle_index on invocie insertion is set
Previously we intentially did not set settled_at and settle_index when
inserting a new invoice as those fields are set when we settle an
invoice through the usual invoice update. As migration requires that we
set these nullable fields, we can safely add them.
2025-01-23 09:11:00 +01:00
Andras Banki-Horvath
115f96c29a
multi: add call to directly insert an AMP sub-invoice 2025-01-23 09:10:59 +01:00
Andras Banki-Horvath
91c3e1496f
sqldb: separate migration execution from construction
This commit separates the execution of SQL and in-code migrations
from their construction. This change is necessary because,
currently, the SQL schema is migrated during the construction
phase in the lncfg package. However, migrations are typically
executed when individual stores are constructed within the
configuration builder.
2025-01-23 09:10:59 +01:00
Andras Banki-Horvath
b789fb2db3
sqldb: add support for custom in-code migrations
This commit introduces support for custom, in-code migrations, allowing
a specific Go function to be executed at a designated database version
during sqlc migrations. If the current database version surpasses the
specified version, the migration will be skipped.
2025-01-23 09:10:59 +01:00
Andras Banki-Horvath
9acd06d296
sqldb: add table to track custom SQL migrations
This commit adds the migration_tracker table which we'll use to track if
a custom migration has already been done.
2025-01-23 09:10:59 +01:00
Andras Banki-Horvath
680394518f
mod: temporarily replace sqldb with local version 2025-01-23 09:10:58 +01:00
Oliver Gugger
a388c1f39d
Merge pull request #9368 from lightningnetwork/yy-waiting-on-merge
Fix itest re new behaviors introduced by `blockbeat`
2024-12-20 07:44:54 -06:00
yyforyongyu
2913f6e4c9
itest: fix flake in testCoopCloseWithExternalDeliveryImpl
The response from `ClosedChannels` may not be up-to-date, so we wrap it
inside a wait closure.
2024-12-20 19:38:15 +08:00
yyforyongyu
76eeae32d6
itest: document and fix wallet UTXO flake 2024-12-20 19:38:14 +08:00
yyforyongyu
7ab4081ffd
lntest: make sure chain backend is synced to miner
We sometimes see `timeout waiting for UTXOs` error from bitcoind-related
itests due to the chain backend not synced to the miner. We now assert
it's synced before continue.
2024-12-20 19:38:14 +08:00
yyforyongyu
1dec926165
workflows: increase num of tranches to 16
Keep the SQL, etcd, bitcoin rpcpolling builds and non-ubuntu builds at 8
since they are less stable.
2024-12-20 19:38:14 +08:00
yyforyongyu
31b66962d8
lntest: properly handle shutdown error
This commit removes the panic used in checking the shutdown log.
Instead, the error is returned and asserted in `shutdownAllNodes` so
it's easier to check which node failed in which test. We also catch all
the errors returned from `StopDaemon` call to properly access the
shutdown behavior.
2024-12-20 19:38:14 +08:00
yyforyongyu
73574d919d
lntest: add timeouts for windows
For Windows the tests run much slower so we create customized timeouts
for them.
2024-12-20 19:38:14 +08:00
yyforyongyu
d7f8fa6ab6
lntest: increase port timeout 2024-12-20 19:38:14 +08:00
yyforyongyu
33b07be8c3
itest: even out num of tests per tranche
Previous splitting logic simply put all the remainder in the last
tranche, which could make the last tranche run significantly more test
cases. We now change it so the remainder is evened out across tranches.
2024-12-20 19:38:14 +08:00
yyforyongyu
c536bc220f
itest: add a prefix before appending a subtest case 2024-12-20 19:38:13 +08:00
yyforyongyu
686a7dd31c
docs: update release notes 2024-12-20 19:38:13 +08:00
yyforyongyu
becbdce64c
lntest: limit the num of blocks mined in each test 2024-12-20 19:38:13 +08:00
yyforyongyu
5236c05dc6
itest+lntest: add new method FundNumCoins
Most of the time we only need to fund the node with given number of
UTXOs without concerning the amount, so we add the more efficient
funding method as it mines a single block in the end.
2024-12-20 19:38:13 +08:00
yyforyongyu
691a6267be
workflows: use btcd for macOS
To increase the speed from 40m per run to roughly 20m per run.
2024-12-20 19:38:13 +08:00
yyforyongyu
77b2fa0271
lntest: make sure policies are populated in AssertChannelInGraph 2024-12-20 19:38:13 +08:00
yyforyongyu
c97c31a70b
lntest: increase node start timeout and payment benchmark timeout 2024-12-20 19:38:12 +08:00
yyforyongyu
efe81f2d3c
itest: track and skip flaky tests for windows
To make the CI indicative, we now starting tracking the flaky tests
found when running on Windows. As a starting point, rather than ignore
the windows CI entirely, we now identify there are cases where lnd can
be buggy when running in windows.

We should fix the tests in the future, otherwise the windows build
should be deleted.
2024-12-20 19:38:12 +08:00
yyforyongyu
e79ad6e5aa
itest: further reduce block mined in tests 2024-12-20 19:38:12 +08:00
yyforyongyu
6f2e7feb94
itest: breakdown testSendDirectPayment
Also fixes a wrong usage of `ht.Subtest`.
2024-12-20 19:38:12 +08:00
yyforyongyu
c7b8379602
itest: break down channel fundmax tests 2024-12-20 19:38:12 +08:00
yyforyongyu
37b8210f37
itest: break down taproot tests 2024-12-20 19:38:12 +08:00
yyforyongyu
efae8ea56f
itest: break down single hop send to route
Also removed the duplicate test cases.
2024-12-20 19:38:11 +08:00
yyforyongyu
c029f0a84f
itest: break down basic funding flow tests 2024-12-20 19:38:11 +08:00
yyforyongyu
c58fa01a66
itest: break down wallet import account tests 2024-12-20 19:38:11 +08:00
yyforyongyu
31aada65a4
itest: break down channel backup restore tests 2024-12-20 19:38:11 +08:00
yyforyongyu
7b1427a565
itest: break down payment failed tests 2024-12-20 19:38:11 +08:00
yyforyongyu
3319d0d983
itest: break down open channel fee policy 2024-12-20 19:38:11 +08:00
yyforyongyu
21c5d36812
itest: break down scid alias channel update tests 2024-12-20 19:38:10 +08:00
yyforyongyu
04a15039d7
itest: break all multihop test cases 2024-12-20 19:38:10 +08:00
yyforyongyu
a76ff79adc
itest: break down utxo selection funding tests 2024-12-20 19:38:10 +08:00
yyforyongyu
b1cb819f07
itest: break down channel restore commit types cases 2024-12-20 19:38:10 +08:00
yyforyongyu
5663edfc2f
itest: break remote signer into independent cases
So the test can run faster.
2024-12-20 19:38:10 +08:00
yyforyongyu
cca2364097
itest: optimize blocks mined in testGarbageCollectLinkNodes
There's no need to mine 80ish blocks here.
2024-12-20 19:38:10 +08:00