Commit graph

18259 commits

Author SHA1 Message Date
Yong
7adc900667
Merge pull request #9448 from yyforyongyu/sweeper-fix
sweep: properly handle failed sweeping txns
2025-02-20 14:33:42 +08:00
yyforyongyu
5dd45660cd
sweep: fix error logging 2025-02-13 23:14:57 +08:00
yyforyongyu
a099261726
sweep: refactor IsOurTx to not return an error
Before this commit, the only error returned from `IsOurTx` is when the
root bucket was not created. In that case, we should consider the tx to
be not found in our db, since technically our db is empty.

A future PR may consider treating our wallet as the single source of
truth and query the wallet instead to check for past sweeping txns.
2025-02-13 23:14:57 +08:00
yyforyongyu
618edabd30
docs: add release notes 2025-02-13 23:14:57 +08:00
yyforyongyu
6130317ee0
itest: split up force close tests
So we can focus on testing normal flow vs persistence flow.
2025-02-13 23:14:57 +08:00
yyforyongyu
d64c3aee7f
sweep: make sure recovered inputs are retried
Previously, when a given input is found spent in the mempool, we'd mark
it as Published and never offer it to the fee bumper. This is dangerous
as the input will never be fee bumped. We now fix it by always
initializing the input with state Init, and only use mempool to check
for fee and fee rate.

This changes the current restart behavior - as previously when a
sweeping tx is broadcast, the node shuts down, when it starts again, the
input will be offered to the sweeper again, but not to the fee bumper,
which means the sweeping tx will stay in the mempool with the last-tried
fee rate. After this change, after a restart, the input will be swept
again, and the fee bumper will monitor its status. The restart will also
behave like a fee bump if there's already an existing sweeping tx in the
mempool.
2025-02-13 23:14:57 +08:00
yyforyongyu
e8a37e1414
sweep: signal tx in markInputFatal
This commit adds the failed tx to the result when marking the input as
fatal, which is used in the commit resolver when handling revoked
outputs.
2025-02-13 23:14:57 +08:00
yyforyongyu
81987f050d
sweep: handle missing inputs during fee bumping
This commit handles the case when the input is missing during the RBF
process, which could happen when the bumped tx has inputs being spent by
a third party. Normally we should be able to catch the spend early via
the spending notification and never attempt to fee bump the record.
However, due to the possible race between block notification and spend
notification, this cannot be guaranteed. Thus, we need to handle the
case during the RBF when seeing a `ErrMissingInputs`, which can only
happen when the inputs are spent by others.
2025-02-13 23:14:56 +08:00
yyforyongyu
0f6fbc9a3b
sweep: refactor handleInitialTxError and createAndCheckTx
This commit refactors `handleInitialTxError` and `createAndCheckTx` to
take a `monitorRecord` param, which prepares for the following commit
where we start handling missing inputs.
2025-02-13 23:14:56 +08:00
yyforyongyu
0dae88acc2
sweep: add createUnknownSpentBumpResult
A minor refactor to break the method `handleUnknownSpent` into two
steps, which prepares the following commit where we start handling
missing inputs.
2025-02-13 23:14:56 +08:00
yyforyongyu
7ae76de726
sweep: add method handleReplacementTxError
This is a minor refactor so the `createAndPublishTx` flow becomes more
clear, also prepares for the following commit where we start to handle
missing inputs.
2025-02-13 23:14:56 +08:00
yyforyongyu
5a358d0f69
sweep: retry sweeping inputs upon TxUnknownSpend
We now start handling `TxUnknownSpend` in our sweeper to make sure the
failed inputs are retried when possible.
2025-02-13 23:14:56 +08:00
yyforyongyu
7f8cbea518
sweep: start tracking inputs spent by unknown tx
This commit adds a new field `InputsSpent` to the `BumpResult` so they
can be used to track inputs spent by txns not recoginized by the fee
bumper.
2025-02-13 23:14:56 +08:00
yyforyongyu
e5eb3a9d63
itest: add fee replacement test 2025-02-13 23:14:55 +08:00
yyforyongyu
53f84ed430
sweep: rename methods for clarity
We now rename "third party" to "unknown" as the inputs can be spent via
an older sweeping tx, a third party (anchor), or a remote party (pin).
In fee bumper we don't have the info to distinguish the above cases, and
leave them to be further handled by the sweeper as it has more context.
2025-02-13 23:14:33 +08:00
yyforyongyu
8a78127876
sweep: remove dead code and add better logging 2025-02-13 23:14:33 +08:00
yyforyongyu
59164959a3
sweep: handle unknown spent in processRecords
This commit refactors the `processRecords` to always handle the inputs
spent when processing the records. We now make sure to handle unknown
spends for all backends (previously only neutrino), and rely solely on
the spending notification to give us the onchain status of inputs.
2025-02-13 23:14:32 +08:00
yyforyongyu
26b461b8dc
sweep: add a new event TxUnknownSpend 2025-02-13 23:14:32 +08:00
yyforyongyu
12e4a2b558
sweep: add method getSpentInputs
To track the input and its spending tx, which will be used later to
detect unknown spends.
2025-02-13 23:14:32 +08:00
Oliver Gugger
2ec972032e
Merge pull request #9517 from yyforyongyu/fix-coverage
workflows: skip coverage error in job `finish`
2025-02-13 09:11:57 -06:00
yyforyongyu
2b0c25e31d
workflows: skip coverage error in job finish
Also updates the coverage action used.
2025-02-13 22:05:38 +08:00
Oliver Gugger
d5ac05ce87
Merge pull request #9501 from yyforyongyu/getinfo-blockheight
rpcserver: check `blockbeatDispatcher` when deciding `isSynced`
2025-02-13 04:17:45 -06:00
Yong
9c2c95d46f
Merge pull request #9478 from ellemouton/graph3
discovery+graph:  move funding tx validation to the gossiper
2025-02-13 15:00:14 +08:00
Elle Mouton
e5db0d6314
graph+discovery: move funding tx validation to gossiper
This commit is a pure refactor. We move the transaction validation
(existence, spentness, correctness) from the `graph.Builder` to the
gossiper since this is where all protocol level checks should happen.
All tests involved are also updated/moved.
2025-02-12 15:48:08 +02:00
Elle Mouton
39bb23ea5e
discovery: lock the channelMtx before making the funding script
As we move the funding transaction validation logic out of the builder
and into the gossiper, we want to ensure that the behaviour stays
consistent with what we have today. So we should aquire this lock before
performing any expensive checks such as building the funding tx or
valdating it.
2025-02-12 13:59:09 +02:00
Oliver Gugger
693b3991ee
Merge pull request #9508 from ellemouton/ignoreSendCoverageFail
.github: ignore Send coverage errors
2025-02-12 05:23:29 -06:00
Elle Mouton
7853e36488
graph+discovery: calculate funding tx script in gossiper
In preparation for an upcoming commit which will move all channel
funding tx validation to the gossiper, we first move the helper method
which helps build the expected funding transaction script based on the
fields in the channel announcement. We will still want this script later
on in the builder for updating the ChainView though, and so we pass this
field along with the ChannelEdgeInfo. With this change, we can remove
the TapscriptRoot field from the ChannelEdgeInfo since the only reason
it was there was so that the builder could reconstruct the full funding
script.
2025-02-12 13:15:54 +02:00
Elle Mouton
8a07bb0950
discovery: prepare tests for preparing the mock chain
Here, we add a new fundingTxOption modifier which will configure how we
set-up expected calls to the mock Chain once we have moved funding tx
logic to the gossiper. Note that in this commit, these modifiers don't
yet do anything.
2025-02-12 13:15:54 +02:00
Elle Mouton
22e391f055
discovery: add AssumeChannelValid config option
in preparation for later on when we need to know when to skip funding
transaction validation.
2025-02-12 13:15:54 +02:00
Elle Mouton
00f5fd9b7f
graph: add IsZombieEdge method
This is in preparation for the commit where we move across all the
funding tx validation so that we can test that we are correctly updating
the zombie index.
2025-02-12 13:15:54 +02:00
Elle Mouton
870c865763
graph: export addZombieEdge and rename to MarkZombieEdge
The `graph.Builder`'s `addZombieEdge` method is currently called during
funding transaction validation for the case where the funding tx is not
found. In preparation for moving this code to the gossiper, we export
the method and add it to the ChannelGraphSource interface so that the
gossiper will be able to call it later on.
2025-02-12 13:15:53 +02:00
Elle Mouton
a67df6815c
.github: ignore Send coverage errors
Sometimes only the "Send coverage" step of a CI job will fail. This
commit turns this step into a "best effort" step instead so that it
does not block a CI job from passing.

It can, for example, often happen that a single job is re-run from the
GH UI, it then passes but the "Send coverage" step fails due to the
"Can't add a job to a build that is already closed." error meaning that
the only way to get the CI step to pass is to re-push and retrigger a
full CI run.
2025-02-12 12:34:02 +02:00
Oliver Gugger
4dbbd837c0
Merge pull request #9496 from ellemouton/graph4
graph: remove redundant iteration through a node's persisted channels
2025-02-12 03:36:32 -06:00
Olaoluwa Osuntokun
7b294311bc
Merge pull request #9150 from yyforyongyu/fix-stuck-payment
routing+htlcswitch: fix stuck inflight payments
2025-02-11 18:02:02 -08:00
yyforyongyu
759dc2066e
docs: update release notes 2025-02-12 09:48:03 +08:00
yyforyongyu
59759f861f
rpcserver: check blockbeatDispatcher when deciding isSynced
This commit changes `GetInfo` to include `blockbeatDispatcher`'s current
state when deciding whether the system is synced to chain. Previously we
check the best height against the wallet and the channel graph, we
should also do this to the blockbeat dispatcher to make sure the
internal consumers are also synced to the best block.
2025-02-12 09:48:02 +08:00
yyforyongyu
89c4a8dfd7
chainio: add method CurrentHeight
Add a new method `CurrentHeight` to query the current best height of the
dispatcher.
2025-02-12 09:48:02 +08:00
Oliver Gugger
fbc668ca53
Merge pull request #9500 from MPins/release-note-0.20.0
doc: creating release-notes-0.20.0.md
2025-02-11 02:22:38 -06:00
Elle Mouton
5c2c00e414
graph/db: remove GraphCacheNode interface
With the previous commit, the AddNode method was removed and since that
was the only method making use of the ForEachChannel on the
GraphCacheNode interface, we can remove that method. Since the only two
methods left just expose the node's pub key and features, it really is
not required anymore and so the entire thing can be removed along with
the implementation of it.
2025-02-11 08:19:33 +02:00
MPins
097c262341 doc: creating file release-notes-0.20.0.md 2025-02-10 23:50:04 -03:00
Elle Mouton
90179b651e
graph/db: remove unnecessary AddNode method on GraphCache
The AddNode method on the GraphCache calls `AddNodeFeatures` underneath
and then iterates through all the node's persisted channels and adds
them to the cache too via `AddChannel`.

This is, however, not required since at the time the cache is populated
in `NewChannelGraph`, the cache is populated will all persisted nodes
and all persisted channels. Then, once any new channels come in, via
`AddChannelEdge`, they are added to the cache via AddChannel. If any new
nodes come in via `AddLightningNode`, then currently the cache's AddNode
method is called which the both adds the node and again iterates through
all persisted channels and re-adds them to the cache. This is definitely
redundent since the initial cache population and updates via
AddChannelEdge should keep the cache fresh in terms of channels.

So we remove this for 2 reasons: 1) to remove the redundent DB calls and
2) this requires a kvdb.RTx to be passed in to the GraphCache calls
   which will make it hard to extract the cache out of the CRUD layer
and be used more generally.

The AddNode method made sense when the cache was first added in the
code-base
[here](369c09be61 (diff-ae36bdb6670644d20c4e43f3a0ed47f71886c2bcdf3cc2937de24315da5dc072R213))
since then during graph cache population, nodes and channels would be
added to the cache in a single DB transaction. This was, however,
changed [later
on](352008a0c2)
to be done in 2 separate DB calls for efficiency reasons.
2025-02-10 17:10:53 +02:00
Oliver Gugger
d10ab03b75
Merge pull request #9480 from ellemouton/autopilotRefactor
graph+autopilot: remove `autopilot` access to raw `graphdb.ChannelGraph`
2025-02-10 09:07:47 -06:00
Oliver Gugger
2a0dca77a0
Merge pull request #9495 from ziggie1984/fix-graphbuilder-flake
fix graphbuilder flake
2025-02-10 08:43:59 -06:00
ziggie
6373d84baf
graph: fix flake in unit test 2025-02-10 14:07:04 +01:00
Oliver Gugger
6eb8f1f6e3
Merge pull request #9477 from ellemouton/graph2
discovery+graph: various preparations for moving funding tx validation to the gossiper
2025-02-10 05:58:05 -06:00
Oliver Gugger
6f312d457f
Merge pull request #9451 from Juneezee/minmax
refactor: replace min/max helpers with built-in min/max
2025-02-10 05:57:34 -06:00
Oliver Gugger
6bf6603fb8
Merge pull request #9492 from yyforyongyu/itest-flake-interceptor
itest: fix flake in `testForwardInterceptorRestart`
2025-02-10 05:50:13 -06:00
Elle Mouton
3d0ae966c8
docs: update release notes 2025-02-10 09:46:15 +02:00
Elle Mouton
e7988a2c2b
autopilot: remove access to *graphdb.ChannelGraph
Define a new GraphSource interface that describes the access required by
the autopilot server. Let its constructor take this interface instead of
a raw pointer to the graphdb.ChannelGraph.
2025-02-10 09:46:15 +02:00
Elle Mouton
9b86ee53db
graph+autopilot: let autopilot use new graph ForEachNode method
Which passes a NodeRTx to the call-back instead of a `kvdb.RTx`.
2025-02-10 09:46:15 +02:00