Fixes#6396.
This commit fixes a panic that occurred when trying to sign for a
Taproot output without specifying the full UTXO information for each
input. Instead of panicking an error is now returned.
In this commit, we start to ignore the option to allow the caller to use
the legacy onion payload. The new payload is much more flexible and
efficient, so there's really no reason to still use it, other than for
backwards compatibility tests. Our existing tests that exercise the
legacy feature uses a build tag, which forces nodes to not advertise the
new payload format, which then forces path finding to include the legacy
payload, so we can be confident that route is still being tested.
The existence of this option (which actually makes the TLV payload
opt-in for `SendToRoute` users) makes it harder to remove it from the
protocol all together. With this PR, we take a step forward to allowing
such a change which is being tracked on the spec level at:
https://github.com/lightning/bolts/pull/962.
In a future release, we'll move to remove the field all together.
Ignoring the field today doesn't seem to have any clear downsides, as
most payments always include the MPP payload (due to payment secrets),
so this shouldn't impact users in a significant way.
The `log.go` file of each subserver is a copy and paste of the others.
All of them have the same typo, an extra space in a comment, and the
the definition of a function not used anywhere.
Was discovered in a race unit test in lightning node connect that uses
the websocket proxy to connect to the hashmail server on the client
side.
By not shadowing the err variable we end up reading and writing to the
same variable from two different goroutines, which causes the data race.
Because Taproot key spend only spends don't allow us to re-construct the
spent pkScript from the witness alone, we cannot support registering
spend notifications for v1 pkScripts only. We instead require the
outpoint to be specified. This commit makes it possible to only match by
outpoint and also adds an itest for it.
To help debug remote signing issues, it's helpful to get the raw PSBT
that failed to be parsed. This is necessary since serializing an invalid
PSBT is allowed and the checks only fail when trying to de-serialize
such an invalid packet.
In this commit, we add a new field to the WalletBalance call that
permits users to account for the set of outputs that may be locked due
to a pending transaction. Without this field any time users locked
outputs for things like PSBT signing, then they disappear from the
WalletBalance call, which may cause a panic.
In this commit we move the tracking of the outstanding intercepted htlcs
to InterceptableSwitch. This is a preparation for making the htlc
interceptor required.
Required interception involves tracking outstanding htlcs across
multiple grpc client sessions. The per-session routerrpc
forwardInterceptor object is therefore no longer the best place for
that.
This commit was previously split into the following parts to ease
review:
- 2d746f68: replace imports
- 4008f0fd: use ecdsa.Signature
- 849e33d1: remove btcec.S256()
- b8f6ebbd: use v2 library correctly
- fa80bca9: bump go modules
Previously, we'd always add up to the maximum number of hop hints
(and beyond!) when selecting hop hints in our first pass. This
change updates hop hint selection to always stick to our hop hint
limit, and to the "hop hint factor" that we scale our invoices by.
This change will result in selecting fewer channels in our first
pass if their total inbound capacity reaches our hop hint factor.
This prevents us from revealing as many private channels as
before, but has the downside of providing fewer options for
payers.
In this commit, we increase the legacy fee limit threshold (the amount
below which we'll allow 100% of funds to go to fees for the non-v2 RPC
calls) from 50 sats to 1k sats.
The API doc generator failed after merging #6149 because of the `lncli
importgraph` tag in the proto file.
We could also fix the API doc generator by adding the "dev" build tag,
but not including any dev only features in the API docs is probably the
preferred path.
Update SelectHopHints to return a set of hop hints that can be
converted to route hints / functional options by the caller. This
change allows external code to call SelectHopHints to get a set of
route hints and use them as it likes, rather than limiting the
return value to functional options for invoice creation.
When updating the channel routing policy, we encounter an inaccurate
precision error when calculating the routing fee. The issue stems from
the way the IEEE 754 standard works.
The solution here is to add a uint64 parameter (as mentioned in the
issue) and keep the float64 fee_rate parameter but rounding the product
of the base and fee rate.
This commits adds the devrpc package which implements a subserver that
adds clean separation for RPC calls useful for development and
debugging. This subserver is only compiled in if the dev tag is set.
Furthermore the commit adds the devrpc.ImportGraph call which can
import a graph dump obtained from another node by calling DescribeGraph.
Since the graph dump does not include the auth proofs, the imported
channels will be considered private.
This commit adds the `force` flag to the `XImportMissionControl` RPC
which allows skipping rules around the pair import except for what is
mandatory to make values meaningful. This can be useful for when clients
would like to forcibly override MC state in order to manipulate routing
results.
We use SignOutputRaw which expects a witness script being set, even for
P2WKH. There is a special case in SignOutputRaw for the case where the
script is a p2wkh script, then the input script is reconstructed
correctly for the sighash.
This commit fixes a potential blocking when notifying invoice updates.
When a new subscription client is created followed by an immediate
cancel, it's likely the client will be removed from the registry's
map(noop) and then added to its map again. This subscription will then
be kept in registry until lnd is restarted. Another more serious issue
is when multiple subscriptions are made for the same invoice, when the
above case happens, other subscriptions may never send invoice updates
because a previous client has a stopped notification queue that blocks
following notifications.
when payment_hash or final_cltv_delta and payment_request was set, the error message showed that the parameters shouldn't be set with dest instead of payment_request
[skip ci]
This commit adds a new unique intercept message ID that is different for
each message sent to an RPC middleware. This will allow us to re-purpose
the existing request ID field into its original intention, which was to
have a unique ID for a full gRPC request/response pair or gRPC stream as
a whole.
Clarification that KeyDescriptor describes the public key in
raw_key_bytes and the key_loc identifies the private key. Helps
clarify responses from calls like DeriveKey and DeriveNextKey from
WalletKit.
Fixes#5899
This commit ensures htlcs currently held by the interceptor
Are not sent again. This prevents potential races in the user code that
may lead to loosing funds.
In this commit, we add a new `LookupInvoiceV2` method attached to the
invoice RPC sub-server. Compared to the existing version, this version
allows an invoice to be looked up by an invoice ref. This enables users
to query an AMP invoice based on a specific _set ID_, so they can get
the information related to the set of cumulative settles to that
invoices.
In order to support the full range of on-chain functionality, including
importing watch-only accounts in the watch-only instance, we need to
forward some calls like creating new addresses or importing accounts to
the remote signing instance.
To simplify the API surface of a remote signer even more, we refactor
the SignMessage and SignMessageCompact calls to only accept a key
locator as we always know what key we're using for signing anyway.
To make it possible to use a remote lnrpc server as a signer for our
wallet, we need to change our main interface to sign the message instead
of the message's digest. Otherwise we'd need to alter the
lnrpc.SignMessage RPC to accept a digest instead of only the message
which has security implications.
To make it possible to use a remote signrpc server as a signer for our
wallet, we need to change our main interface to sign the message instead
of the message's digest. Otherwise we'd need to alter the
signrpc.SignMessage RPC to accept a digest instead of only the message
which has security implications.
The FundingPsbtFinalize step is a safety measure that assures the final
signed funding transaction has the same TXID as was registered during
the funding flow and was used for the commitment transactions.
This step is cumbersome to use if the whole funding process is completed
external to lnd. We allow the finalize step to be skipped for such
cases. The API user/script will need to make sure things are verified
(and possibly cleaned up) properly.
With go 1.17 a change to the build flags was implemented:
https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
The formatter now automatically adds the forward-compatible build tag
format and the linter checks for them, so we need to include them in our
code.
With the middleware handler in place, we now need to add a new gRPC
interceptor to the interceptor chain that will send messages to the
registered middlewares for each event that could be of interest to them.
With this commit we introduce the concept of RPC middleware: A mechanism
similar to the existing channel or HTLC interceptors but this time for
gRPC messages themselves.
An RPC middleware can register itself to the main RPC server to get
notified each time a new gRPC request comes in, a gRPC response is sent
back or a streaming RPC is connected. The middleware can
validate/inspect incoming requests and modify/overwrite outgoing
responses.
Since this also opens the door for malicious software to interfere with
lnd in a negative way, we bind everything to macaroons with custom
caveat conditions: A middleware declares upon registration which custom
caveat name it can handle. Only client requests that send a macaroon
with that custom caveat will then be given to the middleware for
inspection. The only exception is if the middleware instead registers
to use the read-only mode. In that mode it will be able to intercept
all requests/responses, even those not made with a special encumbered
macaroon. But the middleware won't be able to alter responses in the
read-only mode. Therefore requests with the default, unencumbered macaroons
can never be modified by any middleware.
Fixes#5680.
To make sure we're always reading from the WebSocket connection, we need
to always have an ongoing (but blocking) conn.ReadMessage() call going
on. To achieve this, we do the read in a separate goroutine and write to
a buffered channel. That way we can always read the next message while
the current one is being forwarded. This allows incoming ping messages
to be received and processed which then leads to the deadlines to be
extended correctly.
The request context was not properly used to pass it along to the gRPC
endpoint which caused streaming calls to still be active on the gRPC
side even if the WS side already hung up.
We also issue an explicit close on the forwarding writer to signal when
the WS side was closed.
The RPC DeletePayment allows deleteing single payment from its ID. When calling with `FailedHtlcsOnly` set in the request only failed HTLCs of this payment will be deleted.
selectHopHints is the function which constructs hophints otherwise found
in an invoice created with the private flag.
In this commit, we expose that functionality a little more to workaround
needing to create an invoice to retrieve routing hints. Of course, this
is not a perfect solution as the functionality is exposed exclusively to
go users.
We'll want to re-use the PSBT funding functionality from the wallet kit
sub server in the main RPC server. To be able to dynamically obtain an
instance of the wallet kit server we need to know its name under which
it registers itself in the list of sub server. We export the name so we
don't have to hard code it in the main server.
This field will be examined later down the stack along with the set of
feature bits to determine if explicit channel commitment type
negotiation is possible or not.
As a preparation to not have a local and remote version of the database
around anymore, we rename the variables into what their actual function
is. In case of the RPC server we even directly use the channel graph
instead of the DB instance. This should allow us to extract the channel
graph into its own, separate database (perhaps with better access
characteristics) in the future.
To avoid a naming conflict with etcd, we rename our very generic
rpc.proto to lightning.proto to match the service name that's declared
within. This will break many external tutorials and possibly also our
API docs but the change needs to be done eventually.
As a preparation for the migration to the grpc-gateway/v2 library we
declare each service's REST annotations in its own file. This is
optional in the v1 library but mandatory in v2.
This commit changes missioncontrol's store update from per payment to
every second. Updating the missioncontrol store on every payment caused
gradual slowdown when using etcd.
We also completely eliminate the use of the cursor, further reducing
the performance bottleneck.
Until now, clients of SubscribeHTLCEvents didn't have access to the settled preimage. The API allows to intercept forward event and to be updated on forward events however the forward+settle event does not include the payment preimage. This pr changes allows it.
Increases the default MPP expiry from 1 hour to 1 day. For the new AMP
invoices, we increase the interval to 1 month. The longer time frames
for AMP invoices is used so that the invoice can be pseudo reused as
implemented in the prior commit.
The BOLT 11 default of 1 hour is still preserved whenever the field is
missing in the payment request itself, but as of this commit the field
will always be set by lnd.
This permits an AMP invoice to be "pseudo-reusable", where the invoice
paramters can be used multiple times so long as a new payment address is
supplied. This prevents additional round trips between payer and payee
to obtain a new invoice, even though the payments/invoices won't be
logically associated via the RPC interface like they would when the full
reusable invoices are deployed.
An often requested feature is to use the abandonchannel API in regular
builds and not only dev builds to get rid of stuck channels that had
their funding transaction invalidated.
The initial reason for putting the call behind the build flag was a
safety concern to make sure nobody uses this on active channels by
accident.
Fixes#5307.
Corrects some typos in the REST annotations for the routerrpc subserver.
Unfortunately these mistakes aren't caught by the current version of the
grpc-gateway library. But we intend to update to v2 soon which will
output alerts as mentioned in #5307.
With this patch, we'll fail out earlier in the cycle in case of
some wonky parameters, and not leave zombie payments in the router
which currently are not cleaned up.
This commit adds a new "waiting to start" state which may be used to
query if we're still waiting to become the cluster leader. Once leader
we advance the state to "wallet not exist" or "wallet locked" given
wallet availablity.
Fixes the logged error "WS: error closing upgraded conn: tls: failed to
send closeNotify alert (but connection was closed anyway): write tcp4
<ip>-><ip>: write: connection reset by peer" that is caused by the
client closing the connection on its end.
The grpc-gateway library that is used to transform REST calls into gRPC
uses a different method for reading a request body stream depending on
whether the RPC is a request-streaming one or not. We can't really find
out what kind of RPC the user is calling at runtime, so we add a new
parameter to the proxy that lists all request-streaming RPC calls.
In any case the client _has_ to send one request message initially to
kick off the request processing. Normally this can just be an empty
message. This can lead to problems if that empty message is not
expected by the gRPC server. But for the currently existing two
client-streaming RPCs this will only trigger a warning
(HTLC interceptor) or be ignored (channel acceptor).
Since we want to support AMP payment using a different unique payment
identifier (AMP payments don't go to one specific hash), we change the
nomenclature to be Identifier instead of PaymentHash.
We'd never decrement the number of pending backups upon a watchtower
accepting one, making it confusing for users to determine whether their
backups have actually been accepted. Along the way, we also rename
NumTasksReceived to NumTasksPending to better reflect its purpose.
This commit adds a []*lnrpc.NodeAddress typed node_addresses field
on the NodeUpdate message of SubscribeChannelGraph to mirror the
addresses field in the LightningNode message of DescribeGraph.
Fixes https://github.com/lightningnetwork/lnd/issues/4084
This commit deprecates/replaces the old field `sat_per_byte` with
`sat_per_vbyte`. While the old field suggests sat per byte, it’s
actually using sat per virtual byte. We use the Hidden param to hide all
the deprecated flags. These flags won't show up in help menu onwards,
while stay valid that can be passed from cli. Thus bash scripts
referencing these fields won't be broken.
In order to be able to register the subservers with the root grpc server
before we have all dependencies available, we wrap them in an
GrpcHandler struct. This struct will initially hold an empty reference
to the subservers, which allows us to register with the GRPC server, and
later populate and create the subserver instance.
This allows to use FowardingHistory rpc method to receive the data exactly
as it's stored in lnd and to synchronize incrementally the history to an
external database.
We also test that legacy keysend payments are promoted to AMP payments
on the receiver-sdie by asserting basic properties of the fields
returned via the rpc.
In this commit, we raise the default value for the `MaxParts` field from
1 to 16. This change was motivated by the fact that many users either
forget, or don't even know this field is there in the first place. A
value of 16 was chosen rather arbitraliy (other than power of 2). In
the future, we should tune this value based on the expected number of
payment attempts for a given payment amount.
This allows users to see progress whenever the docker image is
[re]built, and (esp on non-linux hosts) track the size of the build
context being uploaded. Currently no output is displayed, so it's hard
to attribute the source of latency, e.g. network latency, building
layers, a large work directory, etc.
Now that we have a base docker image that has all our RPC compilation
dependencies installed, we can also run the mobile RPC compilation
there. This removes the need to install falafel and goimports on the
local machine.
This commit aims to make it easier for developers to compile our
protobuf definitions. They now only need to have docker installed
instead of a whole set of binaries and libraries all pinned to very
specific versions.
