Commit Graph

12091 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
9264185f5b
Merge pull request #5101 from guggero/macaroon-interceptor
Add macaroon based RPC middleware interceptor
2021-09-20 19:15:04 -07:00
Olaoluwa Osuntokun
ac5f88bae8
Merge pull request #5618 from Crypt-iQ/coop_switch_sync_08092021
multi: optimistically shutdown link during coop close
2021-09-20 18:09:07 -07:00
Olaoluwa Osuntokun
5e6532594c
Merge pull request #5683 from guggero/websocket-write-deadline
lnrpc: Fix WebSocket write deadline not being extended
2021-09-20 17:07:41 -07:00
Oliver Gugger
3f7909b48f
doc: add release notes 2021-09-20 17:04:39 +02:00
Oliver Gugger
4b43e977b2
lntest: add RPC middleware itests 2021-09-20 17:04:39 +02:00
Oliver Gugger
0cc260bfc7
itest: add _test suffix to test file 2021-09-20 17:04:38 +02:00
Oliver Gugger
efe5f6ae90
multi: add RPC middleware interception
With the middleware handler in place, we now need to add a new gRPC
interceptor to the interceptor chain that will send messages to the
registered middlewares for each event that could be of interest to them.
2021-09-20 17:04:29 +02:00
Oliver Gugger
75ca574790
lnrpc+rpcperms: add middleware handler
With this commit we introduce the concept of RPC middleware: A mechanism
similar to the existing channel or HTLC interceptors but this time for
gRPC messages themselves.
An RPC middleware can register itself to the main RPC server to get
notified each time a new gRPC request comes in, a gRPC response is sent
back or a streaming RPC is connected. The middleware can
validate/inspect incoming requests and modify/overwrite outgoing
responses.

Since this also opens the door for malicious software to interfere with
lnd in a negative way, we bind everything to macaroons with custom
caveat conditions: A middleware declares upon registration which custom
caveat name it can handle. Only client requests that send a macaroon
with that custom caveat will then be given to the middleware for
inspection. The only exception is if the middleware instead registers
to use the read-only mode. In that mode it will be able to intercept
all requests/responses, even those not made with a special encumbered
macaroon. But the middleware won't be able to alter responses in the
read-only mode. Therefore requests with the default, unencumbered macaroons
can never be modified by any middleware.
2021-09-20 13:12:49 +02:00
Oliver Gugger
918e021177
config+sample-lnd.conf+lncfg: add RPC middleware config 2021-09-20 13:05:47 +02:00
Oliver Gugger
8509d92070
log+rpcperms: add custom RPCP logger to rpcperms package
The custom RPC middleware logic that we are going to add in the next
commits will need to log under their own sub logger so we add one with a
new subsystem name.
2021-09-20 13:05:47 +02:00
Oliver Gugger
7b822f41df
lncli: add custom caveats to bakemacaroon
With the new condition and checker in place, we can give the end user
the ability to add such a custom caveat to a baked macaroon.
There won't be an RPC counterpart for this operation since all first party
caveats currently are only added on the client side.
2021-09-20 13:05:47 +02:00
Oliver Gugger
045765111a
multi: use safe copy for macaroons
Fixes #4383 by adding a new SafeCopyMacaroon function that correctly
clones all caveats and prevents modifications on the copy from affecting
the original.
2021-09-20 13:05:46 +02:00
Oliver Gugger
538175f487
macaroons: add custom caveat contraint and checker
The way the macaroon bakery library lnd uses works is that one has to
register a Checker method for each caveat name that should be supported.
Since we want to allow fully customizable custom caveats we add another
layer of naming to the caveat by splitting the condition of the "outer"
caveat into two pieces, the custom caveat name and the actual custom
caveat condition.
The custom Checker function only checks that the format is correct and
that there is a handler available for a custom condition. It does not
check the condition itself, however. If the passed in acceptor signals
acceptance of a custom caveat then the bakery accepts the macaroon as a
whole (given its signature, standard caveats and permissions are all
correct) and assumes that another component down the line will make sure
the actual custom condition of a caveat is valid.
2021-09-20 13:05:46 +02:00
Oliver Gugger
96ea4bf05e
rpcserver+macaroons: extract RawMacaroonFromContext
We'll re-use the code for extracting a macaroon from a request context
later on so we extract it into its own exported function.
2021-09-20 13:05:33 +02:00
Oliver Gugger
4b7452a35e
lnrpc+itest: fix write deadline issue with WS ping
Fixes #5680.
To make sure we're always reading from the WebSocket connection, we need
to always have an ongoing (but blocking) conn.ReadMessage() call going
on. To achieve this, we do the read in a separate goroutine and write to
a buffered channel. That way we can always read the next message while
the current one is being forwarded. This allows incoming ping messages
to be received and processed which then leads to the deadlines to be
extended correctly.
2021-09-20 12:21:03 +02:00
Oliver Gugger
5f94ebbd7d
lnrpc: use request context in WebSocket proxy
The request context was not properly used to pass it along to the gRPC
endpoint which caused streaming calls to still be active on the gRPC
side even if the WS side already hung up.
We also issue an explicit close on the forwarding writer to signal when
the WS side was closed.
2021-09-20 12:07:46 +02:00
Oliver Gugger
29a8661517
Merge pull request #5640 from bhandras/kvdb-prefetch
kvdb+channeld: extend `kvdb` with `Prefetch` for prefetching buckets in one go and speed up payment control by prefetching payments on hot paths
2021-09-20 09:42:18 +02:00
Oliver Gugger
d9f0f07142
Merge pull request #5748 from Roasbeef/flake-block-diff
routing: add wait.NoError to TestBlockDifferenceFix assertion [skip ci]
2021-09-20 08:25:27 +02:00
Olaoluwa Osuntokun
650827aade
routing: add wait.NoError to TestBlockDifferenceFix assertion
This fixes a flake I've seen in the wild lately:
```
--- FAIL: TestBlockDifferenceFix (0.01s)
    router_test.go:4335: height should have been updated to 5, instead got 4
FAIL
FAIL	github.com/lightningnetwork/lnd/routing	3.865s
FAIL
```

We wrap things in an assertion loop to ensure that timing quirks don't
cause the test to fail sporadically.
2021-09-17 17:02:03 -07:00
Olaoluwa Osuntokun
6be472eb98
Merge pull request #5737 from naveensrinivasan/naveen/feat/fix-jwt-osv
mod : Replace for OSV GO-2020-0017 JWT
2021-09-17 16:30:06 -07:00
Olaoluwa Osuntokun
854d8bcd39
Merge pull request #5722 from xanoni/dedupe-pubkey-log-output
server.go: dedupe pubkey output in debug/log msgs
2021-09-17 16:28:15 -07:00
Oliver Gugger
5f43ef5816
Merge pull request #5751 from guggero/use-kvdb-1.0.2
mod: update kvdb to kvdb/v1.0.2 [skip ci]
2021-09-17 15:19:17 +02:00
Oliver Gugger
63233ff66c
mod: update kvdb to kvdb/v1.0.2 [skip ci] 2021-09-17 13:08:08 +02:00
Oliver Gugger
583ccfeca4
Merge pull request #5611 from yyforyongyu/itest-flake-chan-open
itest: fix test flakes from open channel not found and tx not found in mempool
2021-09-17 09:51:17 +02:00
yyforyongyu
87ab4de149
docs: add release note 2021-09-17 07:51:07 +08:00
yyforyongyu
66dae6ecf7
itest: put node.CloseChannel inside wait 2021-09-17 07:50:44 +08:00
yyforyongyu
e0e1bfb935
lnd: change start/stop order of subsystems
This commit adds the part of the changes made in this PR:
https://github.com/lightningnetwork/lnd/pull/1783. The origin PR is
quite outdated, instead of rebasing it the relevant changes are taken
out and put into this commit.
2021-09-17 07:50:44 +08:00
yyforyongyu
64f4e21ab4
contractcourt+lnd: add debug log 2021-09-17 07:50:44 +08:00
yyforyongyu
7038d0e5c8
itest: fix typo 2021-09-17 07:50:43 +08:00
yyforyongyu
cdec34c5f7
itest: remove the method waitForChannelUpdate
This commit removes the method waitForChannelUpdate, and uses
node.WaitForChannelPolicyUpdate instead.
2021-09-17 07:50:43 +08:00
yyforyongyu
06fa17513c
itest: move tests by their category 2021-09-17 07:50:43 +08:00
yyforyongyu
87c13d31b4
itest: watch channel policy updates in harness node 2021-09-17 07:50:43 +08:00
yyforyongyu
d2277ac915
itest: replace chanOpen bool with chanWatchType 2021-09-17 07:50:42 +08:00
yyforyongyu
a58543d1c7
itest: remove extra graph topology subscription 2021-09-17 07:50:42 +08:00
yyforyongyu
92cd6657c5
lntest: refactor handle close channel update 2021-09-17 07:50:42 +08:00
yyforyongyu
0701834a5d
lntest: refactor handle update open channel 2021-09-17 07:50:42 +08:00
yyforyongyu
a1024163fe
itest: add more verbose log and print node state 2021-09-17 07:50:42 +08:00
yyforyongyu
eadbd69882
routing: increase log level when notifying topology change 2021-09-17 07:50:41 +08:00
Oliver Gugger
44d73d7e3e
Merge pull request #5743 from guggero/amp-string-nil
record: fix nil pointer in log string
2021-09-16 20:16:01 +02:00
Oliver Gugger
dd3f3e7ce5
record: fix nil pointer in log string 2021-09-16 19:51:03 +02:00
naveen
8b4bbfbf61 mod : Replace for OSV GO-2020-0017 JWT
The github.com/dgrijalva/jwt-go is no longer maintained
has this CVE https://github.com/advisories/GHSA-w73w-5m7g-f7qc

But it is being used by https://github.com/etcd-io/etcd/issues/13254
which `lnd` uses.

The https://github.com/golang-jwt/jwt/releases/tag/v3.2.1 is a fork of
the same version which was 3.2.0 with the security fix.
2021-09-16 11:23:28 +00:00
xanoni
a17d161e0c server.go: dedupe pubkey output in debug/log msgs
Default human readable format of NetAddress already contains pubkey:
a329c80612/channeldb/migration/lnwire21/netaddress.go (L38-L47)
2021-09-16 04:25:44 -04:00
Oliver Gugger
7d012ae581
Merge pull request #5738 from naveensrinivasan/naveen/fix/dns
mod: bump miekg/dns to next version
2021-09-16 09:42:16 +02:00
Oliver Gugger
d9534ea108
Merge pull request #5405 from ErikEk/routing-updatechanpolicy-chan-check
routing: report invalid channels in updatechanpolicy call
2021-09-16 09:41:02 +02:00
naveen
ae6f771443 mod: bump miekg/dns to next version
The dns has https://deps.dev/go/github.com%2Fmiekg%2Fdns/v1.1.25 has
GO-2020-0012 https://storage.googleapis.com/go-vulndb/golang.org/x/crypto.json

Updated the dependency with the fix.
2021-09-15 23:11:04 +00:00
Oliver Gugger
15ec974b15
Merge pull request #5734 from bhandras/itest_parallelism
build: fix itest parallelism
2021-09-15 20:15:29 +02:00
Oliver Gugger
3c15e3d81a
Merge pull request #5686 from yyforyongyu/fix-health-check
healthcheck: quit checking when max attempts are reached
2021-09-15 17:41:44 +02:00
yyforyongyu
3b4965d776
docs: update release note for healthcheck fix 2021-09-15 19:52:25 +08:00
yyforyongyu
d19ee28089
healthcheck: stop checking when max attempts are reached
This commit adds a check to the returned error from calling retryCheck
such that when the max number of attempts is reached, the health check
will quit.
2021-09-15 19:52:03 +08:00
yyforyongyu
3204e2d74b
multi: add shutdown logs in subservers
This commit adds a simple shutdown to every subserver to assist
debugging.
2021-09-15 19:52:03 +08:00