Commit Graph

11014 Commits

Author SHA1 Message Date
ghubstan
de50692daa
Add rpc wallet protection endpoints
Implemented lockwallet, unlockwallet, removewalletpassword, and
setwalletpassword methods with basic error handling.

Also added basic error handling to existing getbalance method,
and removed unused BalancePresentation from CoreAPI.

TODO:  update help text
2020-04-29 19:25:23 -03:00
Chris Beams
4277e62135
Merge pull request #4199 from cbeams/refactor-rpc-server
Refactor rpc server implementation
2020-04-29 16:23:42 +02:00
Chris Beams
e03c46106b
Merge pull request #4189 from ghubstan/simple-rpc-auth
Implement simple password-based gRPC authentication
2020-04-29 16:17:04 +02:00
Christoph Atteneder
2e3e811bf6
Merge pull request #4114 from cbeams/git-lfs
Track p2p data store files using Git LFS
2020-04-29 16:15:06 +02:00
Christoph Atteneder
5fd05cf660
Merge pull request #4206 from sqrrm/permutation-fix
Permutation fix
2020-04-29 15:56:45 +02:00
Chris Beams
51fc2710ad
Track p2p data store files using Git LFS
The large binary objects in p2p/src/main/resources/ are updated on every
Bisq release with the latest network data to avoid the need for new Bisq
clients to download all of this information from the network, which
would easily overload seed nodes and generally bog down the client.

This approach works well enough for its purposes, but comes with the
significant downside of storing all of this binary data in Git history
forever. The current version of these binary objects total about 65M,
and they grow with every release. In aggregate, this has caused the
total size of the repository to grow to 360M, making it cumbersome to
clone over a low-bandwith connection, and slowing down various local Git
operations.

To avoid further exacerbating this problem, this commit sets these files
up to be tracked via Git LFS. There's nothing we can do about the 360M
of files that already exist in history, but we can ensure it doesn't
grow in this unchecked way going forward. For an understanding of how
Git LFS works, see the reference material at [1], and see also the
sample project and README at [2].

The following command was used to track the files:

    $ git lfs track "p2p/src/main/resources/*BTC_MAINNET"
    Tracking "p2p/src/main/resources/AccountAgeWitnessStore_BTC_MAINNET"
    Tracking "p2p/src/main/resources/BlindVoteStore_BTC_MAINNET"
    Tracking "p2p/src/main/resources/DaoStateStore_BTC_MAINNET"
    Tracking "p2p/src/main/resources/ProposalStore_BTC_MAINNET"
    Tracking "p2p/src/main/resources/SignedWitnessStore_BTC_MAINNET"
    Tracking "p2p/src/main/resources/TradeStatistics2Store_BTC_MAINNET"

We are using GitHub's built-in LFS service here, and it's important to
understand that there are storage and bandwidth limits there. We have
1G total storage and 1G per month of bandwidth on the free tier. We will
certainly exceed this, and so must purchase at least one "data pack"
from GitHub, possibly two. One gets us to 50G storage and bandwith.

In an attempt to avoid unnecessary LFS bandwidth usage, this commit also
updates the Travis CI build configuration to cache Git LFS files, such
that they are not re-downloaded on every CI build (see [3] and [4]
below). With that out of the way, the variable determining whether we
exceed the monthly limit is how many clones we have every month, and
there are many, though it's not clear how many are are Travis CI and how
many are users / developers.

Tracking these files via LFS means that developers will need to have Git
LFS installed in order to properly synchronize the files. If a developer
does not have LFS installed, cloning will complete successfully and the
build would complete successfully, but the app would fail when trying to
actually load the p2p data store files. For this reason, the build has
been updated to proactively check that the p2p data store files have
been properly synchronized via LFS, and if not, the build fails with a
helpful error message. The docs/build.md instructions have also been
updated accordingly.

It is important that we make this change now, not only to avoid growing
the repository in the way described above as we have been doing now for
many releases, but also because we are now considering adding yet more
binary objects to the repository, as proposed at
https://github.com/bisq-network/projects/issues/25.

[1]: https://git-lfs.github.com
[2]: https://github.com/cbeams/lfs-test
[3]: https://docs-staging.travis-ci.com/user/customizing-the-build/#git-lfs
[4]: https://github.com/travis-ci/travis-ci/issues/8787#issuecomment-394202791
2020-04-29 12:10:11 +02:00
sqrrm
92cd7ec5a6
Test permutations level by level 2020-04-29 01:00:53 +02:00
chimp1984
05106dfdac
Fix issue with shutdown
The getAllConnections() call in the while loop always returned the same
number of nodes so the timeout of 15 sec was always triggered.

We now wait for the shutdown handlers of the connections and if all are
called we run our handler. If it takes longer as our timeout of 3 sec.
the shutdown handler gets called by the timeout.
2020-04-28 15:20:31 -05:00
chimp1984
77015fea17
Add resync from resources button
We add a second button (displayed as first) to the preferences UI for
resync from latest resources. We add a warning to the resync from
genesis as it takes very long time and causes heavy network load for
seeds.
The resync button at the stateMonitor does now a resync from resources,
not from genesis.
We add handling of the UnconfirmedBsqChangeOutputList file as well.
In Filemanager we add a check if file exists.
2020-04-28 15:18:11 -05:00
chimp1984
77bd421a00
Remove dev logs 2020-04-28 13:25:56 -05:00
Christoph Atteneder
91ce44ad96
Merge pull request #4102 from stejbac/tidy-persistable-envelope-inheritance
Tidy PersistableEnvelope inheritance
2020-04-28 16:22:41 +02:00
chimp1984
54e280c78d
Add optimized permutation algo
WIP, needs more tests, comments, clean up....
2020-04-28 00:40:41 -05:00
Chris Beams
3badf299e3
Add 'apiPort' configuration option 2020-04-27 19:49:10 +02:00
Chris Beams
82f1b66818
Refine GrpcServer log output 2020-04-27 19:49:10 +02:00
Chris Beams
a33e030e04
Rename BisqGrpcServer => GrpcServer 2020-04-27 19:49:10 +02:00
Chris Beams
2f7d0f6320
Use var declarations where appropriate 2020-04-27 19:49:09 +02:00
Chris Beams
1f8839e3ac
Make gRPC service classes non-static
So they can easily access the enclosing BisqGrpcServer#coreApi field.
2020-04-27 19:49:09 +02:00
Chris Beams
edbc81c5b4
Rename gRPC service classes from *{Impl=>Service} 2020-04-27 19:49:09 +02:00
Chris Beams
376d11dd1e
Sort static inner classes at bottom 2020-04-27 19:49:09 +02:00
Chris Beams
a52be594cb
Eliminate static fields in BisqGrpcServer 2020-04-27 19:49:09 +02:00
Chris Beams
f2028f2cef
Remove section comments from BisqGrpcServer 2020-04-27 19:49:08 +02:00
Chris Beams
01580ae36a
Remove StopServer rpc method
There is no actual requirement for this method, so removing it. It also
helps improve the implementation by removing the need for the static
'instance' field.
2020-04-27 19:49:08 +02:00
Christoph Atteneder
18476e48b2
Merge pull request #4197 from freimair/pricenode_logs
Reduce log output for price nodes
2020-04-27 14:49:13 +02:00
Christoph Atteneder
d35ab9f9fa
Merge pull request #4192 from devinbileck/add-btc-node-qxjrxmhyqp5vy5hj
Add BTC node qxjrxmhyqp5vy5hj.onion
2020-04-27 14:44:10 +02:00
Florian Reimair
1b2c6c90c0
Reduce log output for price nodes
In case a price feed has an error, a stacktrace dumped in the logs
for each incoming getAllMarketPrice request. In detail, there is one
line of log stating that there is a getAllMarketPrice request, one
line of log stating that there is an error and around 150 lines of
stracktrace.
This fills up the logs quite fast, around 7.5G per day which is
unnecessary.
This commit only outputs the stacktrace in debug log configuration.
2020-04-27 09:56:21 +02:00
ghubstan
cfb7e32e70
Remove note to self 2020-04-26 16:54:10 -03:00
Chris Beams
312ef30b70
Revert marking password as required in JOpt parser
This is a partial reversion of the earlier commit. Marking the password
option as required at the parser level made it impossible to run
./bisq-cli without options or arguments and get the help text. This is a
useful thing to do, and not worth creating a bad user experience to get
the free required option error handling and error messaging.
2020-04-26 21:44:03 +02:00
Chris Beams
a6a8702084
Touch up help output
Stop attempting to align Option and Method description columns with one
another. It's a moving target as we add options and method names, and
this help output format will probably change in the future anyway.
2020-04-26 21:29:29 +02:00
Chris Beams
f5803492bd
Add comment explaining exception message mangling 2020-04-26 21:27:24 +02:00
Chris Beams
f4b4f5ad89
Update cli/test.sh instructions
Don't instruct the user to create a fresh data directory every time, as
this takes quite a bit longer to initialize the wallet than running
against the same data directory repeatedly. Just be clear that the
requirement is an unencrypted wallet with 0 BTC balance.
2020-04-26 21:10:25 +02:00
Chris Beams
5fa7939ec4
Set --password as a required option in JOpt parser 2020-04-26 21:05:22 +02:00
Chris Beams
822e6813f8
Touch up test descriptions and parameter naming 2020-04-26 21:04:06 +02:00
Chris Beams
a3f9faf7ab
Move cli-test.sh => cli/test.sh
- Ensure script runs from root directory regardless of where it is
   invoked from

 - Touch up documentation and whitespace
2020-04-26 20:46:52 +02:00
Chris Beams
ceaf20a161
Clean up whitespace in cli-test.sh 2020-04-26 20:35:09 +02:00
Chris Beams
510e84d30a
Add expect-based cli test suite
Co-authored-by: ghubstan <36207203+ghubstan@users.noreply.github.com>
2020-04-26 20:30:58 +02:00
Chris Beams
b7fda8de5e
Declare channel outside try/catch
No StatusRuntimeException can be thrown from this code, so it is not
necessary to include in the try block.
2020-04-26 19:38:06 +02:00
Chris Beams
0a2aac00d3
Shutdown channel using a JVM shutdown hook
Versus needing to manage calling custom shutdown method in all the right
places.
2020-04-26 19:35:48 +02:00
Chris Beams
e10e29a211
Handle OptionException immediately 2020-04-26 19:32:14 +02:00
ghubstan
dee5e4cf7e
Revert 16c2efc
No need to strip quotes from password.  The problem was a bug
in my expect/tcl script's quote escape syntax.
2020-04-26 10:29:01 -03:00
ghubstan
56f2923ade
Remove redundant text from console err msg
The client was displaying
	Error: UNAUTHENTICATED: incorrect 'password' rpc header value
from the StatusRuntimeException message.

Strip "UNAUTHENTICATED: " from the exception message string before
printing it.
2020-04-25 18:16:23 -03:00
ghubstan
16c2efc8e1
Allow double-quoted multiword apiPassword
Strip double quotes from :cli --password arg before it is passed
to server.  Otherwise, a correct "password" will fail authentication.
2020-04-25 17:52:54 -03:00
Chris Beams
d923d07781
Fix typo in help output 2020-04-25 13:34:32 +02:00
Chris Beams
20f563ae17
Reduce PasswordAuthInterceptor visibility to package-private
And remove unused @Slf4j lombok annotation
2020-04-25 13:32:40 +02:00
Chris Beams
653856f79b
Refactor 'auth*' naming to 'password'
To increase simplicity and make the implementation more
intention-revealing.
2020-04-25 13:28:10 +02:00
Chris Beams
42e9bb1433
Refine help output 2020-04-25 11:31:19 +02:00
Chris Beams
c1931d2fb2
Use 'method' vs 'command' naming 2020-04-25 11:27:19 +02:00
Chris Beams
7bf854c86e
Improve error handling
Fail fast if the specified command is unknown
2020-04-25 11:22:01 +02:00
Chris Beams
c6e5670fbd
Print nested exception error message on connect failure 2020-04-25 10:42:53 +02:00
Chris Beams
b67ad6cd74
Print help text when no command is specified 2020-04-25 10:25:54 +02:00
Chris Beams
423b2ad4ae
Handle OptionException gracefully
When e.g. a given option is not recognized, print a clean error message
and exit 1 as opposed to printing a stack trace.
2020-04-25 10:10:27 +02:00