Change jsonrpc4j version from 1.5.3 to 1.6.0.bisq.1, forked to the Bisq
repo from the recent 1.6.0 release. The forked version changes the class
'com.googlecode.jsonrpc4j.HttpException' to be public, instead of (prob.
mistakenly) package private, so we can avoid using reflection to catch
it and re-throw as a 'bisq.network.http.HttpException'. Remove the now
unused constructors from the latter.
As part of this, upgrade Jackson to the latest stable (2.12.1) release,
since jsonrpc4j now depends on a newer version than the previous 2.8.10.
Migrate RpcService over to the new block notification daemon and RPC
client based on jsonrpc4j. Drop in own DTO classes in place of the ones
defined by btcd-cli4j and rename requestBtcBlock & addNewBtcBlockHandler
to requestDtoBlock & addNewDtoBlockHandler respectively.
Also remove now redundant filtering from the logback config and update
grade-witness.
Create a new 'BitcoindClient' interface and a corresponding builder, to
replace the old 'com.neemre.btcdcli4j.core.client.BtcdClientImpl' class
from the btcdcli4j library. This is instantiated by jsonrpc4j using a
dynamic proxy. It provides only a cut down version of the bitcoind RPC
API, exposing the methods 'getblock', 'getblockcount' & 'getblockhash',
as they are the only ones currently being used by RpcService.
Add corresponding Jackson-annotated DTO classes to model the JSON
structures returned by bitcoind, very similar to the classes provided by
btcdcli4j. Note that we use Double instead of BigDecimal to represent
fractional fields (difficulties + coin amounts in BTC), as they have
more consistent Jackson (de)serialisation and appear to be able to
faithfully round-trip numeric fields produced by bitcoind. Also note
that doubles can faithfully represent any valid decimal BTC amount (that
is, with 8 d.p. of precision) up to 21 million.
For now, keep the old BtcdClientImpl instance used by RpcService in
place, as the btcdcli4j block notification daemon is dependent upon it
and would also need to be replaced.
Also add unit tests for BitcoindClient which test against sample regtest
responses, using a mock HttpURLConnection.
- Show full stack traces in console
- Do use previously cached test outputs
- Do not show skipped test name in console (too noisy)
- Show test run summary, including number of skipped tests
- Show note about skipped test info in the html report
- Show link to html report on test SUCCESS
Update build.gradle to rely on the netlayer libraries from the bisq-network repo. The library versions (commit IDs) remain the same, only the repo from which they are pulled is changed.
Extend the gradle script with tasks that use jpackage to generate Bisq binaries. The kind of binaries generated depend on the OS where this is executed.
The packaging of binaries can be started by calling:
./gradlew --console=plain packageInstallers
from the root project folder.
Include the application version number in the jar manifest. This makes it easier later on, in the packaging process, to programmatically retrieve the build version from a reliable source.
Updated the commons-lang3 version to the latest one, to avoid the existence of multiple versions used by different projects.
Before, v3.8 was used in most projects and v3.9 was brought as a transitive dependency in pricenode. So both were used by different projects. With this commit, all projects use v3.11, since it is explicitly defined in build.gradle.
This resolves an exception thrown by jdeps when trying to list the module dependencies of the desktop-vXYZ-SNAPSHOT-all.jar.
Update the gradle dependency to JavaFX 14.
This brings to Bisq the latest JavaFX fixes and improvements, especially
in the areas of UI performance, memory management and security.
JavaFX can be upgraded independently of the JDK used to build the
application, so this change is modular and does not affect other parts
of the build process.
Related / likely related to: #350#2135#2509#3128#3307#3308#3343#3430#3657#3677#3683#3686#3786#3787#3892#3917#3918#3936
This commit & PR upgrades JFoenix from v9.0.6 to v9.0.10, to avoid an
NPE thrown when registering a DisputeAgent in an arbitartor (regtest)
desktop's account view.
The JFoenix com.jfoenix.adapters.ReflectionHelper class has a getField
method that silently swallows a Throwable and returns null. After
clicking ALT-D or ALT-N in the an arbitrator's desktop -> accounts view
(to register dispute agents) a private field cannot be accessed via
reflection, and bisq.desktop.components.JFXTextFieldSkinBisqStyle#updateTextPos()
throws an NPE.
The cause of the NPE is due to a failure to create the textNode value
in the JFXTextFieldSkinBisqStyle constructor:
textNode = ReflectionHelper.getFieldContent(TextFieldSkin.class, this, "textNode");
If this happens,the UI becomes unusable -- many views are blank.
Exclude these tests from the default gradle test task. However, allow the option of having them included by specifying an optional gradle parameter (-Dtest.pricenode.includeSpotProviderTests=true).
This change moves gRPC boilerplate classes from the :core.grpc pkg
into a new :daemon.grpc pkg.
* The :core.grpc pkg was renamed :core.api, and no longer has any
dependencies on gRPC libraries.
* All core service classes in the :core.api pkg are now package
protected, excepting CoreApi, making CoreApi the only possible
entry point for all Grpc*Service -> -Core*Service calls.
* All grpc service classes in the :daemon.grpc pkg are now package
protected, excepting GrpcServer; the only class depending on
Grpc*Service class is GrpcServer.
* gRPC dependencies were moved from the gradle.build file's :core
subproject to :daemon.
Disable BitcoinAverage provider. Keep it registered as a provider to
ensure that the data structure returned by the pricenode to the Bisq
clients contain the hardcoded "btcAverageTs" key.
Revert from latest v5.0.0 to v4.2.2, since the newer version libraries
are compiled with Java 11, so they cannot be used as part of the Bisq
build process which still partially relies on Java 10.
Add support for a few exchanges to demonstrate and test the pricenode
aggregate rates.
The chose exchanges were selected because they each provide a varied
list of fiat and altcoins, with a substantial overlap between them. This
provides a robust initial set of datapoints and scenarios for aggregate
rates.
This change reorganizes the ':apitest' subproject to conform to a
typical gradle project, where all JUnit test cases are located in
the subproject's test sources folder. This makes running tests
from an IDE or gradle command line interface work as expected.
It will also help keep Travis CI configuration simple.
To avoid interfering in normal builds, the gradle ':apitest test' task
is disable by default; API tests will only run when a '-Dforce-true'
system property is passed to gradle.
To run API tests, run a normal build and install dao-setup files:
./gradlew clean build :apitest:installDaoSetup
Then run the tests:
./gradlew :apitest:test -Dforce=true
Try to avoid adding the '-Dforce=true' property to any other gradle
tasks, because this enables the ':apitest test' task, and would kick
off API tests before a normal build completed.
The build.gradle file was modified to support this code reorg, and
the 'org.junit.jupiter' dependendency was upgraded to v5.6.2 -- only
in the ':apitest:test' dependency definiitions, not anywhere else in
the bisq dependency definitions. The upgrade is necessary for
running ordered tests.
Since the scaffolding may be set up from either test cases (under the
test src folder), or a class under the main src folder, some changes
were made to ensure configuration paths are correct for either use
case. For example, when the 'bisq-apitest' script is run from the root
project directory, the current working directory is the root project
directory. When gradle or an IDE is used to run @Test cases, the
current working directory is :apitest subproject directory.
The main source's ApiTestMain class has been stripped down, and exists
only to keep the gradle build happy -- it needs a 'mainClassName'
property. But this main driver does have uses. See the class comments.
The other changes in this commit were made to fix style and syntax
problems.
Added :proto to the :apitest classpath for access to grpc
service stubs (to be) used in method (unit) tests. Added new
GrpcStubs class to expose the grpc service stubs to method and
scenario tests.
The larger goal of :apitest is end to end testing, where :cli's
console output is checked for correctness.
This change partially addresses two other important use cases:
* "method" testing -- an analog to unit testing
* "scenario" testing -- an analog to narrow functional testing
For example, tests in the apitest.method package will directly
call grpc services, and asserts will be made on the return values
instead of console output.
Tests in the apitest.scenario package will check correctness
for broader use cases, such as funding a wallet, encrypting then
unlocking a wallet for a specific time frame, or checking error
messages from the server when a "getbalance" call is made after
an "unlockwallet" timeout has expired.
The broader end to end tests will not use grpc stubs.
The large binary objects in p2p/src/main/resources/ are updated on every
Bisq release with the latest network data to avoid the need for new Bisq
clients to download all of this information from the network, which
would easily overload seed nodes and generally bog down the client.
This approach works well enough for its purposes, but comes with the
significant downside of storing all of this binary data in Git history
forever. The current version of these binary objects total about 65M,
and they grow with every release. In aggregate, this has caused the
total size of the repository to grow to 360M, making it cumbersome to
clone over a low-bandwith connection, and slowing down various local Git
operations.
To avoid further exacerbating this problem, this commit sets these files
up to be tracked via Git LFS. There's nothing we can do about the 360M
of files that already exist in history, but we can ensure it doesn't
grow in this unchecked way going forward. For an understanding of how
Git LFS works, see the reference material at [1], and see also the
sample project and README at [2].
The following command was used to track the files:
$ git lfs track "p2p/src/main/resources/*BTC_MAINNET"
Tracking "p2p/src/main/resources/AccountAgeWitnessStore_BTC_MAINNET"
Tracking "p2p/src/main/resources/BlindVoteStore_BTC_MAINNET"
Tracking "p2p/src/main/resources/DaoStateStore_BTC_MAINNET"
Tracking "p2p/src/main/resources/ProposalStore_BTC_MAINNET"
Tracking "p2p/src/main/resources/SignedWitnessStore_BTC_MAINNET"
Tracking "p2p/src/main/resources/TradeStatistics2Store_BTC_MAINNET"
We are using GitHub's built-in LFS service here, and it's important to
understand that there are storage and bandwidth limits there. We have
1G total storage and 1G per month of bandwidth on the free tier. We will
certainly exceed this, and so must purchase at least one "data pack"
from GitHub, possibly two. One gets us to 50G storage and bandwith.
In an attempt to avoid unnecessary LFS bandwidth usage, this commit also
updates the Travis CI build configuration to cache Git LFS files, such
that they are not re-downloaded on every CI build (see [3] and [4]
below). With that out of the way, the variable determining whether we
exceed the monthly limit is how many clones we have every month, and
there are many, though it's not clear how many are are Travis CI and how
many are users / developers.
Tracking these files via LFS means that developers will need to have Git
LFS installed in order to properly synchronize the files. If a developer
does not have LFS installed, cloning will complete successfully and the
build would complete successfully, but the app would fail when trying to
actually load the p2p data store files. For this reason, the build has
been updated to proactively check that the p2p data store files have
been properly synchronized via LFS, and if not, the build fails with a
helpful error message. The docs/build.md instructions have also been
updated accordingly.
It is important that we make this change now, not only to avoid growing
the repository in the way described above as we have been doing now for
many releases, but also because we are now considering adding yet more
binary objects to the repository, as proposed at
https://github.com/bisq-network/projects/issues/25.
[1]: https://git-lfs.github.com
[2]: https://github.com/cbeams/lfs-test
[3]: https://docs-staging.travis-ci.com/user/customizing-the-build/#git-lfs
[4]: https://github.com/travis-ci/travis-ci/issues/8787#issuecomment-394202791
Merging PR #4096, which moved protobuf defs out of core and common,
left :seednode without its required dependency on guava, causing
NoSuchMethodErrors.
This change forces :relay's grpc-auth version to match the
version of all other io.grpc-* dependencies, and gets rid of
the duplcate gson dependency v2.8.2
Replaced the Scanner input read loop with upgraded joptsimple
dependency. Cli now takes a single, non-option program argument, runs
it and exits. Also removed the "stop client" command because there is
no input loop, but shutdown() is called for orderly channel shudown
before the jvm terminates. Also changed cmd syntax from camel case
to lowercase, mimicking bitcoin-cli.
Configured logback to supress all debug & info level netty output, and
bypassed logback to print results to System.out.
This is done primarily for concision. This change also repackages
bisq.grpc => bisq.proto.grpc in anticipation of repackaging the
definitions in pb.proto from 'protobuf' to 'bisq.proto'. There should
not be any compatibility issues with doing this, but it's out of scope
here. When complete, the relationship between bisq.proto.grpc and
bisq.proto will be more intuitively clear, i.e. that bisq.proto.grpc has
certain dependencies on bisq.proto classes, but not the other way
around.
Protobuf definition files were moved from common and core to a new
protodefinition subproject.
The two main reasons for doing this are to speed up builds by not
having to regenerate common and core protobuf classes
every time a change is made in those subprojects, and to remove
the grpc cli's direct dependency on core, and the transitive dependency
on common.
In order to accomplish this, cli's BisqCliMain was stripped of
its dependencies on common and core. Cli can only get the version
and balance now.
gRPC stub boilerplate was moved from BisqCliMain to a CliCommand
class to avoid some of the bloat that is going to happen as the
read-response loop supports more rpc commands.
Some apache & logback dependency versions were bumped up, some
transitive dependencies declared as direct dependencies, and some
new exclusions were added to reduce the number of duplicated
dependencies in the build.
Also updated gradle-witness.gradle.
Here are some of the duplicated apache and logback dependencies
in the current build:
commons-codec-1.10.jar
commons-codec-1.9.jar
commons-io-2.4.jar
commons-io-2.6.jar
commons-logging-1.1.1.jar
commons-logging-1.2.jar
httpclient-4.0.1.jar
httpclient-4.5.3.jar
httpclient-4.5.5.jar
httpcore-4.0.1.jar
httpcore-4.4.6.jar
httpcore-4.4.9.jar
logback-classic-1.1.10.jar
logback-classic-1.1.11.jar
logback-core-1.1.10.jar
logback-core-1.1.11.jar
Upgraded:
codecVersion 1.9 -> 1.13
ioVersion 2.4 -> 2.6
langVersion 3.4 -> 3.8
httpclientVersion 4.5.3 -> 4.5.12
slf4jVersion 1.7.22 -> 1.7.25
New dependency declarations:
loggingVersion 1.2
httpcoreVersion 4.4.13
This commit reduces the number of build dependencies by 10.
Partial fix for #4086
