This change upgrades log4j to patch fixes for recently documented
CVE-2021-45046 CVE-2021-45105 vulnerabilities related to the Log4Shell
exploit.
Like the earlier fix, Bisq does not appear to be vulnerable to these
exploits because it does not use log4j directly, only transitively
depends on it. Nevertheless, the upgrade is still the safe bet.
This commit upgrades our transitive dependency on Log4J 2 from 2.14.1 to
the newly-released 2.15.0 to avoid the CVE described at
https://www.lunasec.io/docs/blog/log4j-zero-day/.
We do not use log4j directly anywhere in our codebase, so our exposure
to this exploit was already mitigated if not eliminated, but Spring Boot
depends on Log4J 2 internally. This commit upgrades Spring Boot's
underlying dependency on Log4J to 2.15.0 in the manner recommended at
https://github.com/spring-projects/spring-boot/issues/28958.
This is in preparation for addressing log4j 2 zero day exploit described
at https://www.lunasec.io/docs/blog/log4j-zero-day/. See full details
in the next commit.
Bringing in the dependency-management plugin results in many changes to
our Gradle verification metadata file, but all are BOM / POM / Module
manifests. No additional jar or code dependencies have been whitelisted
with this change.
This commit upgrades our transitive dependency on Log4J 2 from 2.14.1 to
the newly-released 2.15.0 to avoid the CVE described at
https://www.lunasec.io/docs/blog/log4j-zero-day/.
We do not use log4j directly anywhere in our codebase, so our exposure
to this exploit was already mitigated if not eliminated, but Spring Boot
depends on Log4J 2 internally. This commit upgrades Spring Boot's
underlying dependency on Log4J to 2.15.0 in the manner recommended at
https://github.com/spring-projects/spring-boot/issues/28958.
This is in preparation for addressing log4j 2 zero day exploit described
at https://www.lunasec.io/docs/blog/log4j-zero-day/. See full details
in the next commit.
Bringing in the dependency-management plugin results in many changes to
our Gradle verification metadata file, but all are BOM / POM / Module
manifests. No additional jar or code dependencies have been whitelisted
with this change.
This change sets java source and class generation version targets to 11.
The Bisq distribution is built with JDK 11, but source target has remained at 1.10.
Upgrading allows devs to use some Java syntax features available @since 11, and it
might help anyone building the src avoid confusion over which JDK they should use
(minimum is JDK 11).
See https://docs.gradle.org/current/userguide/java_plugin.html#sec:java-extension
Problem: a
NoSuchMethodError: 'java.util.stream.Collector
com.google.common.collect.ImmutableMultiset.toImmutableMultiset()'
exception was being thrown when testing the previously-merged upgrade to
Gradle 7.3, as described at keybase://chat/bisq#testing/2466.
Solution: This problem is similar to the issue reported at
jeremylong/DependencyCheck#3221. The source of the problem was multiple
conflicting guava jars on the runtime classpath. This commit upgrades to
guava 30.1.1-jre which ensures a single jar on the classpath.
Problem: a
NoSuchMethodError: 'java.util.stream.Collector
com.google.common.collect.ImmutableMultiset.toImmutableMultiset()'
exception was being thrown when testing the previously-merged upgrade to
Gradle 7.3, as described at keybase://chat/bisq#testing/2466.
Solution: This problem is similar to the issue reported at
jeremylong/DependencyCheck#3221. The source of the problem was multiple
conflicting guava jars on the runtime classpath. This commit upgrades to
guava 30.1.1-jre which ensures a single jar on the classpath.
Problem: after the upgrade from 6.6.1 to 7.3, the usual invocation of,
e.g. `./bisq-pricenode 2 2` started failing as reported by @emzy at [1].
bisq@ubuntu-4gb-fsn1-1:~$ /bisq/bisq/bisq-pricenode 2
Error: Could not find or load main class 2
Caused by: java.lang.ClassNotFoundException: 2
Solution: for unknown reasons, the bisq-pricenode script worked as
expected under 6.6.1, i.e. contained the fq main class name in its
scripted invocation of `java -jar ...`, but under 7.3, this main class
name was missing. Through trial and error, it turns out that setting
`mainClassName` explicitly in the :pricenode subproject configuration
solves this problem and makes the start script work as expected.
Presumably, this problem arose in conjunction with the major version
upgrade of the spring boot Gradle plugin that was necessary when
upgrading to Gradle 7.3, but this has not been verified.
[1]: https://github.com/bisq-network/bisq/pull/5824#issuecomment-968276686
Problem: ClassNotFoundError was getting thrown when attempting to run,
e.g. the `./bisq-desktop` or `./bisq-pricenode` start scripts.
Solution: Gradle's posix start scripts were changed significantly
between 6.6.1 and 7.3, including a change to the way the current
directory is determined. This change updates the way we customize start
script generation to allow running them from the root of the repo. This
change only affects the unix / posix variants. The Windows .bat files do
not need to be adapted similarly.
As mentioned in a prior commit, the upgrade to Gradle 7.x results in
many more dependency declarations in the file, many of which are
effectively duplicates. This change does not attempt to eliminate those
duplications in any clever way, but rather just tidies up and organizes
all dependency declarations by sorting them alphabetically.
This commit removes the use of our fork of the gradle-witness plugin in
favor of Gradle's relatively new built-in dependency verification
feature [1].
The gradle/verification-metadata.xml file was produced using the
following command:
./gradlew --write-verification-metadata sha256 build
Where `build` is the usual `gradle build` command. All dependency
configurations are resolved this way, and written out to the file.
The resulting file contains 273 unique dependency declarations, as
compared to just 64 in our now-removed gradle-witness.gradle file. This
means that the coverage of dependencies verified is much more complete.
The new file contains the same sha256 checksums for each dependency as
the old file. This was manually spot-checked for a significant number of
the dependencies.
Like with gradle-witness, builds will break when dependencies are
upgraded (and now also when they are added). To fix these breakages, the
`--write-verification-metadata sha256` option must be provided to the
build. Note that new entries will be added for upgraded depedencies, but
old entries are not removed automatically from the file. These must be
removed manually.
[1]: https://docs.gradle.org/current/userguide/dependency_verification.html
This commit does what is necessary to upgrade from Gradle 6.6.1 to
7.3, including:
- generating the new Gradle wrapper
- replacing uses of 'compile' with 'implementation'
- replacing uses of 'testCompile' with 'testImplementation'
Moving from *compile to *implementation results in many more duplicated
dependency declarations throughout the file. These will be tidied up in
a subsequent commit.
Several dependencies needed to be upgraded in order to support this
change. One of them was Spring Boot, from 1.5.1 to 2.5.6. This is a
major upgrade that contained some breaking changes to the Spring Boot
Actuator. These changes required the removal of the pricenode's
/getVersion endpoint.
The Gradle Witness plugin has been disabled in this commit, because it
uses the now-removed 'compile' configuration. Use of the Witness plugin
will be removed entirely in a subsequent commit in favor of using
Gradle's new built-in dependency verification feature.
Improve logging
Add BsqBlockStore to protobuf
Remove DaoStateMonitoringService field
Do not persist the blocks in daoState anymore.
This improves persistence performance and reduces memory
requirements for snapshots.
Change jsonrpc4j version from 1.5.3 to 1.6.0.bisq.1, forked to the Bisq
repo from the recent 1.6.0 release. The forked version changes the class
'com.googlecode.jsonrpc4j.HttpException' to be public, instead of (prob.
mistakenly) package private, so we can avoid using reflection to catch
it and re-throw as a 'bisq.network.http.HttpException'. Remove the now
unused constructors from the latter.
As part of this, upgrade Jackson to the latest stable (2.12.1) release,
since jsonrpc4j now depends on a newer version than the previous 2.8.10.
Migrate RpcService over to the new block notification daemon and RPC
client based on jsonrpc4j. Drop in own DTO classes in place of the ones
defined by btcd-cli4j and rename requestBtcBlock & addNewBtcBlockHandler
to requestDtoBlock & addNewDtoBlockHandler respectively.
Also remove now redundant filtering from the logback config and update
grade-witness.
Create a new 'BitcoindClient' interface and a corresponding builder, to
replace the old 'com.neemre.btcdcli4j.core.client.BtcdClientImpl' class
from the btcdcli4j library. This is instantiated by jsonrpc4j using a
dynamic proxy. It provides only a cut down version of the bitcoind RPC
API, exposing the methods 'getblock', 'getblockcount' & 'getblockhash',
as they are the only ones currently being used by RpcService.
Add corresponding Jackson-annotated DTO classes to model the JSON
structures returned by bitcoind, very similar to the classes provided by
btcdcli4j. Note that we use Double instead of BigDecimal to represent
fractional fields (difficulties + coin amounts in BTC), as they have
more consistent Jackson (de)serialisation and appear to be able to
faithfully round-trip numeric fields produced by bitcoind. Also note
that doubles can faithfully represent any valid decimal BTC amount (that
is, with 8 d.p. of precision) up to 21 million.
For now, keep the old BtcdClientImpl instance used by RpcService in
place, as the btcdcli4j block notification daemon is dependent upon it
and would also need to be replaced.
Also add unit tests for BitcoindClient which test against sample regtest
responses, using a mock HttpURLConnection.
