This is in preparation for addressing log4j 2 zero day exploit described
at https://www.lunasec.io/docs/blog/log4j-zero-day/. See full details
in the next commit.
Bringing in the dependency-management plugin results in many changes to
our Gradle verification metadata file, but all are BOM / POM / Module
manifests. No additional jar or code dependencies have been whitelisted
with this change.
is expected as we shut down the executor immediately.
If not thrown at shutdown or exception is not a RejectedExecutionException
we log a warning and re-throw the exception (to avoid change of behaviour
of current version). The exception is likely not handled by callers and goes up to
the uncaught exception handler.
Use 'java.nio.file.FileSystem' in place of 'java.util.jar.JarFile' to
list all the resources under a given classpath (on Windows & Linux), as
that is a bit neater and potentially more efficient than scanning the
entire ZIP directory structure.
Prevent a "URI is not hierarchical" IllegalArgumentException from the
expression, 'new File(dirUrl.toURI())', which occurs on Linux & Windows
when listing the resource directory of BSQ blocks. Adapt a solution from
StackOverflow which uses two separate code paths depending on the
environment, into the new method 'FileUtil::listResourceDirectory'.
The issue is caused by the resource URL taking one of the two forms:
file:/Users/[USER]/Java/bisq/bisq/p2p/out/production/resources/BsqBlocks_BTC_MAINNET
jar:file:...p2p.jar!/BsqBlocks_BTC_MAINNET
depending on whether the system is OSX or not.
Move timeout before shutdown sequence starts and use a Timer thread instead of
UserThread to avoid that in case the UserThread gets blocked that the timeout
would not get triggered.
Reduce timeout from 20 sec. to 10 sec.
Even if the app is maximized it does not show the full text of the
action button if the text is long. Switching Bisq to German and
maximizing the window reveals the problem.
Fixes#5862
Move timeout before shutdown sequence starts and use a Timer thread instead of
UserThread to avoid that in case the UserThread gets blocked that the timeout
would not get triggered.
Reduce timeout from 20 sec. to 10 sec.
is expected as we shut down the executor immediately.
If not thrown at shutdown or exception is not a RejectedExecutionException
we log a warning and re-throw the exception (to avoid change of behaviour
of current version). The exception is likely not handled by callers and goes up to
the uncaught exception handler.
Use 'java.nio.file.FileSystem' in place of 'java.util.jar.JarFile' to
list all the resources under a given classpath (on Windows & Linux), as
that is a bit neater and potentially more efficient than scanning the
entire ZIP directory structure.
Prevent a "URI is not hierarchical" IllegalArgumentException from the
expression, 'new File(dirUrl.toURI())', which occurs on Linux & Windows
when listing the resource directory of BSQ blocks. Adapt a solution from
StackOverflow which uses two separate code paths depending on the
environment, into the new method 'FileUtil::listResourceDirectory'.
The issue is caused by the resource URL taking one of the two forms:
file:/Users/[USER]/Java/bisq/bisq/p2p/out/production/resources/BsqBlocks_BTC_MAINNET
jar:file:...p2p.jar!/BsqBlocks_BTC_MAINNET
depending on whether the system is OSX or not.
Avoid repurposing the 'ProofOfWork.payload' field for Equihash puzzle
solutions, as that may be of later use in interactive PoW schemes such
as P2P network DoS protection (where the challenge may be a random nonce
instead of derived from the offer ID). Instead, make the payload the
UTF-8 bytes of the offer ID, just as with Hashcash.
Also, make the puzzle seed the SHA-256 hash of the payload concatenated
with the challenge, instead of just the 256-bit challenge on its own, so
that the PoW is tied to a particular payload and cannot be reused for
other payloads in the case of future randomly chosen challenges.
1. Reorder the PoW fields in the 'Filter' proto by field index, instead
of contextually.
2. Deduplicate expression for 'pow' & replace if-block with boolean op
to simplify 'FilterManager::isProofOfWorkValid'.
3. Avoid slightly confusing use of null char as a separator to prevent
hashing collisions in 'EquihashProofOfWorkService::getChallenge'. Use
comma separator and escape the 'itemId' & 'ownerId' arguments instead.
(based on PR #5858 review comments)
Fix a trivial bug in the iterator returned by 'IntListMultimap::get',
caused by mistaken use of the iterator index in place of the key when
doing lookups into the overspill map. This was causing puzzle solutions
to be invalid about 3% of the time, as well as substantially reducing
the average number of solutions found per nonce.
As the fix increases the mean solution count per nonce to the correct
value of 2.0 predicted by the paper (regardless of puzzle params k & n),
inline the affected constants to simplify 'Equihash::adjustDifficulty'.
Remove all the 'challengeValidation', 'difficultyValidation' and
'testDifficulty' BiPredicate method params from 'HashCashService' &
'ProofOfWorkService', to simplify the API. These were originally
included to aid testing, but turned out to be unnecessary.
Patches committed on behalf of @chimp1984.
Change the type of the 'difficulty' field in the Filter & ProofOfWork
proto objects from int32/bytes to double and make it use a linear scale,
in place of the original logarithmic scale which counts the (effective)
number of required zeros.
This allows fine-grained difficulty control for Equihash, though for
Hashcash it simply rounds up to the nearest power of 2 internally.
NOTE: This is a breaking change to PoW & filter serialisation (unlike
the earlier PR commits), as the proto field version nums aren't updated.
Add an abstract base class, 'ProofOfWorkService', for the existing PoW
implementation 'HashCashService' and a new 'EquihashProofOfWorkService'
PoW implementation based on Equihash-90-5 (which has 72 byte solutions &
5-10 MB peak memory usage). Since the current 'ProofOfWork' protobuf
object only provides a 64-bit counter field to hold the puzzle solution
(as that is all Hashcash requires), repurpose the 'payload' field to
hold the Equihash puzzle solution bytes, with the 'challenge' field
equal to the puzzle seed: the SHA256 hash of the offerId & makerAddress.
Use a difficulty scale factor of 3e-5 (derived from benchmarking) to try
to make the average Hashcash & Equihash puzzle solution times roughly
equal for any given log-difficulty/numLeadingZeros integer chosen in the
filter.
NOTE: An empty enabled-version-list in the filter defaults to Hashcash
(= version 0) only. The new Equihash-90-5 PoW scheme is version 1.
Provide a utility method, 'Equihash::adjustDifficulty', to linearise and
normalise the expected time taken to solve a puzzle, as a function of
the provided difficulty, by taking into account the fact that there
could be 0, 1, 2 or more puzzle solutions for any given nonce. (Wagner's
algorithm is supposed to give 2 solutions on average, but the observed
number is fewer, possibly due to duplicate removal.) For tractability,
assume that the solution count has a Poisson distribution, which seems
to have good agreement with the tests.
Also add some (disabled) benchmarks to EquihashTest. These reveal an
Equihash-90-5 solution time of ~146ms per puzzle per unit difficulty on
a Core i3 laptop, with a verification time of ~50 microseconds.
Add a numeric version field to the 'ProofOfWork' protobuf object, along
with a list of allowed version numbers, 'enabled_pow_versions', to the
filter. The versions are taken to be in order of preference from most to
least preferred when creating a PoW, with an empty list signifying use
of the default algorithm only (that is, version 0: Hashcash).
An explicit list is used instead of an upper & lower version bound, in
case a new PoW algorithm (or changed algorithm params) turns out to
provide worse resistance than an earlier version.
(The fields are unused for now, to be enabled in a later commit.)
Run the initial XorTable fillup in 'Equihash::computeAllHashes' in
parallel, using a parallel stream, to get an easy speed up. (The solver
spends about half its time computing BLAKE2b hashes before iteratively
building tables of partial collisions using 'Equihash::findCollisions'.)
As part of this, replace the use of 'java.nio.ByteBuffer' array wrapping
in 'Utilities::(bytesToIntsBE|intsToBytesBE)' with manual for-loops, as
profiling reveals an unexpected bottleneck in the former when used in a
multithreaded setting. (Lock contention somewhere in unsafe code?)
