We used checkpoint files from BitcoinJ 0.14.4 and those caused a very
slow initial download. I tested with the WalletAppKit and different
release version sof BitcoinJ and could reproduce the behaviour there.
WIth v0.14.5 it was much better. But also that caused slow startup in
Bisq. After creating a current checkpoint file and using that the
download is now very fast.
Also added logging for versions of BitcoinJ and libDohJ so that we have
better control which version is used.
Prior to this fix, the build failed with the following error:
FAILURE: Build failed with an exception.
* Where:
Build file '/Users/cbeams/Work/bisq-network/bisq/build.gradle' line: 109
* What went wrong:
Execution failed for task ':desktop:installDist'.
> No such property: bat for class: java.lang.String
@cbeams
The copy task below fails because the copied files are conflicting
with the project folder name.
I temporarily delete those files as I think they are duplicated
anyway (bisq-* are probably the one to use). Would be good if those
files don't get created in the first place.
The related *.bat files are also deleted.
I push that change directly as currently both master and release
branch fail to build.
Related to https://github.com/bisq-network/bisq/pull/1959 which
removed .bat files on a non-Windows OS to prevent a cluttered root
directory, this change will delete *nix scripts on a Windows OS.
Previously, start scripts were generated for both *nix and Windows
platforms, resulting in an unnecessarily cluttered root directory.
With this change, both types of script are still generated, but Windows
.bat scripts are deleted immediately afterward if the user is running a
non-Windows OS (unfortunately, there was no clean way to suppress the
generation of these scripts in the Gradle StartScripts API).
See #1956
This change configures the Gradle build to generate "start scripts" for
each Bisq executable (e.g. Bisq Desktop, Bisq Seednode, etc) in the root
project directory, such that after invoking `./gradle build`, the
following executable scripts become available:
~/Work/bisq-network/bisq
$ ls -1 | egrep '(bisq*|lib)'
bisq-desktop
bisq-desktop.bat
bisq-monitor
bisq-monitor.bat
bisq-relay
bisq-relay.bat
bisq-seednode
bisq-seednode.bat
bisq-statsnode
bisq-statsnode.bat
lib
This makes it possible for users (developers) to easily discover and use
these scripts in an idiomatic and platform-agnostic way as opposed to
the previous situation where we would advise users to run e.g.
java -jar desktop/build/libs/desktop-0.8.0-SNAPSHOT-all.jar
This approach works, but is cumbersome and focuses unnecessarily on the
Java-based nature of the project. Now, with the changes in this commit,
the user would simply run:
./bisq-desktop
The 'lib' directory shown above contains all the jar files necessary to
construct classpaths for these various scripts. The 'cleanInstallDist'
task deletes the 'bisq-*' files and the 'lib' directory, and the default
'clean' task has been configured to depend on the 'cleanInstallDist'
task to ensure this cleanup happens automatically when most users would
expect it.
In the future, these same scripts can be used when installing Bisq
executables properly on users' systems via package managers like Brew
and Apt. The goal is to have the user experience around running
`bisq-desktop` (and more importantly, the forthcoming `bisqd`) be
similar in every way to installing and using `bitcoind`, `lnd` and other
idiomatic *nix-style utilities, be they Bitcoin-related or not.
See the changes in docs/build.md and docs/dev-setup.md for a further
sense of the how this change impacts the developer experience.
Updating to the latest shadow plugin version will resolve the following
deprecation warning:
> Task :desktop:shadowJar UP-TO-DATE
Registering invalid inputs and outputs via TaskInputs and TaskOutputs
methods has been deprecated. This is scheduled to be removed in Gradle
5.0. A problem was found with the configuration of task
':desktop:shadowJar'.
See: https://github.com/johnrengelman/shadow/issues/336
This change avoids the as-yet unexplained resolution errors we've been
seeing out of JCenter today, which look like the following:
Could not resolve all files for configuration ':monitor:compileClass
> Could not resolve net.gpedro.integrations.slack:slack-webhook:1.1.
Required by:
project :monitor
> Could not resolve net.gpedro.integrations.slack:slack-webhook:1
> Could not get resource 'http://jcenter.bintray.com/net/gpedr
> Could not HEAD 'http://jcenter.bintray.com/net/gpedro/int
> sun.security.validator.ValidatorException: PKIX path b
For full context, see the Travis CI build failure at
https://travis-ci.org/bisq-network/bisq/builds/451786384#L678-L679.
It is assumed this is an issue on the JCenter side, e.g. something to do
with the interaction of JCenter servers and their SSL implementation
when accessed under OpenJDK 10 and its TrustStore implementation. This
specific failure does not occur under Oracle JDK 11, but other
dependencies do fail in a similar but non-deterministic fashion. In any
case, resolving project dependencies from Maven Central solves the
problem in the meantime.
See also https://github.com/gradle/gradle/issues/6630.
- Move mavenLocal on top to resolve local jars first
- Comment out annotationProcessor and testAnnotationProcessor entries as
they conflict with grpc build (don't know why). Complete project build
works also without the annotationProcessor entries.
- Comment out btcd-cli4j entries in dependencyVerification as they cause
build failure. TODO investigate why...
This does the work that was intended to be done in commit
175e11d81c, but was done by first removing
the dependencyVerification block entirely, and then replacing it with
the output of the `calculateChecksums` task.
The entire process went like this:
1. Remove existing dependencyVerification block from build.gradle
2. Run `./gradlew -q calculateChecksums | grep -v network.bisq:bisq- >> build.gradle`
3. Run `git diff` to see that only the checksums we expect to have
changed have in fact changed (libdohj and bitcoinj in this case).
4. Commit the changes (in this commit)
I'll update the instructions for the dependencyVerification block in a
subsequent commit to make this clearer in the future.
The calculated checksum from the calculateChecksums task did not
reflect updates of libdohj and bitcoinj.
libdohj is a dependency in common but seems the calculateChecksums task
in desktop did not resolve that correctly. It did not change the
checksum after the libdohj version has changed and led to build errors.
Adding the libdohj dependency in Desktop as well solves the issue.
@bceams: If there is a better solution to solve that feel free to
revert and apply....
Setting this value to `false` was a minor optimization to improve build
speed, as we don't need both tar and zip dists to be built (and zip
actually takes a few seconds).
PR #1500 set it to `true`, because it ended up causing build failures
down the road. This commit removes the statement altogether, as `true`
is the default value anyway.
Recent changes to dependencyVerification got committed with 8-space
indentation, causing a diff for the whole block. This reverts it to
4-space, which is what is generated by `gradle -q calculateChecksums`.
It doesn't actually work to build fat executable jars because we always
have bouncycastle jars that need to exist outside of the fat jar. This
change drops building fat jars altogether in favor of using Gradle's
built-in `application` plugin to write simple scripts that invoke java
with the correct classpath against a directory of all dependencies
(including bouncycastle) without the need to have a fat jar in the mix.
Essentially, you should now run:
./gradlew build
./build/app/bin/bisq-desktop [options]
Instead of `java -jar build/libs/bisq-desktop.jar`.
See updated doc/build.md for details.
For reasons detailed in bisq-network/bisq-p2p@b1528bf3.
This change also regenerates the `dependencyVerification` block, which
is necessary in any case after the upgrade of tor binaries, but note
especially that for unknown reasons, commenting out certain dependencies
items is now no longer necessary as it was in commit
53a9009b06.
Problem: The dependency commented out in this commit, when left
uncommented cause any Gradle composite build that includes this
(`bisq-desktop`) build to fail with, for example, the following error:
No dependency for integrity assertion found:
com.github.bisq-network.libdohj:libdohj-core
Solution: Comment them out until someone can dig into the gradle-witness
plugin, figure out why these specific entries cause these failures, and
fix it. Note that the commented entries work as expected when this
Gradle build is used outside the context of a composite build.
This commit is *not* a complete re-working of the scripts under package,
but is rather just enough of a re-working of the
package/osx/create_app.sh script to verify that the new Gradle build
creates a suitable fat executable jar for use in this script.
These changes have been tested with the following commands:
cd package/osx
./create_app.sh
open ../../deploy/Bisq-0.6.7.dmg
Then double-clicking the Bisq icon from the mounted DMG. Bisq starts up
as expected, without errors.
Further work will be required to update the package/osx/finalize.sh
script as well as the package/linux/ and package/windows/ scripts prior
to the next (0.7.0) release. For now, however, enough work has been done
to verify that the Gradle build creates the fat jar these scripts need.
This performs the same function that the maven-enforcer-plugin did in
the pom removed by the previous commit, and also includes a more
comprehensive / up-to-date set of dependencies.
Note that the gradle-witness jar checked in here is one built from the
pull request in signalapp/gradle-witness#26.
Note that this has not been thoroughly tested and, in particular, my router does not support UPnP so I've only been able to test with --node.useManualPortForwarding=true The project does compile and run successfully though. With manual port forwarding I'm able to connect to the p2p network and view open offers successfully.
Significant changes in the way that controllers and models interact with
backend services. Most important is the introduction of RxJava's
Observable. See individual commit comments for details.
Conflicts were minor, mainly dealing with the fact that MainPM had been
modified in master, but removed completely on the 'cbeams' branch. All
changes have been preserved by carrying them over to MainModel.
* cbeams:
Improve service initialization coordination using rx.Observable
Redesign controller/model types and apply to gui.main.Main*
Conflicts:
build.gradle
src/main/java/io/bitsquare/btc/WalletService.java
src/main/java/io/bitsquare/gui/main/MainPM.java
src/main/java/io/bitsquare/msg/tomp2p/TomP2PNode.java
This change introduces the use of RxJava's Observable [1] to redesign
how we work with non-deterministic and/or event-based information, such
as: connecting to peer-to-peer infrastructure, synchronizing the bitcoin
blockchain, and so on.
Prior to this commit, these activities were initiated in methods like
WalletService#initialize and TomP2PMessageService#init. These methods
accepted 'listener' interfaces, and these listeners' callback methods
would be invoked whenever work progressed, completed, or failed.
This approach required significant coordination logic, which, prior to
this commit, was found primarily in MainModel#initBackend. A primary
goal of the logic found here was to determine when the backend was
"ready". This state was represented in MainModel's `backendReady` field,
which would be set to true once the following three conditions were
satisfied:
1. the message service had finished initialization
2. the wallet service had finished initialization, and
3. the blockchain synchronization had reached 100%
Monitoring these three states was complex, and required hard-to-follow
conditional logic spread across a number of locations in the code. In
any case, however, once these three conditions were satisfied and
backendReady's value was set to true, a listener on the backendReady
field (in MainViewCB#doInitialize) would then populate combo boxes and
pending trade counts in the main view and cause the splash screen to
fade out, rendering the application ready for user interaction.
The introduction of rx.Observable is designed to achieve the same
show-the-splash-screen-until-everything-is-ready functionality described
above, without the complex monitoring, conditional logic and nested
callbacks. This is achieved by modeling each process as an Observable
stream of events. Observables in RxJava can emit any number of events,
and can complete either normally or with an error.
These observables may be 'subscribed' to by any number of subscribers,
and events emitted can be acted upon by instructing the subscriber what
to do `onNext`, `onCompleted`, and `onError`. So for example
WalletService now exposes an Observable<Double> called bootstrapState.
This Observable is subscribed to in MainModel#initBackend in such a way
that every time it emits a new double value (i.e. a new percentage), the
various bootstrap state text labels and progress indicators are updated
accordingly.
Where it gets really interesting, however, is when Observables are
combined. The primary complexity described above is coordinating the
fading out of the splash screen with the completed initialization of all
backend services. As can now be seen in MainModel#initBackend, the
wallet service and message service Observables are simply "merged" into
a single observable and returned. From the MainViewCB side, this "single
backend observable" is subscribed to and, when it completes (i.e. when
all the underlying Observables complete), then combo boxes and pending
trade counts are populated and the splash screen is faded out.
Understanding RxJava, Observables, and the principles of "Functional
Reactive Programming" takes time. It is a paradigm shift in dealing with
concurrency and non-determinism, but one that ultimately rewards those
who take the time. In the end, I believe it's use will result in a
significantly more concise and robust internal architecture for
Bitsquare, and using RxJava's lightweight, well-adopted and
infrastructure-agnostic API leaves us open to using Akka or other more
sophisticated infrastructure later without tying ourselves to those
specific APIs (because virtually anything can be modeled as an
Observable). Achieve these benifits means that core committers will need
to understand how RxJava works, how to think about it, and how to design
using it. I have spent the better part of the last week getting to know
it, and I am certainly still learning. I can recommend many resources to
aid in this process, but having gone through it myself, I recommend that
everyone read at least [1] and [2] first.
[1]: https://github.com/ReactiveX/RxJava/wiki/Observable
[2]: [The introduction to Reactive Programming you've been
missing](https://gist.github.com/staltz/868e7e9bc2a7b8c1f754)
The build now exposes two explicit ShadowJar tasks: one for the main
JavaFX client (`appJar`) and one for the headless bootstrap node
(`bootstrapNodeJar`).
Run as follows:
./gradlew appJar
-- or --
./gradlew bootstrapNodeJar
The resulting executable jar for each will be found in the `build/libs`
directory.
Thanks to @johnrengleman for his help at johnrengelman/shadow#108
Resolves#265Resolves#252
- Introduce a test-time dependency on spring-test module for access to
MockPropertySource and friends.
- Add BitsquareEnvironmentTests and test that property source precedence
works as expected, i.e. that properties supplied on the command line
have highest precedence, overriding those picked up via environment
variables, system properties, the bitsquare.properties file or any of
the other available property sources.
Use of the Spring Environment
-----------------------------
This change replaces the use of the argparse4j library and basic
Properties objects with the Spring Framework's Environment abstraction.
The Environment allows for managing any number of 'property sources' in
a hierarchical fashion, such that a call to
`environment.getProperty("someKey")` iterates through an ordered set of
property sources, returning the first value associated with the given
key.
BitsquareEnvironment, introduced in this commit, eliminates the
functionality previously present in ConfigLoader, modeling the
bitsquare.conf and bitsquare.properties files as Spring Resource
objects, and in turn creating ResourcePropertySources out of them. These
custom property sources are combined with standard property sources
based on system environment variables and Java system properties as well
as a property source based on the command-line arguments passed to a
Bitsquare application to form a unified, one-stop configuration
hierarchy.
For example, let's say a Bitsquare user wishes to customize the port
that his Bitsquare application listens on. The simplest approach
(assuming the user is comfortable with the command line), would be the
following:
java -jar bitsquare.jar --port=1234
where '1234' is the custom port of choice. This is convenient enough for
one-off experimentation, but if the user wishes to make this a permanent
arrangement, he may want to add a `port=1234` entry to his
{bitsquare_app_dir}/bitsquare.conf file.
Alternatively, the user may wish to specify the port value as an
environment variable, e.g.:
PORT=1234 java -jar bitsquare.jar
or with a JVM system property, e.g.:
java -jar -DPORT=1234 bitsquare.jar
With BitsquareEnvironment, and its customized set of PropertySources in
place, the value of the port property may be specified in any of the
ways described above, and it is all handled in a unified way.
Restructuring of *Main classes
------------------------------
This commit also introduces significant changes to the structure of
executable Bitsquare applications. For example, prior to this change,
the io.bitsquare.app.gui.Main class was responsible for being both a
JavaFX Application and a standard Java main class.
Now, however, these concerns have been renamed and separated.
BitsquareApp is the JavaFX Application, and BitsquareAppMain is the Java
main class. Likewise, BootstrapNode has been broken out into
BootstrapNode and BootstrapNodeMain.
A common base class for the *Main classes has been extracted, named
BitsquareExecutable, which creates a template for option parsing,
environment creation, and ultimately application execution that applies
both to the BootstrapNode and BitsquareApp cases.
Improved help text
------------------
With the removal of argparse4j and the introduction of JOpt for argument
parsing, the application's help text has been improved. Use --help to
display this text, where you'll see information about default values,
etc. To do this easily from the Gradle build, run any of the following
commands:
# Display help text
./gradlew run -Pargs="--help"
# Qualify the application name as "Bitsquare-Alice"
./gradlew run -Pargs="--appName=Alice"
# Customize the port
./gradlew run -Pargs="--port=7377"
Renaming of FatalException
--------------------------
Finally, the exception formerly known as io.bitsquare.gui.FatalException
has been moved up a package and generalized to
io.bitsquare.BitsquareException, as it is now used more widely.
Prior to this commit, Gradle was configured to take a -Pargs property
and split the value on whitespace, passing the result to the Gradle
application plugin's 'args' property, for example:
gradle run -Pargs="--id=foo --ip=1.1.1.1 --port=10001"
While this approach works fine when passing a single argument (i.e. when
no space delimiters are required), when multiple arguments are passed,
such as in the example above, it would result in the following error
from Gradle's own command line parser:
Unknown command-line option '--ip'
This commit simply splits the value of -Pargs on commas rather than
spaces, meaning that now multiple -Pargs values should be supplied as
follows:
gradle run -Pargs="--id=foo,--ip=1.1.1.1,--port=10001"
Resolves#264
Use `./gradlew run` to build and run Bitsquare directly.
Use `./gradlew shadowJar` to generate a portable, executable jar.
Use `./gradlew packageNative` to generate an installer binary specific
to the platform you are running on.
In the case of Windows, it is necessary to copy the two .dll files in
the `package/` directory to the directory where Bitsquare.exe is
ultimately installed.
Resolves#243
This commit applies the 'eclipse' Gradle plugin, such that .classpath,
.project and .settings files can be generated using:
./gradlew eclipse
Once the above is complete, import the project into Eclipse with the
following command:
File->Import->Existing projects into workspace
The .gitignore file has been updated accordingly.
Resolves#222
- The 'java' plugin is added implicitly by the gradle/javafx.gradle
script, so there is no longer a need to explicitly apply it.
- The 'application' plugin is no longer necessary now that we are
building native installers with the javafx plugin.
The plugin's jfx* tasks tie into the normal Gradle build lifecycle, such
that `gradle build` will now generate executables and installers
according to the OS on which the build is being run. These files are
output to the `build/distributions` directory.
Installers work as expected OS X and Linux at this point.
Windows installers do build, but a very particular configuration is
necessary on the Windows machine doing the building (this configuration
is to be documented in #109). However, even when the configuration is in
place and the MSI installer is successfully built, there is still a
fatal error at installer execution time relating to a missing
msvp100.dll file. See details at
https://bitbucket.org/shemnon/javafx-gradle/issue/43. An issue has been
created to track this from the Bitsquare side as well--see #108.
The changes made in this commit are based on on the samples at
http://bitbucket.org/shemnon/javafx-gradle and the article at
http://jaxenter.com/tutorial-a-guide-to-the-gradle-javafx-plugin-46270.html
The gradle/javafx.gradle file is copied directly from the sources in the
bitbucket repository above, as is apparently the convention (not sure
why this isn't part of the plugin itself, but that's a question to be
addressed later).
Resolves#66, #100
See #108, #109
Bitsquare depends on specific versions of BitcoinJ and TomP2P that have
not previously been published to any maven repository--until now. Based
on the 'published' branch of the bitcoinj [1] and tomp2p [2] forks under
the bitsquare organization, snapshots have been published to a temporary
repository [3] that has been made available to us by JFrog.
To be as explicit as possible, these custom-published snapshots have
had their version numbers qualified with the short hash of the commit
they were built from. So for bitcoinj, the dependency is no longer
0.12-SNAPSHOT, but 0.12.308de4e-SNAPSHOT. For TomP2P, the version has
gone from 5.0-Alpha24-SNAPSHOT to 5.0-Alpha24.805623c-SNAPSHOT.
Accordingly, these qualified versions are now reflected in the
dependency declarations in build.gradle.
This means that it is no longer necessary to build bitcoinj and tomp2p
locally in order to get up and running with Bitsquare development, and
the README has been updated accordingly. And it also means we can now
set up a Travis CI build with ease (which shall be done with #86).
Resolves#97
[1]: https://github.com/bitsquare/bitcoinj/tree/published
[2]: https://github.com/bitsquare/tomp2p/tree/published
[3]: http://partnerdemo.artifactoryonline.com/partnerdemo/libs-snapshots-local/
For creation of a self-contained executable jar.
Run:
- `./gradle executableJar`
- `java -jar ./build/libs/bitsquare.jar
Note that actually executing the jar fails with an exception from
io.bitsquare.util.AppDirectoryUtil.createDirIfNotExists. This will be
fixed later. The purpose of this commit is only to establish the
executable jar infrastructure.
- Set version at 0.1.0-SNAPSHOT, per http://semver.org practices
- Configure dependencies per existing Maven pom
- Resolve dependencies from jcenter by default, mavenLocal for custom
BitcoinJ and TomP2P binaries (which still must be built separately)
- Copy non-java files (fxml, etc) from src/main/java when processing
resources
`./gradle build` is now completes successfully.
