From 5fb4b2156cb5cb9dd609582bbe2a71e2f6d02715 Mon Sep 17 00:00:00 2001 From: Chris Beams Date: Mon, 2 Dec 2019 19:53:49 +0100 Subject: [PATCH] Refine deploy target for better use of `screen` Problem: Prior to this change, it was necessary to first create and attach to a screen session and then to run `make deploy` within it. This meant extra steps for the user and was generally error-prone. Solution: Usage of screen has been refined such that a screen session named 'localnet' is created on the users behalf without any need to attach to it. Individual node deployment targets such as `make bitcoind`, `make alice`, et al. are issued to new windows within the localnet screen session, and the user is free to attach or not whenever they choose. The result is that a new user can clone the repository and type nothing more than `make deploy` to get up and running with their localnet. This also reverts the changes in commit 97dd342e5 ("Make build target phony") for the following reasons: - As mentioned in that commit message, Gradle was not deleting the its 'build' directory when running `gradle clean`, meaning that the 'build' target was always up-to-date, even after running `make clean`. This made it impossible to get a correct rebuild workflow. On analysis, howewer, this situation was because of a badly behaving Kotlin plugin not cleaning up after itself, leaving a subdirectory at build/kotlin and preventing the build directory itself from being deleted altogether. To address this, the `make clean` target has been updated to `rm -rf build` instead of calling `build gradle`. While it's a workaround until we back out the Kotlin changes that caused this, it does have the added benefit of being faster than invoking `gradle clean`. - By making the 'build' target PHONY, this meant that `./gradlew build` was getting invoked every time a dependent target was called. For example, `make alice` depends on the 'setup' target, which in turn depends on the 'build' target. When calling such targets in isolation, this arrangement works out fine, because the phony 'build' target always runs, invoking `./gradle build`, and the Gradle build completes quickly assuming everything is up-to-date. The problem arises when calling a number of these targets in rapid succession, as we do when calling `make deploy` and running each individual node target in its own screen window. This causes contention in two ways. The first is that these multiple, simultaneous Gradle processes compete for access to an available Gradle daemon, and because each process needs its own, it ends up that as many Gradle daemons get created as Bisq nodes we need to deploy (5 in total). This is a big waste of time and resources. The second way it causes not only contention but outright failure is that each of these builds are operating in the same directory, and while most aspects of the build are in fact up-to-date and therefore not modified in any way, there are exceptions to this rule. The result is that build artifacts, e.g. jars are getting deleted and rebuilt from underneath competing Gradle processes, and all manner of chaos ensues, such as NoClassDefFound errors and much more. This change (reverting 'build' back to a normal, non-phony target) avoids these problems entirely. When running `make deploy`, we run the 'build' target once as a function of the 'deploy' target depending on it. At this point, the 'build' directory exists, and all subsequent node deployment targets, e.g. 'alice', 'bob', etc do not re-run the build target because it is up-to-date. For workflows where the user definitely wants to rebuild prior to redeploying a given node, they can either run `make clean-build`, or drop down to issuing Gradle build commands directly, e.g. `./gradlew :desktop:build` followed by `make desktop`. --- Makefile | 31 ++++++++++++++++--------------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/Makefile b/Makefile index 91f973f7ee..acc93cc737 100644 --- a/Makefile +++ b/Makefile @@ -108,7 +108,7 @@ setup: build .localnet clean: clean-build clean-localnet clean-build: - ./gradlew clean + rm -rf build clean-localnet: rm -rf .localnet ./dao-setup @@ -143,20 +143,21 @@ localnet: .localnet # Deploy a complete localnet by running all required Bitcoin and Bisq # nodes, each in their own named screen window. If you are not a screen -# user, you'll need to run each of the make commands manually in a -# separate terminal or as a background job. -# -# NOTE: You MUST already be attached to a screen session for the -# following commands to work properly. +# user, you'll need to manually run each of the targets listed below +# commands manually in a separate terminal or as background jobs. deploy: setup - screen -t bitcoin make bitcoind - sleep 2 # wait for bitcoind rpc server to start - make block # generate a block to ensure Bisq nodes get dao-synced - screen -t seednode make seednode - screen -t seednode2 make seednode2 - screen -t alice make alice - screen -t bob make bob - screen -t mediator make mediator + # create a new screen session named 'localnet' + screen -dmS localnet + # deploy each node in its own named screen window + targets=('bitcoind' 'seednode' 'seednode2' 'alice' 'bob' 'mediator'); \ + for t in "$${targets[@]}"; do \ + screen -S localnet -X screen -t $$t; \ + screen -S localnet -p $$t -X stuff "make $$t\n"; \ + done; + # give bitcoind rpc server time to start + sleep 5 + # generate a block to ensure Bisq nodes get dao-synced + make block bitcoind: .localnet bitcoind \ @@ -242,4 +243,4 @@ block: -rpcpassword=bsq \ generatetoaddress 1 -.PHONY: build seednode +.PHONY: seednode