Merge pull request #1852 from ManfredKarrer/add-dev-setup-guide

Add dev setup guide

doc/CONTRIBUTING.md

@@ -26,3 +26,8 @@ Copyright
 
 By contributing to this repository, you agree to license your work under the AGPL license.
 Please include the bisq license header into your files.
+
+Compensation for your contributions
+---------
+Bisq is not a company but operates as a [DAO](https://docs.bisq.network/dao/phase-zero.html). For any development work merged into the Bisq master branch you can file a [compensation request](https://github.com/bisq-network/compensation) and earn BSQ (the DAO native token). Learn more about the Bisq DAO at our [docs](https://docs.bisq.network/dao/phase-zero.html) pages.
+

doc/DEV_SETUP.md

@@ -0,0 +1,161 @@
+## Bisq developer guide
+
+This guide describes the development setup for developers who want to contribute to Bisq.
+
+### Getting started
+
+Please follow the instructions in the [README](https://github.com/bisq-network/bisq/tree/master/README.md) file for running Bisq from source code. Be sure to have the right JDK version installed. Best to use IntelliJ IDEA (there is a free community edition). Please read also the [CONTRIBUTING](https://github.com/bisq-network/bisq/tree/master/doc/CONTRIBUTING.md) file.
+
+### Basic setup
+
+For development you usually want to use **regtest** mode with **localhost** (instead using the Tor network). 
+
+You want to run typically a setup with those components:
+- Bitcoin Core or bitcoind in regtest mode
+- A local Bisq seed node
+- A local Bisq arbitrator
+- 2 Bisq trading peers (BTC buyer and BTC seller for executing a trade)
+
+### Bitcoin Core (or bitcoind) in regtest mode
+
+The regtest mode operates a local Bitcoin network on your computer. This environment is ideally suited for testing because you are able to create blocks on demand (no need to wait for confirmations) and you don't need to download the blockchain. By creating blocks you act like a miner and you can generate new Bitcoin.
+You can find more information about the Bitcoin regtest mode [here](https://bitcoin.org/en/developer-examples#regtest-mode).
+
+Navigate to the [bitcoin.conf](https://en.bitcoin.it/wiki/Running_Bitcoin#Bitcoin.conf_Configuration_File) file and set `regtest=1`, or add `-regtest` as a program argument when starting Bitcoin Core.
+
+At first startup you need to create 101 blocks using the command `generate 101` from the terminal inside Bitcoin Core. 101 blocks are required because of the coin maturity (100 blocks) so you need one more to have at least 50 BTC available for spending.
+Later you can create a new blocks with `generate 1`.
+
+
+#### Optional for DAO mode
+
+If you want to run Bisq with DAO mode enabled you need to configure the `bitcoin.conf` file inside the Bitcoin Core data directory as well adding the `blocknotify` file.
+
+**bitcoin.conf:**
+```
+regtest=1
+
+# The default rpcPort for regtest from Bitcoin Core 0.16 and higher is: 18443
+# The default rpcPort for testnet is: 18332
+# For mainnet: 8332
+rpcport=18443  
+
+server=1
+txindex=1
+rpcuser=YOUR_USER_NAME
+rpcpassword=YOUR_PW
+blocknotify=bash [PATH TO DATA DIR]/blocknotify %s
+```
+
+_Please note that `txindex` triggers a resync of the entire blockchain (be aware if you set that on mainnet as that it will take a while). Also take care if you use that setting for mainnet. Extra settings for more security are recommended in mainnet mode._
+
+The `blocknotify` file need to be added to the Bitcoin Core data directory as well:
+```
+#!/bin/bash
+echo $1 | nc -w 1 127.0.0.1 5120
+echo $1 | nc -w 1 127.0.0.1 5121
+echo $1 | nc -w 1 127.0.0.1 5122
+echo $1 | nc -w 1 127.0.0.1 5123
+```
+
+It defines the ports where a new block event gets forwarded. Bisq will listen on that port and each Bisq node need to use a different port. You can add or remove ports from the list inside the file if needed.
+
+
+### Local Bisq P2P network
+
+For the local P2P network we prefer to use `localhost`, not the Tor network as it is much faster. But if needed you can combine any of the following combinations of Bitcoin network mode and P2P network mode:
+- localhost + regtest
+- localhost + testnet
+- localhost + mainnet
+- Tor + regtest
+- Tor + testnet
+- Tor + mainnet
+
+### Program arguments
+
+There are several program arguments required to run in development mode. 
+
+Here is an overview:
+
+- --baseCurrencyNetwork: The BTC network to use. Possible values are: BTC_REGTEST, BTC_TESTNET, BTC_MAINNET (default)
+- --useLocalhostForP2P: Uses localhost instead of Tor for Bisq P2P network
+- --nodePort: Port number for localhost mode. For seed nodes there is a convention with the last digit is marking the network type and there is a list of hard coded seed nodes addresses (see: DefaultSeedNodeAddresses.java). For regtest: 2002 and 3002. For testnet 2001, 3001 and 4001 and for mainnet:  2000, 3000 and 4000. For normal nodes the port can be chosen freely.
+- --myAddress: Needed for seed nodes only (e.g.: `localhost:3002`) 
+- --useDevPrivilegeKeys: Important for dev testing to allow the developer key for arbitration registration
+- --appName: Custom application name which is used for the data directory. It is important to separate your nodes to not interfere... If not set is uses the default `Bisq` directory.
+
+#### Program arguments for DAO mode
+
+--daoActivated: If set to true it enables the DAO mode. For testnet and regtest it is enabled by default.   
+--genesisBlockHeight: If set it overrides the hard coded block height of the genesis tx. Set it to your local genesis tx height.  
+--genesisTxId: If set it overrides the hard coded genesis tx ID. Set it to your local genesis tx ID.  
+--fullDaoNode: If true it enables full DAO node mode (in contrast to default lite node mode). At least one seed node must be running as a full DAO node to support other lite nodes.  
+--rpcUser: RPC user as defined in bitcoin.conf   
+--rpcPassword: RPC pw as defined in bitcoin.conf  
+--rpcPort: RPC port. For regtest 18443   
+--rpcBlockNotificationPort: One of the ports defined in the `blocknotify` file inside the Bitcoin data directory (see: DAO setup for Bitcoin Core).  
+
+### Bisq seednode
+
+For localhost/regtest mode run the SeedNodeMain.java class or the seednode.jar (inside the seednode/build/libs folder) with following program arguments:
+
+`--baseCurrencyNetwork=BTC_REGTEST --useLocalhostForP2P=true --useDevPrivilegeKeys=true --nodePort=2002 --myAddress=localhost:2002 --appName=bisq-BTC_REGTEST_Seed_2002`
+
+### Bisq arbitrator instance
+For localhost/regtest mode run the BisqAppMain.java class or the desktop.jar (inside the desktop/build/libs folder) with following program arguments:
+
+`--baseCurrencyNetwork=BTC_REGTEST --useLocalhostForP2P=true --useDevPrivilegeKeys=true --nodePort=4444 --appName=bisq-BTC_REGTEST_arbitrator`
+
+Once it has started up go to Account and click `cmd +r`. This will open a new tab for `Arbitration registration`. Select the tab and you will see a popup with a pre-filled private key. That is the developer private key (which is only valid if useDevPrivilegeKeys is set) which allows you to register a new arbitrator. Follow the next screen and complete registration.
+
+_Note: You need only register once but if you have shut down all nodes (including seed node) you need to start up the arbitrator again after you start the seed node so the arbitrator re-publishes his data to the P2P network. After it has started up you can close it again. You cannot trade without having an arbitrator available._
+
+### Bisq trade instances
+
+For localhost/regtest mode run the BisqAppMain.java class or the desktop.jar (inside the desktop/build/libs folder) with following program arguments:
+
+`--baseCurrencyNetwork=BTC_REGTEST --useLocalhostForP2P=true --useDevPrivilegeKeys=true --nodePort=5555 --appName=bisq-BTC_REGTEST_Alice`
+and
+`--baseCurrencyNetwork=BTC_REGTEST --useLocalhostForP2P=true --useDevPrivilegeKeys=true --nodePort=6666 --appName=bisq-BTC_REGTEST_Bob`
+
+At this point you can now perform trades between Alice and Bob using your local regtest environment and test from both the buyer's and seller's perspective. You can also open disputes with `cmd+o` and see how the arbitration system works (run the arbitrator in that case as well).
+
+_Remember to generate a new block in the Bitcoin Core console after taking an offer using the command `generate 1` to trigger a block confirmation._
+
+### DAO mode
+If you want to run any instance in DAO mode use those program arguments:
+
+Full node mode:
+
+`--daoActivated=true --genesisBlockHeight=111 --genesisTxId=30af0050040befd8af25068cc697e418e09c2d8ebd8d411d2240591b9ec203cf --baseCurrencyNetwork=BTC_REGTEST --useDevPrivilegeKeys=true --useLocalhostForP2P=true --nodePort=7777 --appName=bisq-BTC_REGTEST_Alice_dao --fullDaoNode=true --rpcUser=YOUR_USER_NAME --rpcPassword=YOUR_PW --rpcPort=18443 --rpcBlockNotificationPort=5120`
+
+Lite node mode:
+
+`--daoActivated=true --genesisBlockHeight=111 --genesisTxId=30af0050040befd8af25068cc697e418e09c2d8ebd8d411d2240591b9ec203cf --baseCurrencyNetwork=BTC_REGTEST --useDevPrivilegeKeys=true --useLocalhostForP2P=true --nodePort=8888 --appName=bisq-BTC_REGTEST_Bob_dao`
+
+_Don't forget to use different rpcBlockNotificationPorts for different full node instances, otherwise only one node will receive the new block event forwarded to that port._
+
+### DAO genesis transaction
+
+#### Use the predefined setup
+
+The creation of the genesis tx is a bit cumbersome. To make it easier to get started you can use the [Bisq_DAO_regtest_setup.zip](https://github.com/bisq-network/bisq/tree/master/doc/Bisq_DAO_regtest_setup.zip) file.
+Extract the file and use those data directories for the Bitcoin Core as well as the Alice and Bob instances which are configured to have the genesis tx as defined in the above program arguments (`30af0050040befd8af25068cc697e418e09c2d8ebd8d411d2240591b9ec203cf` at height `111`).
+
+_You need to adjust the path to the `blocknotify` file inside of `bitcoin.conf` before starting Bitcoin Core._
+
+#### Setup a custom DAO genesis transaction
+
+To create your own genesis transaction follow those steps:
+
+- Send 2.50010000 BTC from Bitcoin Core to another address inside Bitcoin Core (label it with `Genesis funding address`).
+- Go to the send screen and open the coin control feature. Select the labeled transaction output of the address labeled with `Genesis funding address`. Use that as the only input source for the genesis tx.
+- Start Alice in full or lite node mode and go to the DAO/Wallet/Receive screen. Copy the BSQ address and use it for one of the receivers of the genesis tx. When pasting into Bitcoin Core remove the `B` prefix - that prefix is marking a BSQ address but technically it is a BTC address.
+- Do the same with Bob.
+- You send in sum exactly 2.5 BTC to both Alice and Bob. You can choose how to distribute it (e.g. 1 BTC to Alice 1.5 BTC to Bob).
+- Set the miner fee so that it is exactly the remaining 0.00010000 BTC. That might be the tricky part as miner fee selection is not very convenient in Bitcoin Core. In worst case if you cannot get the right miner fee you can add the difference to one of the receivers (e.g. send 1.0000234 BTC instead of 1 BTC).
+
+_Note: It is important that there is exactly 2.5 BTC spent entirely as described, otherwise the genesis tx is invalid._
+
+### Open questions?
+If there are any open questions or instructions are not clear, please add a PR for improving that file and/or join us on Slack and get in touch.

doc/bisq.conf

@@ -1,11 +0,0 @@
-# If that file is available in the data directory the defined properties will be used.
-
-# Here are the typical locations for the data directory:
-
-# macOS: /Users/username/Library/Application Support/bisq/
-# Linux: /home/username/.bisq/
-# Windows XP: C:\Documents and Settings\username\Application Data\bisq\
-# Windows Vista or 7: %appdata%/bisq/
-
-# Supported properties for bitcoin.network=regtest | testnet | mainnet
-bitcoin.network = regtest

doc/outdated/build.md

@@ -1,3 +1,5 @@
+TODO: Outdated, can be maybe removed, but cross check if there is still something useful in there.
+
 Building From Source
 ====================
 

doc/policy-file-setup.md

@@ -1,2 +0,0 @@
-In the IDE you need to add that VM option:
--Djava.security.manager -Djava.security.debug=failure -Djava.security.policy=./core/src/main/resources/bisq.policy

doc/rpc.md

@@ -1,40 +0,0 @@
-Setup or RPC calls to Bitcoin Core
-====================
-
-You need to setup the bitcoin.conf and add a blocknotify sh file. See the examples in the rec_regtest directory.
-
-To enable RPC calls you need to pass those program arguments:
---rpcUser=bisq --rpcPassword=bisqPW --rpcPort=18443 --rpcBlockNotificationPort=4159
-
-The default rpcPort for regtest from Bitcoin Core 0.16 and higher is: 18443
-The default rpcPort for testnet is: 18332
-For mainnet: 8332
-
-If you run 2 clients and want to receive the block notifications on both use different rpcBlockNotificationPorts
-5159 and 4159 are defined in the blocknotify file.
-
-For reg test setup with localhost those are typical program arguments:
---baseCryptoNetwork=btc_regtest --useLocalhostForP2P=true --nodePort=3332 --appName=bisq-LRTAli --fullDaoNode=true --rpcUser=bisq --rpcPassword=bisqPW --rpcPort=18443 --rpcBlockNotificationPort=4159
-
-
-For mainnet:
---fullDaoNode=true --rpcUser=bisq --rpcPassword=bisqPW --rpcPort=8332 --rpcBlockNotificationPort=4159
-
-If you use mainnet it is recommended to use a Bitcoin node with no funds in the wallet to avoid security risks when
-enabling RPC or take sufficient precautions from your network setup.
-
-In the bitcoin.conf file you need to set txindex=1.
-That causes a re-index of the whole data base which takes considerable time with a
-mainnet node.
-
-If you want to dump the blockchain data to json add: --dumpBlockchainData=true (used for BSQ block explorer)
-
-If you use RegTest in development environment you need to create the genesis transaction.
-Create one Bitcoin transaction from Bitcoin Core to one or 2 Bisq instances using the BSQ receive addresses from those apps (1 tx with 2 or more outputs to the Bisq app).
-If you copy the BSQ address and use that in Bitcoin Core you need to remove the "B" at the beginning. This is only for protection to mix up BTC and BSQ addresses but without the B it is a native Bitcoin address.
-Create one block with the debug command line inside Bitcoin Core (generate 1). Look up the block height in the info screen in the debug window.
-Set the block height and transaction ID at with options genesisBlockHeight and genesisTxId.
-Restart the Bisq apps. After that the app will recognize the received Bitcoin as BSQ.
-
-Here are example options for regtest mode:
---daoActivated=true --genesisBlockHeight=111 --genesisTxId=aa92a8d56be3aaafc6b1a8d248ae67c221d78a31de8867a9564e7ae24340b495 --useDevPrivilegeKeys=true --useDevMode=true--baseCurrencyNetwork=BTC_REGTEST  --useLocalhostForP2P=true --nodePort=3612 --appName=bisq-BTC_REGTEST_Bob_dao --fullDaoNode=true --rpcUser=bisq --rpcPassword=bisqPW --rpcPort=18443 --rpcBlockNotificationPort=5159

doc/rpc_regtest/bitcoin.conf

@@ -1,16 +0,0 @@
-regtest=1
-
-# txindex is needed to get non-wallet txs for apps
-txindex=1
-
-#rpc
-whitelist=127.0.0.1
-rpcallowip=127.0.0.1
-
-server=1
-rpcuser=bisq
-rpcpassword=bisqPW
-
-# we want to test with 2 local apps so we need the output at 2 diff ports. 
-# Unix process substitution does not work from Bitcoin Core, so we use a bash script 
-blocknotify=bash [PATH_TO_BTC_APP_DIR]/blocknotify %s

doc/rpc_regtest/blocknotify

@@ -1,4 +0,0 @@
-#!/bin/bash
-
-echo $1 | nc -w 1 127.0.0.1 4159
-echo $1 | nc -w 1 127.0.0.1 5159

doc/scripts/bisq-sn.cron.sh

@@ -1,4 +1,7 @@
 #!/bin/sh
+
+# Script for checking if seed nodes are available
+
 # Restart Bisq seed node daemons whose resident memory (RSS)
 # is over MAX_RSS_MiB.
 #

doc/scripts/run_bisq_via_vps.md

@@ -0,0 +1,63 @@
+## Hosted Bisq using VPS
+
+
+From: https://github.com/bisq-network/exchange/issues/511
+
+It's possible to install and view Bisq using a Google Compute Engine instance. This is handy and allows offers to always be visible, as well as the possibility to inspect the status of the client.
+
+This is how to host Bisq:
+
+Go to Google compute engine (https://cloud.google.com/compute/) and start an instance using the free trial. One gets $300 credit and 60 days trial period. I used one CPU, 40 GByte of boot disk, and the default value of RAM, 3.75 GB.
+I chose Ubuntu 16.04 LTS as the operating system.
+
+How to make a project and start an instance is explained in: https://goo.gl/1Ljy7O which also explains how to install a vnc-server. In short, connect via ssh to your instance, which can be done from the Google compute engine (gce) console, then execute the following script:
+
+```
+sudo apt-get -y update;
+sudo apt-get -y install tightvncserver;
+sudo apt-get -y install aptitude tasksel
+sudo tasksel install gnome-desktop --new-install;
+sudo apt-get -y install gnome-panel gnome-settings-daemon metacity nautilus gnome-terminal;
+sudo apt-get -y install ubuntu-desktop;
+sudo apt-get -y install gnome-session-fallback;
+mkdir .vnc;
+
+echo "#!/bin/sh
+[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
+[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
+xsetroot -solid grey -cursor_name left_ptr 
+vncconfig -iconic &
+x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
+x-window-manager &
+gnome-session &        
+gnome-panel &
+gnome-settings-daemon &
+metacity &
+nautilus &" > .vnc/xstartup;
+chmod 755 .vnc/xstartup;
+
+wget https://github.com/bisq-network/exchange/releases/download/v0.4.9.9.3/Bitsquare-64bit-0.4.9.9.3.deb;
+sudo dpkg -i Bitsquare-64bit-0.4.9.9.3.deb;
+
+```
+I call the script install.sh and execute it by 
+```
+$ bash install.sh
+```
+install.sh can be in $HOME/bin or for simplicity just under $HOME (your home directory).
+Then start vncserver with the command:
+```
+$ vncserver
+```
+which will prompt you for two passwords, one for interactive vnc and one for view-only vnc. The start vncserver again using the following command which allows vnc also after a logout:
+```
+$ nohup vncserver -geometry 1280x1024&
+```
+One then has to open the firewall on the instance which is best explained in: https://goo.gl/1Ljy7O
+This is done on the gce console in the web browser. It seems that google manages a firewall and it is not possible to open it from the instance.
+You can now login to your VPS desktop using the vncviewer on your local computer, using the IP-adress of your instance followed by the right port (5901), exemplified by 123.45.67.89:5901. There should be a nicely looking desktop in your vncviewer, either interactive or view-only depending on which password you used. I use Jump Desk as a vncviewer, which I bought for about $35 from Apple Store. Bisq is found under under the menu Applications/Other and may be opened by clicking it.
+
+I have tried to use vnc on my Android Nexus phone and can interact with Bisq also then (using Jump Desktop). It is thus seems possible to trade using a mobile phone.
+
+
+IMPORTANT  I have found that ssh from the gce console can become nonworking quite easily. I suggest that you find another safer way to access your vps-computer using, e. g., ssh from a terminal. I have had to redo this installation about six times now due to this serious problem.