bisq/.gitattributes

# Auto detect text files and normalize line endings to LF
# This will handle all files NOT found below
* text=auto
# These text files should retain Windows line endings (CRLF)
*.bat           text eol=crlf
# These binary files should be left untouched
# (binary is a macro for -text -diff)
*.bmp           binary
*.gif           binary
*.ico           binary
*.jar           binary
*.jpg           binary
*.jpeg          binary
*.png           binary
p2p/src/main/resources/*BTC_MAINNET filter=lfs diff=lfs merge=lfs -text
Add .gitattributes file 2018-11-16 07:34:05 +01:00			`# Auto detect text files and normalize line endings to LF`
			`# This will handle all files NOT found below`
			`* text=auto`
			`# These text files should retain Windows line endings (CRLF)`
			`*.bat text eol=crlf`
			`# These binary files should be left untouched`
			`# (binary is a macro for -text -diff)`
			`*.bmp binary`
			`*.gif binary`
			`*.ico binary`
			`*.jar binary`
			`*.jpg binary`
			`*.jpeg binary`
			`*.png binary`
Track p2p data store files using Git LFS The large binary objects in p2p/src/main/resources/ are updated on every Bisq release with the latest network data to avoid the need for new Bisq clients to download all of this information from the network, which would easily overload seed nodes and generally bog down the client. This approach works well enough for its purposes, but comes with the significant downside of storing all of this binary data in Git history forever. The current version of these binary objects total about 65M, and they grow with every release. In aggregate, this has caused the total size of the repository to grow to 360M, making it cumbersome to clone over a low-bandwith connection, and slowing down various local Git operations. To avoid further exacerbating this problem, this commit sets these files up to be tracked via Git LFS. There's nothing we can do about the 360M of files that already exist in history, but we can ensure it doesn't grow in this unchecked way going forward. For an understanding of how Git LFS works, see the reference material at [1], and see also the sample project and README at [2]. The following command was used to track the files: $ git lfs track "p2p/src/main/resources/*BTC_MAINNET" Tracking "p2p/src/main/resources/AccountAgeWitnessStore_BTC_MAINNET" Tracking "p2p/src/main/resources/BlindVoteStore_BTC_MAINNET" Tracking "p2p/src/main/resources/DaoStateStore_BTC_MAINNET" Tracking "p2p/src/main/resources/ProposalStore_BTC_MAINNET" Tracking "p2p/src/main/resources/SignedWitnessStore_BTC_MAINNET" Tracking "p2p/src/main/resources/TradeStatistics2Store_BTC_MAINNET" We are using GitHub's built-in LFS service here, and it's important to understand that there are storage and bandwidth limits there. We have 1G total storage and 1G per month of bandwidth on the free tier. We will certainly exceed this, and so must purchase at least one "data pack" from GitHub, possibly two. One gets us to 50G storage and bandwith. In an attempt to avoid unnecessary LFS bandwidth usage, this commit also updates the Travis CI build configuration to cache Git LFS files, such that they are not re-downloaded on every CI build (see [3] and [4] below). With that out of the way, the variable determining whether we exceed the monthly limit is how many clones we have every month, and there are many, though it's not clear how many are are Travis CI and how many are users / developers. Tracking these files via LFS means that developers will need to have Git LFS installed in order to properly synchronize the files. If a developer does not have LFS installed, cloning will complete successfully and the build would complete successfully, but the app would fail when trying to actually load the p2p data store files. For this reason, the build has been updated to proactively check that the p2p data store files have been properly synchronized via LFS, and if not, the build fails with a helpful error message. The docs/build.md instructions have also been updated accordingly. It is important that we make this change now, not only to avoid growing the repository in the way described above as we have been doing now for many releases, but also because we are now considering adding yet more binary objects to the repository, as proposed at https://github.com/bisq-network/projects/issues/25. [1]: https://git-lfs.github.com [2]: https://github.com/cbeams/lfs-test [3]: https://docs-staging.travis-ci.com/user/customizing-the-build/#git-lfs [4]: https://github.com/travis-ci/travis-ci/issues/8787#issuecomment-394202791 2020-04-29 11:59:48 +02:00			`p2p/src/main/resources/*BTC_MAINNET filter=lfs diff=lfs merge=lfs -text`