Because we want to avoid local replace directives (as they make it easy
to screw up things for external applications that use lnd as a library
without us noticing), we're going to switch over to a new process.
See section "Use of Golang submodules" in the code contribution
guideline.
With this commit we add a new restriction that checks that the
username in the signature file (manifest-<username>-<version>.sig) actually
does have a signing key and that the signature was created with that key.
The signature is only counted towards the minimum of 5 signatures if
that check is successful.
The changes in this commit were inspired by @kixunil in #5048.
Instead of importing the keys from Keybase, we add all the signing keys
to the repository. In addition to having the key file present each key's
ID must also be added to the verification script. This acts as a double
check that the correct key is added and makes a key change more
explicit.
This removes a vulnerability brought up by @Kixunil where both the lnd
and lncli binaries are executed to obtain their version before they have
been verified against the release. A malicious binary could have already
compromised the user's system before any of the checks had been
performed.
This commit it split out from the doc changes so that it can easily be
cherry-picked to master/0.13.0.
Without this commit, the build may break if a PR commit is created, as
that shows the `PR_NUMBER` field we parse out as "master", reflecting
that its a merge commit into the master branch.
In this commit, we add a simple bash script to parse out the current PR
number from an environment variable in the GH actions context, and use
that to check to see if the PR has been referenced in the release notes
or not. This isn't 100% fool proof, but it should catch most of the
common cases.
The shasum command isn't available in Alpine linux while the sha256sum
command isn't available on MacOS. We add a simple switch that tries to
detect which one is available.
Due to a misunderstanding of how the gpg command line options work, we
didn't actually create detached signatures because the --clear-sign
flag would overwrite that. We update our verification script to now only
download the detached signatures and verify them against the main
manifest file.
We also update the signing instructions.
Instead of only allowing the installed versions of lnd and lncli to be
verified, we now also support specifying explicit paths to binaries that
we want to verify.
When verifying the release signatures, we don't want to fail if a
signer's signature is not available in the gpg key ring. Instead we just
don't want to count the signature for now and still succeed if there's
at least one other good sig with a known key.
The verification script makes sure the hashes of the binaries inside of
a docker image match those of an official release.
The script first downloads all signatures, validates them, then compares
the hashes of the installed binaries to those contained in the detached
signature files.
We add a new make goal called release-install that creates the same
reproducible binaries as the release script would create, but only for
the current OS/architecture. It then installs those binaries to the
system's GOBIN directory.
To allow easy verification of individual binaries (instead of just the
packag tarballs/zips), we also add the hashes of lnd and lncli binaries
to the manifest. We do the same in the docker build.
A number of tricks need to be applied to the tar and zip commands
respectively to make sure they always produce deterministic archives.
This includes setting the file timestamps to a fixed date and setting
specific ownership attributes (tar) or no attributes at all (zip).
To make sure we build the exact version of btcd that is referenced in
the project's go.mod file and to not overwrite any binary the user might
already have installed on the system, we compile btcd into an explicit
file in the itest directory.
This should also speed up invocations of "make itest-only" because the
test harness doesn't always compile btcd on its own.
We also fix a bug with the version parsing where adding a "replace"
directive in the go.mod would result in the awk commands to extract the
wrong version. Because we no longer use the DEPGET goal to build and
install btcd, using a replace directive now actually works for itests.
Because the bitcoind mirror is extremely slow, we spend at least 2 to 3
minutes of each bitcoind related test on just downloading the binary. We
can achieve the same result by just pulling the docker image and
extracting the binary from that.
Because this script is now run from the main Makefile, it is more
suitable to be located in the main scripts folder where the other
build related scripts reside.
