go-algorand icon indicating copy to clipboard operation
go-algorand copied to clipboard

make install needs to be simplified.

Open winder opened this issue 2 years ago • 3 comments

Subject of the issue

While briefly investigating #3913 it became clear that our scripts are an unstable foundation. This has been brought up before with #2131, but I'd like to make a more concrete suggestion for how to start.

While trying to figure out where build flags come from I went through the exercise of reviewing the call tree and was surprised by what I found:

  1. make install depends on build, but lets ignore that for a moment.
    • build: is an alias for buildsrc
    • buildsrc depends on libsodium and some scripts
    • finally go install is called with some flags which change based on the architecture.
  2. make install calls scripts/dev_install.sh
  3. dev_install.sh is a 30 line script which simply calls scripts/local_install.sh
  4. local_install.sh is a 70 line script that does little more than call scripts/build_package.sh.
    • It also inexplicably calls update.sh depending on what environment variables are set, which would download external binaries for no reason.
  5. build_package.sh which calls "scripts/build_prod.sh"
    • after calling build_prod.sh, it inexplicably copies binaries into ./tmp/dev_pkg
  6. build_prod.sh is a 20 line script with literally one line that isn't a comment.
    • it calls make build which the Makefile has already called.

These sorts of issues cause all downstream items to be complicated including:

  • testing
  • continuous integration
  • docker containers
  • releases

First steps

  1. remove build and install targets.
  2. rename buildsrc to install.

Acceptance Criteria

Acceptance criteria is in the first steps above (but also search for uses of the targets).

If feeling ambitious, go deeper to simplify the build scripting. Consider build recipe to create build targets up front, as opposed to recalculation in between steps. However, this is just a stepping stone.

winder avatar Apr 24 '22 18:04 winder

Also, there is evidence that we are not running all the checks in CI. For example consider bundleGenesisInject.go which was out of sync with bundle_genesis_json.sh until this PR. You would think that CI should not be letting builds through.

tzaffi avatar Apr 25 '22 15:04 tzaffi

Dependencies are going to require careful revalidation, especially through testing.

onetechnical avatar Jul 05 '22 17:07 onetechnical

Despite a desire to completely overhaul the entire process, starting with this ticket might be a good thing to get us a deep understanding of how these scripts work plus their dependencies.

onetechnical avatar Aug 08 '22 17:08 onetechnical