rescript-compiler icon indicating copy to clipboard operation
rescript-compiler copied to clipboard

Boost CI speed

Open cometkim opened this issue 1 year ago • 7 comments

Strategies

Mostly using actions/cache

build-rewatch

Skip build if there are no changes in rewatch sources and its dependencies.

build

Cache APT dependencies

Pre-install Linux dependencies more efficiently using cache-apt-pkgs-action

Cache OPAM installation

Skip OPAM installation as much as possible.

Once it installed, there are three paths can be cached:

  • ${{ runner.tool_cache }}/opam/...: Shared storage for binaries installed by setup-* actions. It will contain opam executable.
  • ~/.opam: The home directory of OPAM, every modification by OPAM will placed there.
  • _opam: Local switch

After caching these three, we can restore OPAM installation by adding PATH and exporting opam env.

Install JSOO ahead of the job

It can be cached with other deps

Results

build-rewatch on ubuntu-latest:

  • before: 50s~90s
  • after (cached): ~15s

build on ubuntu-latest:

  • before: ~360s
  • after (cached): ~120s

cometkim avatar Jun 29 '24 20:06 cometkim

The Windows build failure is due to shell compatibility. ~~Fix should be trivial~~ Danm it wasn't

cometkim avatar Jun 30 '24 01:06 cometkim

Note: managing cache

The cache of a public repository is limited to 10GB in total. Once a repository has reached its maximum cache storage, the cache eviction policy will create space by deleting the oldest caches in the repository.

If we want to invalidate the cache at some point, there are several options:

  • Change the cache key in the workflow file.
  • Change the cache key by combining it with repository variables. Then admins can expire multiple caches at once without creating a new commit.
  • Delete directly from the management UI ([repo_url]/actions/caches)
  • Use the GitHub CLI. Need gh extension install actions/gh-actions-cache

cometkim avatar Jun 30 '24 07:06 cometkim

We can get much faster feedback by separating misc checks like format or lint from the build and running them upfront.

cometkim avatar Jun 30 '24 07:06 cometkim

still trying to fix windows one...

cometkim avatar Jun 30 '24 11:06 cometkim

hahahahahaha I won this war

cometkim avatar Jun 30 '24 16:06 cometkim

uh. a new error comes from... tmp file? I have no ideas about this :/

Run if git diff --name-only --exit-code HEAD^ HEAD -- jscomp/syntax; then
D:\a\_temp\4f0df85e-0898-4cc0-8a90-5090447cadce.sh: line 6: syntax error: unexpected end of file

cometkim avatar Jun 30 '24 16:06 cometkim

@cknitt I think this will work stably. any remaining feedback?

cometkim avatar Jun 30 '24 17:06 cometkim