golem icon indicating copy to clipboard operation
golem copied to clipboard

Speed up CI by caching compiled WASMs in Github Actions cache

Open vigoo opened this issue 1 year ago • 2 comments

We could probably significantly speed up the CI test tasks by caching the test component's compiled native images in a Github Action cache.

This is not straightforward to do, as we cannot use Golem's built-in compilation cache - it is per component, and many tests are intentionally creating new components even though they use the same source WASM file. So this should be some kind of wasm hash-based file system cache, enabled only for tests, and explicitly saved/restored on CI.

Note: The wasmtime CLI supports something like this (https://docs.wasmtime.dev/cli-cache.html), so maybe there is something available in wasmtime that makes implementing this easy - but even if not, it can be done the same way as the live golem compilation cache works.

vigoo avatar Sep 19 '24 14:09 vigoo

The cache key for this must be separate and only calculated from the test component WASMs, which are changing relatively infrequently.

vigoo avatar Sep 19 '24 14:09 vigoo

The aim is for multiple pipelines to use the same built component, or from within a given run, should we use the component N times without recompiling it?

Mb about the possibly naive question, but I am currently trying to gain experience around this specific project/codebase

wesleymatosdev avatar Oct 10 '24 05:10 wesleymatosdev