go-ethereum icon indicating copy to clipboard operation
go-ethereum copied to clipboard

Offline cache for github worker

Open karalabe opened this issue 1 year ago • 3 comments
trafficstars

We have a github action and a dedicated worker for it to run tests fast... but the thing caches Go build files into GitHub (uploading and downlaoding 3.5GB), so it takes 8 minutes to start the tester, 20 secs to test and 8 minutes to stop the tester. Crazy. We need to make the tester cache locally, since our dedicated builder is a unique snowflake.

Also should enable race detector runs on it.

karalabe avatar Jun 04 '24 13:06 karalabe

Is there any specific Github Actions Workflow YAML file that you would like to consider? I could only find go.yml. I would suggest modifying the Workflow YAML file. Default paths of interest are ~/.cache/go-build and $GOPATH/pkg/mod. Might be more paths are of interest.

For developers who are not maintainers of this repository, I would suggest performing a personal fork and test over it.

I attach my timing results in the following screenshot for a Fork of the current project, where go.yml is executed using my own machine. Kindly observe the first iteration to be approximately 4 minutes duration. However, for iterations after it, times seem to be optimized. image

sisco0 avatar Jun 23 '24 06:06 sisco0

@karalabe @parithosh had something in the pipe to solve this. Not sure what the current state is.

Basically, the cache is big because it's never cleared. It should be cleared more often.

gballet avatar Oct 15 '24 09:10 gballet

also FYI it is a local cache, it's just that copying it takes forever

gballet avatar Oct 15 '24 09:10 gballet

cc @elasticroentgen , How/when do we want to migrate the geth github actions runner into your infra?

parithosh avatar Oct 25 '24 15:10 parithosh

as stated on our devops ticket https://github.com/ethereum/devops/issues/1507 i've set up a potential solution for this, but never received feedback on it. If it works as expected (see the linked issue) i can assist in making the switch.

elasticroentgen avatar Oct 25 '24 15:10 elasticroentgen