nerdctl icon indicating copy to clipboard operation
nerdctl copied to clipboard

Experimental: GHA cache image build dependencies

Open apostasie opened this issue 1 year ago • 1 comments

Folllow-on to discussion in #3580

What this does:

  • enable GHA cache for docker build
  • in the Dockerfile, separate third-party dependency build and nerdctl integration test building
  • separate out the step that builds these dependencies from the steps that prepare the integration image

The result is:

  • instead of building everything 9 times (4 rootful, 4 rootless, ipv6) , we only build the dependencies 4 times
  • the dependencies stage is then cached, and subsequent runs on the same branch will not rebuild it
  • note that we SHOULD get the cache from the main branch as well, so, once it is merged / hot, new PRs should benefit from the cache as well on their first run

The key benefit here is really to significantly reduce the network traffic required to produce out test images (hence reducing the opportunity for failure due to third-party server hiccups).

Incidentally, we will also get a small speed boost for the overall run - from about 4 min 30 seconds on average, to 3 mins 30 seconds.

Obviously, GHA cache is not "free", as in: it takes time to retrieve and time to store - so, part of the time gains from not-building are negated by the cache r/w.

Nevertheless, this looks promising for increased reliability (and reduced transactions with docker hub / debian / ubuntu).

Further refactoring / changes to the Dockerfile could bring more stuff in the deps stage. This PR has been conservative on that front and staid with the minimal possible changes there.

Finally note that I would rather have #3535 merged first - and this rebased on top of it.

apostasie avatar Oct 21 '24 06:10 apostasie