juice icon indicating copy to clipboard operation
juice copied to clipboard

[CI] cache artifacts better / increase efficiency

Open drahnr opened this issue 4 years ago • 1 comments

currently everything is recompiled + fetched from remote which is not necessary

  • [ ] figure out how to use sccache from concourse without exposing the host network, probably via DNS entries
  • [ ] inject a squid based http cache as proxy for concourse
  • [x] use concourse's directory caching feature for caching the local cache of cargo
  • [x] flatten container images by using better concourse resource types without docker, i.e. https://github.com/concourse/registry-image-resource + https://github.com/vito/oci-build-task - currently blocked by https://github.com/concourse/registry-image-resource/issues/87

drahnr avatar Mar 23 '20 08:03 drahnr

Recompilation is mostly avoided now, by caching $CARGO_HOME between runs which helps significantly.

Lately the biggest bottleneck has been containerized io performance when executing and slow image retrieval due to quay.io having put some funny limits into place.

drahnr avatar Mar 30 '21 08:03 drahnr