PINT icon indicating copy to clipboard operation
PINT copied to clipboard

Speed up CI

Open mattsse opened this issue 4 years ago • 5 comments

Issue summary

  • The CI appears to recompile everything on every PR, ideally we should cache at least all the polkadot/substrate related dependencies.

Other information and links

mattsse avatar Sep 14 '21 08:09 mattsse

Already cached with sccache

  • https://github.com/ChainSafe/PINT/blob/84acda4f4ff00467d351dc9a8476ef60a40e174e/.circleci/config.yml#L90
  • https://github.com/ChainSafe/PINT/blob/84acda4f4ff00467d351dc9a8476ef60a40e174e/.circleci/config.yml#L53

clearloop avatar Sep 14 '21 08:09 clearloop

hmm judging from the CI logs, it appears the environment is always set up new and all dependencies installed, but I don't really have much experience with configuring this for optimization.

mattsse avatar Sep 14 '21 09:09 mattsse

Caching target directory will cause some related issues, for example

  • we remove a feature and related code in one PR
  • we get back the feature but forget to get back the related code
  • the project still can compile

clearloop avatar Sep 15 '21 03:09 clearloop

Please add your planning poker estimate with ZenHub @clearloop

mattsse avatar Sep 15 '21 14:09 mattsse

FYI

  • There is a tool for optimizing docker layer caching for rust (see https://github.com/LukeMathWalker/cargo-chef)
  • But this only works for local development and circleci docker caching is working poorly. So I would suggest using kaniko cache strategy (see https://github.com/GoogleContainerTools/kaniko#caching), this allows storing cache layers in docker registry.

ptqa avatar Nov 02 '21 11:11 ptqa