buzzmobile icon indicating copy to clipboard operation
buzzmobile copied to clipboard

Cache the installation build with DoCIF

Open irapha opened this issue 8 years ago • 4 comments

DoCIF has the option to cache the installation build with Docker, which cuts down on ci checks' compute time. We'd like to enable that.

This issue is blocked on us having used the install script for development for a while, to make sure the install is working properly, and avoid having to deal with a bad install being cached.

irapha avatar Jan 15 '17 22:01 irapha

I was thinking about this, and I reached out to @jgkamat yesterday. However I'm not sure its an issue as long as we aren't building PySide, pyqt4 installs quickly and our recent builds are ~8 minutes (which is faster than, for example robojacket's roboracing-software build, which is a similar project). While its a thing to think about, I'm by no means worried yet.

In any case, the method seems to be to create an account on dockerhub and do it that way (which would also allow you to download a docker image to local as a build environment). Hopefully Jay can elaborate.

joshuamorton avatar Jan 15 '17 22:01 joshuamorton

Oh I didn't mean to imply this was an issue. It's a nice-to-have :)

Cutting down those 8 minutes would make me happy haha

irapha avatar Jan 15 '17 22:01 irapha

aight. If on the other hand we do start building with pyside (which we can't with indigo), then we will need something, because it would likely double our build time if not more.

joshuamorton avatar Jan 15 '17 23:01 joshuamorton

Hi!

If you need any help setting up build caching, I can help out. The result would mean that your builds will probably take 1-2 minutes to get dependencies, so if your dependency install step takes more than that, this might be a good option for you. I've been seeing a lot of fluctuations recently though, with downloading taking ~3 minutes, and I think that may be a recent change in CircleCI's infrastructure. I'll have to work on that a bit.

When a baseimage is built (everything in the SETUP_COMMAND variable), DoCIF will commit a snapshot of the built baseimage, and upload it to the docker hub. On the next build, DoCIF will take a sha of the files defined in SETUP_SHA_FILES, and check the docker hub to see if a previously built image exists, and if so, it will download the image from the docker hub instead of building it from scratch.

In order to do this, you will need to make sure your SETUP_SHA_FILES is accurate, SETUP_COMMAND is accurate, and your DOCKER_*_VAR variables are set with your credentials from the docker hub.

Let me know if you run into any issues, (either here or on an issue on DoCIF), I'd be happy to help you out!

EDIT: you will also need to set BASEIMAGE_REPO and set PUSH_BASEIMAGE to true.

jgkamat avatar Jan 15 '17 23:01 jgkamat