pack
pack copied to clipboard
buildpacks
Local Layer Cache
Description
Operations like pack rebase and pack create-builder can be very slow when using the docker daemon. This happens because layers must be exported from docker and then loaded back in to create new images.
Proposed solution
This could be a lot faster if pack kept a layer cache on the host filesystem. Then, images would only need to read from the docker daemon if there was a cache-miss on the desired layer. This would be similar to how the launch-cache works in the lifecycle.
For example, if I am rebasing an image with 10 layers, 6 of which are run image layers the following happens
- The old app image is saved from docker
- A new image is loaded into the daemon containing layers from the new run-image + the 4 app specific layers from disk
Step 1 can be very slow, especially if the stack has a large base image.
If pack had a layer cache and that layer cache already contained the app layers (from a previous rebase, for example) we could skip the first step and proceed directly to step 2.
create-builder could be sped up for similar reasons. If I am creating a builder from a series of daemon image build packages pack currently saves every buidpack to disk before creating (loading) the builder. With a layer cache, pack wouldn't need to waste time saving buildpacks that it had used before.
Describe alternatives you've considered
- Drop support for the daemon ;)
- Deal with it
Additional context
Lifecycle uses a CachingImage to produce behavior similar to what is describe above without changing the image creation code. However, this shouldn't be used directly because it cleans up unused layers after each build. pack would probably want to allow the cache to grow forever (and provide an easy way to prune it).
I'm honestly a little lost on the term cache given there are various caching layers in place. I wonder if a formal RFC for this proposal would provide some clarity and better format for discussion.
