ch-image: use previously prepared files

[j-ogas]

Related to #1309.

ch-image doesn't track or store prepared files at push time.

There is not currently any support for re-using any previously prepared files already in the upload cache, because we don't yet have a way to know if these have changed until they are already built.

[BenStormer]

The build-cache enables us to track and map previously prepared upload files, e.g., layer tarballs, manifest, and config metadata, to images in the build-cache storage.

I propose the following behavior changes:

• build cache disabled: no changes, prepare upload files at push time as we do now

• build cache enabled:

  1. move “data created” config metadata to build process, at the end of the main loop. This prevents the image from getting a different timestamp each time it is pushed

  2. store prepared image files as follows in ulcache. - SHA256.tar.gz (layer(s) - NAME.STATE_ID.config.json - NAME.STATE_ID.manifest.json

  3. at push time, if all files exist, use them; otherwise, prepare the files starting with the layer(s), config, and then manifest. Replace any existing files.

  4. create a “ch-image upload_cache" subcommand with "--clean” option. Default prints cache contents and perhaps total size of ulcache.

[reidpr]

In general this seems like a reasonable approach to me. A few comments:

1. I think there might be two separate issues being conflated here. One is that Charliecloud generated different files every time you push (e.g., the configs have timestamps in them). The second is that files are generated anew every time. Does it make sense to address these separately?

2. Item 2.ii seems like it depends on multi-layer push. That is, how much value is there in time savings to push an identical single-layer image more than once?

3. Is there a reason to have an upload cache disabled mode?

4. The upload cache probably needs a size limit.