singularity icon indicating copy to clipboard operation
singularity copied to clipboard

Ability to build images without online library

Open nr23730 opened this issue 3 years ago • 1 comments

Is your feature request related to a problem? Please describe. Currently (Tue, 01/18/2022) the sylabs cloud is down for maintenance. Building images that rely on the library is not possible, even if the image is already locally cached. See following error message: FATAL: While pulling library image: error fetching image: image does not exist in the library: debian:10 (amd64)

Describe the solution you'd like I'd like to see a change of behavior in a way where building an image is still possible by using the local image cache. Of course a warning to the user would be appropriate.

Describe alternatives you've considered Another option might be to have a prompt stating something like "Connecting to the library failed. Do you want to use the latest locally cached version the image instead?".

nr23730 avatar Jan 18 '22 15:01 nr23730

Agreed this would be useful. At present Singularity only stores cache entries by an image's content hash, and does not have a record of the tags associated with a cache entry, so it has to use the server to resolve the tag :10 to a content hash before checking presence in the cache directory.

This fairly minimal approach to caching information has benefits given we are running across multiple hosts without a coordinating daemon, since the image hashes are unique and don't change, and atomic operations on files are sufficient to add new cache entries when needed, in a way that avoids clashes when multiple systems attempt to do this concurrently. Keeping a tag -> hash dictionary on the system and maintaining it on changes is difficult to do reliably when the cache is on shared filesystems with potentially unknown availability / scope of locking.

I'll tentatively add some more investigation of this to the roadmap.

dtrudg avatar Jan 20 '22 14:01 dtrudg