[daemon] unaware of a custom URL changing

[Saviq]

Steps:

• multipass launch snapcraft:core16
• modify src/daemon/custom_image_host.cpp to change the URL the core16 alias points to
• multipass launch snapcraft:core16

Expected:

• new image is downloaded and used

Current:

• cached image is used

I thought we were using the image URL to id the image, so should realize there's a new one?

[townsend2010]

Thinking about this some more, I'm thinking we need to change the launch behavior when dealing with cached images. I propose the following:

• By default, launch should download a newer image, if available.
  • If determining if a newer version is not possible, ie, network issues, just use the cached image and print out a message on the client saying as such.
• Introduce a new --use-cached launch option. This will force launch to use the cached version even if a newer version is available.
  • If this option is given and no cached image is available, the operation should fail.

Any thoughts on my proposal? I've looked over the code and trying to force the "custom" remotes to always use the newest version vs. using the release and daily remotes to use the cached versions is not going to be trivial to implement and besides, I prefer everything to be handled in a consistent manner.

[Saviq]

I think we decided to launch cached because that makes the user experience a bit nicer - in theory the image should be at most 6h old. There's caveats like network being unavailable when we try to refresh (which we should cater for, ideally listening for the system's network state events) of course.

So I'd go for the opposite after all - a --refresh-images flag or so, which would, on launch ensure the freshest image is used.

That doesn't mean that what this bug is about (the URL of an image changing) should only take effect after the 6h timeout… The URL should be part of the cache key, and requesting an image from the cache after a URL change would be a cache miss.

Images coming from streams would have to be handled differently indeed.

[townsend2010]

Ok, so I think the way to fix this is to save the remote's URL in the JSON containing the image information. Then when fetching image info for launch, it will compare the current URL with what is saved in the JSON, and if different, then the image is invalidated and a new image will be downloaded. In the cases where the daemon starts and the json doesn't have the remote's URL, I suppose we treat that as we do now and use the cached image. Thoughts?

[Saviq]

@townsend2010 this would work for our custom remote, but for streams-sourced images, every update is a new URL, so that can't be a universal treatment.

[townsend2010]

@Saviq, I mean the URL of the remote itself. For example, we use http://cloud-images.ubuntu.com/releases/ for the release remote and http://cloud-images.ubuntu.com/daily/ for the daily remote. Those URL's should not change, but if they ever do, then I think the cached images should be invalidated since it would probably be a whole new thing.

[Saviq]

@townsend2010 I think we just need to take a step back and identify the valid cache keys for each of the use cases. When we introduce configurable user images, we won't have a "remote" in that sense, rather it will just be the image URL.

[townsend2010]

@Saviq, I agree this needs more thought. Let's discuss further next week :wink:

[Saviq]

Just to record what we discussed:

• every image host (streams or otherwise) needs to be able to declare two refresh cycles:
  1. query resolution (e.g. "bionic", "16.04", "http://..." to URL)
  2. image refresh
• by default, stream-based ones will have e.g. a 6h cycle for both
• by default, custom ones need "null" for the query resolution (i.e. always resolve the URL)
• the image cache then becomes a simple file cache keyed on URL, with support for "prefer-{cache,network}" behaviour
• users will have an option to configure the cache behaviour for each of those
• and a way to ask to --prefer-{cache,network} when launching (UX TBD)