Image deployment process needs improvement

[Pascal666]

As far as I can tell, newly built images are being deployed to the image server without any testing being done first. Once deployed they are tested, and if they fail, they are simply left there.

It would be nice if:

1. Images were tested before being deployed to the image server and only deployed if they pass.
2. Images on the image server that fail testing were removed.

Thank you.

[Pascal666]

https://jenkins.linuxcontainers.org/job/lxd-test-images/lastFailedBuild/restrict=master,type=container/console shows 5 broken container images are currently on the image server.

https://jenkins.linuxcontainers.org/job/lxd-test-images/lastFailedBuild/restrict=master,type=vm/console shows 5 broken VM images are currently on the image server.

[stgraber]

It's currently not something that we have the time or resources to do.

I agree it'd be nice to have images be gated on the test but that'd require running tests as images get built, then have the images be pulled from Jenkins rather than the image server, be tested and have the publishing logic look for the test result when scanning Jenkins for things to publish.

It's certainly doable but would likely take a week or so of my time to put in place and would increase resource usage on our side quite significantly too.

Until then, https://images.linuxcontainers.org/ has a pretty clear warning that those are unofficial image that are built for convenience and that we don't make any guarantees about them.

[Pascal666]

Would it be possible to only update the alias once an image has passed tests? So still deploy them to the image server as soon as they are built but don't update the alias, modify the tester to pull down the latest fingerprint for each image type and update the alias for any images that pass?

[stgraber]

No as there's no such thing as an alias on the image server. It's just an image index and it's the LXD client which locally generates those aliases, picking the image for your local architecture, that's compressed with something your machine can decompress and then picking the most recent build.

[stgraber]

If the image server was running LXD, it's something we could do, but the image server is a static web server providing simplestreams metadata (https://images.linuxcontainers.org/streams/v1/images.json)

[tomponline]

Would be good to consider looking at this issue again based on https://github.com/lxc/lxd/issues/10309

[stgraber]

We have a card for it, just don't have time to work on it.

[kolkov]

I can's see any Gentoo images when call lxc-create t download https://github.com/lxc/lxc/issues/4129

[dontlaugh]

Perhaps there is a way to get some improvement with less effort, while maintaing the best-effort/"unofficial" stance of the LXD team's image servers.

One idea is to introduce an entirely new server, for example stable:, and images that pass the existing test suite could be copied over after the fact.

Presumably this wouldn't break existing clients, but - if a new remote is added - allow some people to opt-in to a marginally more stable upstream.

[stgraber]

Images are now automatically tested prior to publishing.