Test applications built using `spin registry push`

[jsturtevant]

I'd love to add as a follow-up, testing applications built using spin registry push.

Originally posted by @radu-matei in https://github.com/spinkube/containerd-shim-spin/pull/36#pullrequestreview-1929364851

[rajatjindal]

could we please assign this to me. I wil pick this up next once my in-flight PR's are done.

[vdice]

@rajatjindal checking in to see if you are still working on this. This testing gap came up again recently in https://github.com/spinkube/containerd-shim-spin/issues/79.

[rajatjindal]

Hi @vdice

Thank you for the ping

This issue is one of the first things i have planned for next week.

This week i was busy with some other high priority stuff on my side.

[rajatjindal]

I have started work on this, and based on preliminary investigation, I think I am going to make following changes at a highlevel (this may change a bit once i actually start the implementation).

• break make up target into make up (to create k3d cluster) and make docker-images-import (to import images built using docker build into k3d cluster)
• Then I am going to make changes to all used spin.toml files to have build command associated with the components. (this is not really required for docker image build, but required for spin registry push)
• Next I am going to make changes to k3d config to start with an inbuilt registry support (https://k3d.io/v5.2.0/usage/registries/#using-k3d-managed-registries). This is where we will push OCI artifacts from spin registry push
• Once we have registry, I am going to introduce a make spin-registry-push target in Makefile. This will build and push spin apps to the k3d managed registry.
• now we can have these new OCI artifacts tested by adding support for them in workloads/*.yaml files and reusing same integration tests.

Questions:

• does that sound like a reasoanable plan? any suggestions are most welcome.
• one of the thing we have observed with testing in Spin is that building test artifacts on the fly adds up very quickly and results in slow CI times. any concerns with checked-in wasm files along side the testcases?

Thanks Rajat Jindal

[vdice]

That all sounds like a good way to go to me!

one of the thing we have observed with testing in Spin is that building test artifacts on the fly adds up very quickly and results in slow CI times. any concerns with checked-in wasm files along side the testcases?

Could we build the test apps concurrently with the usual shim build stage? The app test job/workflow will of course need to depend on that stage completing... but at least other items needed (shim build/test) are happening as well, so not a total waste.

Oh, we'll also want to push the same test apps to this project's ghcr.io repo (as we've been doing for the Docker-based apps) so that they can be publicly accessible. Just want to mention in case this affects your structuring -- though totally understandable to relegate for a follow-up. (And technically, if we don't mind the tests waiting, we could only push to ghcr.io instead of loading into a configured k3d registry and then have the tests pull from ghcr.io... defer to your judgement on what is best here. Definitely understand wanting to prioritize CI efficiency.)

[rajatjindal]

PR #90 opened for adding tests with "spin registry push"

[vdice]

Looks like this can be closed per https://github.com/spinkube/containerd-shim-spin/pull/90 right @rajatjindal?