operator-framework
Can operator-sdk run bundle use local image?
Type of question
Hi, I am writing operator using operator-sdk. Recently I was looking for a solution on how to do e2e testing. I chose to use kind as a temporary test environment.
I started the kind node locally, and then built the operator image and bundle image locally. I used kind load docker-image to load the two images to the kind cluster. When I tried to initialize the test suite environment through operator-sdk run bundle before the test, I found that operator-sdk would pull the images through the network.
In CI environment, these operations are temporary, in order to verify that there is no problem with my logic, I think operator-sdk should first try to use the local bundle image, instead of pulling the remote repository, otherwise you need to provide a test repository to store the temporary image or special version. I think this mechanism is bad for development testing.
I want to ask is this something wrong with me or does operator-sdk run bundle always pull the image of the remote repository?
Question
What did you do?
make bundlemake bundle-buildkind load docker-image quay.io/zncdata/spark-k8s-operator-bundle:vv0.1.0operator-sdk run bundle quay.io/zncdata/spark-k8s-operator-bundle:vv0.1.0
output:
INFO[0001] trying next host - response was http.StatusNotFound host=quay.io
FATA[0001] Failed to run bundle: pull bundle image: error pulling image quay.io/zncdata/spark-k8s-operator-bundle:vv0.1.0: error resolving name for image ref quay.io/zncdata/spark-k8s-operator-bundle:vv0.1.0: quay.io/zncdata/spark-k8s-operator-bundle:vv0.1.0: not found
What did you expect to see?
operator-sdk shoule use local image builded with docker, rather then pull bundle.
What did you see instead? Under which circumstances?
above
Environment
Operator type:
golang
Kubernetes cluster type:
$ operator-sdk version
operator-sdk version: "v1.33.0", commit: "542966812906456a8d67cf7284fc6410b104e118", kubernetes version: "v1.27.0", go version: "go1.21.5", GOOS: "darwin", GOARCH: "arm64"
$ go version (if language is Go)
go version go1.21.5 darwin/arm64
$ kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.4", GitCommit:"f89670c3aa4059d6999cb42e23ccb4f0b9a03979", GitTreeState:"clean", BuildDate:"2023-04-12T12:13:53Z", GoVersion:"go1.19.8", Compiler:"gc", Platform:"darwin/arm64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.3", GitCommit:"25b4e43193bcda6c7328a6d147b1fb73a33f1598", GitTreeState:"clean", BuildDate:"2023-06-15T00:38:14Z", GoVersion:"go1.20.5", Compiler:"gc", Platform:"linux/arm64"}
Additional context
By reading the source code, I found that the mirror implementation logic supports getting the image from the local, but the local mirror logic is not checked at the time of calling.
https://github.com/operator-framework/operator-sdk/blob/4abd483aa4004af83b3c05ddf9d7a7e4d4653cc7/internal/registry/image.go#L30-L72
But at the time of the call, no local judgment is made
https://github.com/operator-framework/operator-sdk/blob/4abd483aa4004af83b3c05ddf9d7a7e4d4653cc7/internal/olm/operator/helpers.go#L37-L40
I was wondering if anyone could fix this?
I recommend installing local registry like instructed here: https://kind.sigs.k8s.io/docs/user/local-registry/
After that, you can push to that registry and run a local bundle. For example, my quick test:
➜ cass-operator git:(master) ✗ bin/operator-sdk run bundle localhost:5001/k8ssandra/cass-operator-bundle:v1.20.0-dev.faa4d28-20240404 --use-http
INFO[0008] Creating a File-Based Catalog of the bundle "localhost:5001/k8ssandra/cass-operator-bundle:v1.20.0-dev.faa4d28-20240404"
INFO[0008] Generated a valid File-Based Catalog
INFO[0016] Created registry pod: 001-k8ssandra-cass-operator-bundle-v1-20-0-dev-faa4d28-20240404
INFO[0016] Created CatalogSource: cass-operator-catalog
INFO[0016] OperatorGroup "operator-sdk-og" created
INFO[0016] Created Subscription: cass-operator-v1-20-0-dev-faa4d28-20240404-sub
INFO[0023] Approved InstallPlan install-kbnqd for the Subscription: cass-operator-v1-20-0-dev-faa4d28-20240404-sub
INFO[0023] Waiting for ClusterServiceVersion "default/cass-operator.v1.20.0-dev.faa4d28-20240404" to reach 'Succeeded' phase
INFO[0024] Waiting for ClusterServiceVersion "default/cass-operator.v1.20.0-dev.faa4d28-20240404" to appear
INFO[0026] Found ClusterServiceVersion "default/cass-operator.v1.20.0-dev.faa4d28-20240404" phase: Pending
INFO[0028] Found ClusterServiceVersion "default/cass-operator.v1.20.0-dev.faa4d28-20240404" phase: Installing
INFO[0089] Found ClusterServiceVersion "default/cass-operator.v1.20.0-dev.faa4d28-20240404" phase: Succeeded
INFO[0089] OLM has successfully installed "cass-operator.v1.20.0-dev.faa4d28-20240404"
➜ cass-operator git:(master) ✗
Personally I always have local registry installed in my kind setups, since it gives more "e2e" feeling in that sense.
Issues go stale after 90d of inactivity.
Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.
If this issue is safe to close now please do so with /close.
/lifecycle stale
+1 I think this one would be useful. I've tried to setup the local flag, but there are other parts of the execution that are pulling the image later in order to get the bundle into a directory.
Stale issues rot after 30d of inactivity.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.
If this issue is safe to close now please do so with /close.
/lifecycle rotten /remove-lifecycle stale