Devfile custon image ( I can't use custon image in minikube)

[ronaldor1968]

Summary

I'm having trouble using a custom Docker image in my Eclipse Che workspace with minikube. I've created a devfile specifying the custom image, and the image is successfully loaded into Minikube as confirmed by minikube image ls.

This is the devfile, still very simple.

schemaVersion: 2.2.0 metadata: name: minimal-test version: 2.0.2 displayName: Minimal test for eclipse-che description: Minimal test for eclipse-che tags: - Java - Maven projectType: maven language: Java components:

• name: glassfish3 container:
  image: docker.io/library/glassfish3:latest memoryLimit: 4096Mi

I created the image with the following command:

minikube image build -t glassfish3:latest .

Any ideas how to resolve this?

Relevant information

No response

[tolusha]

@ronaldor1968 Try this

kubectl patch -n eclipse-che checluster/eclipse-che  --type=json -p="[{\"op\": \"replace\", \"path\": \"/spec/devEnvironments/imagePullPolicy\", \"value\": \"IfNotPresent\"}]"

[ronaldor1968]

Hello,

Thanks for your help

it still not working

Error: ImagePullBackOff

I applied the patch and checked that everything was ok.

kubectl -n eclipse-che get checluster/eclipse-che -o yaml | grep imagePullPo
imagePullPolicy: IfNotPresent

But aparent the error still appears to be that the image is not accessible.

Thanks again for the help.

[tolusha]

@ronaldor1968

Could you please:

• show the screenshort with the error
• minikube image ls | docker.io/library/glassfish3:latest
• kubectl get devworkspaceoperatorconfigs.controller.devfile.io -n eclipse-che devworkspace-config -o yaml

[ronaldor1968]

[tolusha]

Hello. I tried to reproduce the problem. So, I was able successfully start a workspace after applying the patch. Dashboard firstly shows a warning, but then workspace is up and running.

[1] kubectl patch -n eclipse-che checluster/eclipse-che --type=json -p="[{"op": "replace", "path": "/spec/devEnvironments/imagePullPolicy", "value": "IfNotPresent"}]"

[tolusha]

@ronaldor1968 Please, feel free to reopen the issue if you have any questions.

[ronaldor1968]

Hello,

Sorry for the delay in responding, I was on vacation.

I still can't get it to work. Is there any way to increase the log in a verbose way to try to understand more what is going on?

Thanks again for the help.

[ronaldor1968]

[ronaldor1968]

[ronaldor1968]

Never ends ingress processing

[tolusha]

What is the minikube version?

You can grab logs from operators and ingress controller:

kubectl logs deploy/che-operator -n eclipse-che -f
kubectl logs deploy/devworkspace-controller-manager -n devworkspace-controller -f
kubectl logs deploy/ingress-nginx-controller -n ingress-nginx -f

[ronaldor1968]

minikube 1.33.1-1

in che-operator log i have this message compatible com ingress not start:

/che-operator/vendor/github.com/devfile/devworkspace-operator/controllers/controller/devworkspacerouting/devworkspacerouting_controller.go:225 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile /che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:122 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:323 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:274 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:235 2024-08-27T17:32:10Z INFO controllers.DevWorkspaceRouting Reconciling DevWorkspaceRouting {"Request.Namespace": "admin-che", "Request.Name": "routing-workspace6059402fd3504cf3", "devworkspace_id": "workspace6059402fd3504cf3"} 2024-08-27T17:32:10Z ERROR controllers.DevWorkspaceRouting Error syncing ingresses {"Request.Namespace": "admin-che", "Request.Name": "routing-workspace6059402fd3504cf3", "devworkspace_id": "workspace6059402fd3504cf3", "error": "admission webhook "validate.nginx.ingress.kubernetes.io" denied the request: host "admin-minimal-test-code-redirect-3.192.168.59.101.nip.io" and path "/" is already defined in ingress admin-che/workspace3fa0c9c58eb345e6-glassfish3-13133-code-redirect-3"} github.com/devfile/devworkspace-operator/controllers/controller/devworkspacerouting.(*DevWorkspaceRoutingReconciler).Reconcile /che-operator/vendor/github.com/devfile/devworkspace-operator/controllers/controller/devworkspacerouting/devworkspacerouting_controller.go:225 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile /che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:122 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:323 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:274 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /che-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:235

[ronaldor1968]

nginxcontroller.log operator.log

[ronaldor1968]

I don't see references to failure to load the image.

The error is no longer the same, apparently the image was loaded.

[tolusha]

2024-08-27T17:32:10Z ERROR controllers.DevWorkspaceRouting Error syncing ingresses {"Request.Namespace": "admin-che", "Request.Name": "routing-workspace6059402fd3504cf3", "devworkspace_id": "workspace6059402fd3504cf3", "error": "admission webhook "validate.nginx.ingress.kubernetes.io" denied the request: host "admin-minimal-test-code-redirect-3.192.168.59.101.nip.io" and path "/" is already defined in ingress admin-che/workspace3fa0c9c58eb345e6-glassfish3-13133-code-redirect-3"}

@AObuchow Could you have a look please at that error?

[AObuchow]

2024-08-27T17:32:10Z ERROR controllers.DevWorkspaceRouting Error syncing ingresses {"Request.Namespace": "admin-che", "Request.Name": "routing-workspace6059402fd3504cf3", "devworkspace_id": "workspace6059402fd3504cf3", "error": "admission webhook "validate.nginx.ingress.kubernetes.io" denied the request: host "admin-minimal-test-code-redirect-3.192.168.59.101.nip.io" and path "/" is already defined in ingress admin-che/workspace3fa0c9c58eb345e6-glassfish3-13133-code-redirect-3"}

@AObuchow Could you have a look please at that error?

I believe the error is showing that when trying to create the ingress for workspace6059402fd3504cf3 there is already an existing ingress for another workspace workspace3fa0c9c58eb345e6 with the same host & path.

Looking at the operator logs, it seems that the old workspace workspace3fa0c9c58eb345e6 encountered some error when reconciling its DevWorkspaceRouting:

{
  "controller": "devworkspacerouting",
  "controllerGroup": "controller.devfile.io",
  "controllerKind": "DevWorkspaceRouting",
  "DevWorkspaceRouting": {
    "name": "routing-workspace3fa0c9c58eb345e6",
    "namespace": "admin-che"
  },
  "namespace": "admin-che",
  "name": "routing-workspace3fa0c9c58eb345e6",
  "reconcileID": "d6baef23-c78c-4311-9982-e4b969618f0d",
  "error": "Operation cannot be fulfilled on devworkspaceroutings.controller.devfile.io \"routing-workspace3fa0c9c58eb345e6\": StorageError: invalid object, Code: 4, Key: /registry/controller.devfile.io/devworkspaceroutings/admin-che/routing-workspace3fa0c9c58eb345e6, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 0bd1da0a-8425-4106-8403-8abd2b6cb037, UID in object meta: "
}

After this error occurs, we stop reconciling the workspace and never clean it up/delete it.

@ronaldor1968 My suggestion is to delete all devworkspaces before trying to create you workspace again kubectl delete dw --all -n admin-che. Also, are you able to get the "empty workspace" sample to work from the Che Dashboard? In other words: is it only your devfile with a custom image that's failing to pass the preparing ingress step? Or are all workspaces encountering this issue.

It's worth noting that there might be something else going on that's causing the Precondition failed: UID in precondition: ..., UID in object meta: error message. I found this issue https://github.com/kubernetes/kubernetes/issues/82130 though I'm not sure how it relates to this case currently.

[ronaldor1968]

Hello,

I had already removed the workspace before. I did it again using the proposed command and the same thing happened.

"kubectl delete dw --all -n admin-che
devworkspace.workspace.devfile.io "minimal-test" deleted"

I think I'm going to destroy the minikube instance and build one again from scratch, as I've already done a lot of tests on this instance trying to solve the local image problem, maybe something is corrupt. What do you think?

[ronaldor1968]

hello,

I destroyed the minikube and did the entire installation, applied the patch and when I did the test it returned the image loading error.

I found this case, which I'm not sure has been resolved since it was finished by the robot.

https://github.com/kubernetes/minikube/issues/16036

[ronaldor1968]

The error I have in the console is: Error creating DevWorkspace deployment: Container glassfish3 has state CrashLoopBackOff, so I'm going to reinstall everything again and double the minikube memory and cpu.

[ronaldor1968]

I'm going to consider this problem as closed, the error must now be due to the image that works perfectly in podman and must not be working well in minikube. The image is loading correctly, which was the initial reason for the bug. After so many attempts, minikube became unstable, now it can clearly be seen that the initial patch resolved the image loading problem. I will now investigate the image problem. Thanks everyone.