Install on Kind with local registry lacks info about helm chart on a kind cluster with local repo

[matzew]

Requirement

In here https://github.com/apache/camel-k/blob/main/docs/modules/ROOT/pages/installation/platform/kind.adoc

it would be nice if shown how to helm install on kind, with a local registry.

Problem

Unclear how to

helm install \
  --generate-name \
  --set platform.build.registry.address=${REGISTRY_ADDRESS} \
  --set platform.build.registry.insecure=true \
  camel-k/camel-k

w/ kind (and local registry)

Proposal

add instructions for that

Open questions

No response

[squakez]

Running a local registry on Kind is a bit hacky (see https://camel.apache.org/camel-k/2.3.x/installation/registry/own.html#configuring-registry-run-docker). This is the reason why there is no "official" documentation around that.

[matzew]

The documentaton there seems to be for minikube (at least the text implies that), and in kube there is actually a pod.

For kind it works like this: https://kind.sigs.k8s.io/docs/user/local-registry/

and the doc there states, that you can use it as kind-registry:5000 inside the cluster.

However using that, I am getting:

camel-k-operator-59c546df55-zb57n camel-k-operator {"level":"info","ts":"2024-06-18T10:18:43Z","logger":"camel-k.controller.integration","msg":"Integration error","request-namespace":"default","request-name":"beer-source-pipe","api-version":"camel.apache.org/v1","kind":"Integration","ns":"default","name":"beer-source-pipe","reason":"Error","error-message":"Back-off pulling image \"kind-registry:5000/default/camel-k-kit-cpolsiek39ld6n7ftug0@sha256:00ca681c11c1a1a66d01c1cbd93a020d45b1397af37e5c7938be2b4f6e046e86\""}

NOTE: I am with podman

[christophd]

yes, you can reference the registry with kind-registry:5000.

I am also running on Podman and I had to tell Camel K that this is an insecure registry otherwise I had same image pull backoff errors. I am doing the install with kamel install though adding the --registry-insecure option. This has done the trick for me with Camel K running on Kind and Podman.

[christophd]

See https://camel.apache.org/camel-k/2.3.x/installation/platform/kind.html for more details and the kamel install command at the very end of the article

[matzew]

Thanks @christophd I'd like to actually not use the kamel CLI, just helm, like:

helm install \
  --generate-name \
  --set platform.build.registry.address=http://kind-registry:5000 \
  --set platform.build.registry.insecure=true \
  camel-k/camel-k

But using that, I am getting the above discussed Back-off pulling image error on the Operator when I am applying a Pipe manifest

[squakez]

--set platform.build.registry.address=http://kind-registry:5000 is maybe --set platform.build.registry.address=kind-registry:5000

[christophd]

@matzew sorry, I have overlooked that you already set insecure registry with --set platform.build.registry.insecure=true

for sure there is some piece missing with Helm installation on Kind with Podman because I am successfully using the same setup. The only difference I see is that I am installing with kamel CLI instead of Helm. Have you had success with this kamel CLI type of installation already? just to be sure that it is a missing piece of Helm based Camel K install.

I think the pull back-off is not limited to Pipes is it? Maybe you can verify some other integration. It should be failing, too.

[christophd]

BTW I have had similar pull image back-off errors earlier on Minikube with Podman because the container-runtime cri-o kept using https to access the registry regardless of setting insecure=true for the registry. I could not resolve this for Minikube so I have switched to Kind cluster where it is working as expected with Podman

[matzew]

For minikube/podman -> works fine for me :sweat_smile:

But for kind/local-registry/podman, I think it boils down to:

camel-k-operator-59c546df55-g4rxp camel-k-operator {"level":"info","ts":"2024-06-19T14:16:16Z","logger":"camel-k.maven.build","msg":"Containerizing application to kind-registry:5000/default/camel-k-kit-cppeft3qokjt6n5a1m5g:2044..."}
camel-k-operator-59c546df55-g4rxp camel-k-operator {"level":"info","ts":"2024-06-19T14:16:16Z","logger":"camel-k.maven.build","msg":"Base image 'eclipse-temurin:17' does not use a specific image digest - build may not be reproducible"}
camel-k-operator-59c546df55-g4rxp camel-k-operator {"level":"info","ts":"2024-06-19T14:16:16Z","logger":"camel-k.maven.build","msg":"Getting manifest for base image eclipse-temurin:17..."}
camel-k-operator-59c546df55-g4rxp camel-k-operator {"level":"info","ts":"2024-06-19T14:16:16Z","logger":"camel-k.maven.build","msg":"Building extra files layer..."}
camel-k-operator-59c546df55-g4rxp camel-k-operator {"level":"error","ts":"2024-06-19T14:16:17Z","logger":"camel-k.maven.build","msg":"I/O error for image [kind-registry:5000/default/camel-k-kit-cppeft3qokjt6n5a1m5g]:","stacktrace":"github.com/apache/camel-k/v2/pkg/util/log.Logger.Error\n\tgithub.com/apache/camel-k/v2/pkg/util/log/log.go:81\ngithub.com/apache/camel-k/v2/pkg/util/maven.normalizeLog\n\tgithub.com/apache/camel-k/v2/pkg/util/maven/maven_log.go:83\ngithub.com/apache/camel-k/v2/pkg/util/maven.MavenLogHandler\n\tgithub.com/apache/camel-k/v2/pkg/util/maven/maven_log.go:54\ngithub.com/apache/camel-k/v2/pkg/util.scan\n\tgithub.com/apache/camel-k/v2/pkg/util/command.go:76\ngithub.com/apache/camel-k/v2/pkg/util.RunAndLog.func1\n\tgithub.com/apache/camel-k/v2/pkg/util/command.go:55\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\tgolang.org/x/[email protected]/errgroup/errgroup.go:78"}
camel-k-operator-59c546df55-g4rxp camel-k-operator {"level":"error","ts":"2024-06-19T14:16:17Z","logger":"camel-k.maven.build","msg":"    java.net.UnknownHostException","stacktrace":"github.com/apache/camel-k/v2/pkg/util/log.Logger.Error\n\tgithub.com/apache/camel-k/v2/pkg/util/log/log.go:81\ngithub.com/apache/camel-k/v2/pkg/util/maven.normalizeLog\n\tgithub.com/apache/camel-k/v2/pkg/util/maven/maven_log.go:83\ngithub.com/apache/camel-k/v2/pkg/util/maven.MavenLogHandler\n\tgithub.com/apache/camel-k/v2/pkg/util/maven/maven_log.go:54\ngithub.com/apache/camel-k/v2/pkg/util.scan\n\tgithub.com/apache/camel-k/v2/pkg/util/command.go:76\ngithub.com/apache/camel-k/v2/pkg/util.RunAndLog.func1\n\tgithub.com/apache/camel-k/v2/pkg/util/command.go:55\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\tgolang.org/x/[email protected]/errgroup/errgroup.go:78"}
camel-k-operator-59c546df55-g4rxp camel-k-operator {"level":"error","ts":"2024-06-19T14:16:17Z","logger":"camel-k.maven.build","msg":"    kind-registry: Name or service not known","stacktrace":"github.com/apache/camel-k/v2/pkg/util/log.Logger.Error\n\tgithub.com/apache/camel-k/v2/pkg/util/log/log.go:81\ngithub.com/apache/camel-k/v2/pkg/util/maven.normalizeLog\n\tgithub.com/apache/camel-k/v2/pkg/util/maven/maven_log.go:83\ngithub.com/apache/camel-k/v2/pkg/util/maven.MavenLogHandler\n\tgithub.com/apache/camel-k/v2/pkg/util/maven/maven_log.go:54\ngithub.com/apache/camel-k/v2/pkg/util.scan\n\tgithub.com/apache/camel-k/v2/pkg/util/command.go:76\ngithub.com/apache/camel-k/v2/pkg/util.RunAndLog.func1\n\tgithub.com/apache/camel-k/v2/pkg/util/command.go:55\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\tgolang.org/x/[email protected]/errgroup/errgroup.go:78"}
camel-k-operator-59c546df55-g4rxp camel-k-operator {"level":"info","ts":"2024-06-19T14:16:17Z","logger":"camel-k.maven.build","msg":"The base image requires auth. Trying again for eclipse-temurin:17..."}
camel-k-operator-59c546df55-g4rxp camel-k-operator {"level":"info","ts":"2024-06-19T14:16:18Z","logger":"camel-k.maven.build","msg":"Using base image with digest: sha256:3a8a4b367562fe1dc5bb13c28dcdfd069e545c5f99747a82800fb96f99f1ba1f"}
camel-k-operator-59c546df55-g4rxp camel-k-operator {"level":"info","ts":"2024-06-19T14:16:19Z","logger":"camel-k.maven.build","msg":""}
camel-k-operator-59c546df55-g4rxp camel-k-operator {"level":"info","ts":"2024-06-19T14:16:19Z","logger":"camel-k.maven.build","msg":"A new version of jib-maven-plugin (3.4.3) is available (currently using 3.4.1). Update your build configuration to use the latest features and fixes!"}

Here is a more complete log from the operator: https://gist.githubusercontent.com/matzew/a9944eb61bf83bf8090160864c95104c/raw/6f0c9b46aa8c05ba374d233f3f40b90848b2494d/gistfile1.txt

Here is how I install it on Kind: https://github.com/matzew/kn-plugin-quickstart/commit/baeeaac249faf5b4c874b7e257833d88fafa353a

...
			if installCamel {
				if err := install.CamelK("kind-registry:5000"); err != nil {
					return fmt.Errorf("install Apache Camel K: %w", err)
				}
			}
...

And the install.CamelK is here: https://github.com/matzew/kn-plugin-quickstart/blob/baeeaac249faf5b4c874b7e257833d88fafa353a/pkg/install/install.go#L208-L234

[github-actions[bot]]

This issue has been automatically marked as stale due to 90 days of inactivity. It will be closed if no further activity occurs within 15 days. If you think that’s incorrect or the issue should never stale, please simply write any comment. Thanks for your contributions!