kafdrop icon indicating copy to clipboard operation
kafdrop copied to clipboard

Multi-arch/arm64 support

Open rtammekivi opened this issue 1 year ago • 5 comments

This PR consists of several small changes:

  • Replaced com.spotify:docker-maven-plugin with io.fabric8:docker-maven-plugin Old library is deprecated and does not have buildx support
  • Modified GHA to build and push in the same job stage Needed for buildx built images
  • Added buildx and QEMU actions to GHA Needed for multi-arch Docker images

I did a brief test on my personal repo by changing GHA code a bit to publish to personal registry and to see if :latest tagging still works as expected. Also ran this on my x86_64 laptop and additionally a t4g.small (aarch64) VM. Code for the test: https://github.com/Blefish/kafdrop/tree/multi-arch-test And the test images: https://github.com/Blefish/kafdrop/pkgs/container/kafdrop

Closes #627

rtammekivi avatar Aug 01 '24 10:08 rtammekivi

@Blefish Thanks for this contribution! I need to take a close look at this, but I do not have the time now due to private circumstances. It might take a month or so before I get back on this.

Bert-R avatar Aug 03 '24 14:08 Bert-R

@Bert-R any update on this!

maipal-c avatar Sep 07 '24 06:09 maipal-c

@Blefish @Bert-R By the way when i teried kafdrop with image - ghcr.io/blefish/kafdrop:latest

ERROR LOGS -


2024-09-07 08:06:21.299 ERROR 1 [  XNIO-1 task-2] i.u.s.a.LoggingExceptionHandler          : UT005023: Exception handling request to /error

java.lang.RuntimeException: jakarta.servlet.ServletException: Request processing failed: freemarker.core.InvalidReferenceException: The following has evaluated to null or missing:
==> error.error  [in template "error.ftlh" at line 3, column 22]

----
Tip: It's the step after the last dot that caused this error, not those before it.
----
Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----

----
FTL stack trace ("~" means nesting-related):
	- Failed at: #macro header title  [in template "lib/template.ftlh" in macro "header" at line 17, column 1]
	- Reached through: @template.header "${error.error}"  [in template "error.ftlh" at line 3, column 1]
----
	at io.undertow.servlet.spec.HttpServletResponseImpl.doErrorDispatch(HttpServletResponseImpl.java:170)
	at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:283)
	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:132)
	at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
	at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
	at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:256)
	at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:101)
	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:393)
	at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:859)
	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
	at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282)
	at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: jakarta.servlet.ServletException: Request processing failed: freemarker.core.InvalidReferenceException: The following has evaluated to null or missing:
==> error.error  [in template "error.ftlh" at line 3, column 22]

PLATEFORM INFO -

OS- linux (arm64)
OS Image - Amazon Linux 2023.5.20240819
Kernel version- 6.1.102-111.182.amzn2023.aarch64
Container runtime- containerd://1.7.17-k3s1
Kubelet version- v1.29.7+k3s1
aws instance type - r8g.medium

DEPLOYMENT CONFIG -

apiVersion: apps/v1
kind: Deployment
metadata:
   name: kafdrop
   labels:
      app.kubernetes.io/name: kafdrop
spec:
   replicas: 1
   selector:
      matchLabels:
         app.kubernetes.io/name: kafdrop
   template:
      metadata:
         labels:
            app.kubernetes.io/name: kafdrop
      spec:
         serviceAccountName: kafdrop
         automountServiceAccountToken: true
         initContainers:
            - name: attach-init-secrets
              image: redhat/ubi8-minimal
              imagePullPolicy: IfNotPresent
              volumeMounts:
                 - name: init-secrets
                   mountPath: /init-secrets
              command:
                 - "/bin/bash"
                 - "-c"
                 - |
                    JWT=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token) ; 
                    vault_token=$(curl -s --request POST --data '{"jwt": "'$JWT'", "role": "core-kafka"}' http://vault-server-headless.security:8200/v1/auth/in-cluster/login | grep -oP '"client_token":"\K[^"]*') ;
                    secrets=$(curl -s --header "X-Vault-Token: $vault_token" http://vault-server-headless.security:8200/v1/in-cluster/data/core/kafka) ;
                    client_password=$(echo $secrets | grep -oP '"KAFKA_CLIENT_PASSWORDS":"\K[^"]*') ;
                    if [ -z "$client_password" ]; then
                       echo "Failed to get client password from vault. Please update core-kafka role in kubernetes auth method" ;
                       exit 1 ;
                    fi ;
                    cat <<EOF > /init-secrets/broker.properties
                    security.protocol=SASL_PLAINTEXT
                    sasl.mechanism=SCRAM-SHA-256
                    sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="user1" password="$client_password";
                    EOF
         containers:
            - name: kafdrop
              image: ghcr.io/blefish/kafdrop:latest
              imagePullPolicy: IfNotPresent
              ports:
                 - name: http
                   containerPort: 80
                   protocol: TCP
              resources:
                 limits:
                    cpu: 500m
                    memory: 512Gi
                 requests:
                    cpu: 10m
                    memory: 128Mi
              env:
                 - name: SERVER_PORT
                   value: "80"
                 - name: KAFKA_BROKERCONNECT
                   value: "kafka-controller-0.kafka-controller-headless.core.svc.cluster.local:9092"
                 - name: KAFKA_PROPERTIES_FILE
                   value: "/tmp/init-secrets/broker.properties"
                 - name: SERVER_SERVLET_CONTEXTPATH
                   value: "/"
                 - name: JVM_OPTS
                   value: "-Xms32M -Xmx64M"
                 - name: JMX_PORT
                   value: "8686"
                 - name: KAFKA_TRUSTSTORE
                   value: ""
                 - name: KAFKA_KEYSTORE
                   value: ""
                 - name: KAFKA_TRUSTSTORE_FILE
                   value: "kafka.truststore.jks"
                 - name: KAFKA_KEYSTORE_FILE
                   value: "kafka.keystore.jks"
              volumeMounts:
                 - name: init-secrets
                   mountPath: /tmp/init-secrets
                   readOnly: true
              livenessProbe:
                 httpGet:
                    path: /actuator/health
                    port: http
                    scheme: HTTP
                 timeoutSeconds: 5
                 periodSeconds: 10
                 failureThreshold: 5
              readinessProbe:
                 httpGet:
                    path: /actuator/health
                    port: http
                    scheme: HTTP
                 timeoutSeconds: 5
                 periodSeconds: 5
                 failureThreshold: 5
              startupProbe:
                 httpGet:
                    path: /actuator/health
                    port: http
                    scheme: HTTP
                 periodSeconds: 3
                 failureThreshold: 100
         volumes:
            - name: init-secrets
              emptyDir: {}

i am not sure if its code error or image error

maipal-c avatar Sep 07 '24 08:09 maipal-c

Yup I noticed that too, at random times.

Seemed to happen more often when using SSL security protocol with kafka. It looked to me like some sort of kafka communications error at first glance.

I am not sure, but I think it happened to me as well on the official release, but I need to confirm this.

rtammekivi avatar Sep 09 '24 09:09 rtammekivi

@Blefish when i remove the env variable SERVER_PORT it works fine on default port. your arm image too

maipal-c avatar Sep 09 '24 10:09 maipal-c

Rebased on latest master

rtammekivi avatar Nov 11 '24 14:11 rtammekivi

@Blefish Thanks again for this contribution! I have merged it and the build passed successfully. We now have a snapshot image at Docker hub, with amd64 and arm64 images. If someone can test these images and report back here, that would be great.

Bert-R avatar Nov 15 '24 11:11 Bert-R

@Bert-R i just upgraded to obsidiandynamics/kafdrop:4.0.3-SNAPSHOT (arm image). it worked without any issue in first Go.

  • plateform - kubernetes
  • OS - amazon Linux 2023
  • CPU - aws graviton3 (m7g)

However i felt something weird in logging (formatting) maybe i am wrong-

Screenshot 2024-11-17 at 12 59 24 AM

i'll keep it running. will report here if got any issues.

maipal-c avatar Nov 16 '24 19:11 maipal-c

@maipal-c If you are referring to the first weird characters, I suspect it is just the Kafdrop logo not rendered correctly, probably due to the font/spacing. https://patorjk.com/software/taag/#p=display&f=Alligator2&t=KAFDROP%204

davideicardi avatar Nov 17 '24 09:11 davideicardi

@davideicardi yea it is as issue of lens k8s IDE.

Kafdrop is all good. i am attaching the log file kafdrop-9d756d49d-5cb6f.log

maipal-c avatar Nov 17 '24 09:11 maipal-c

@maipal-c Thanks for the confirmation! @Blefish Compliments to you!

Bert-R avatar Nov 18 '24 11:11 Bert-R