bug: kubesphere/fluent-bit:v1.9.9 fluent-bit-watcher not executable

[ITler]

trafficstars

Describe the issue

Running kubesphere/fluent-bit:v1.9.9 gives output:

exec /fluent-bit/bin/fluent-bit-watcher

To Reproduce

docker run --rm -it kubesphere/fluent-bit:v1.9.9

of cause same when deployed via fluent-operator master in Kubernetes (kind)

Expected behavior

it runs, showing proper output similar to v1.9.8 :smile:

Your Environment

- Fluent Operator version: current master 91e1e98e0c08dd298e9f2dcf081ccbd20e6c27ea, but irrelevant here
- Container Runtime: Docker version 20.10.18, build b40c2f6
- Operating system: Ubuntu 20 LTS
- Kernel version: 5.15.0-48-generic

How did you install fluent operator?

No response

Additional context

No response

[benjaminhuo]

@wenchajun would you help to check if any of the changes introduced in v1.5.1 is related to this? If the official fluentbit:v1.9.9 has the same problem or the fluentbit image we made has the problem?

[ITler]

I thought the /fluent-bit/bin/fluent-bit-watcher is the extra piece you are adding to the official fluentbit image, no?

[benjaminhuo]

I thought the /fluent-bit/bin/fluent-bit-watcher is the extra piece you are adding to the official fluentbit image, no?

Yes, the fluentbit in our custom image is started by /fluent-bit/bin/fluent-bit-watcher

[perezjasonr]

just wanted to report I also got this. I remember before I didnt get this, so does anyone know what version we might need to roll back to in order to get it working again?

edit: nvm you said "it runs, showing proper output similar to v1.9.8" so that answers that question, anyways for thoroughness heres my output:

kubectl get pods -n fluent
NAME                               READY   STATUS             RESTARTS      AGE
fluent-bit-bkzbj                   0/1     CrashLoopBackOff   4 (46s ago)   2m21s
fluent-bit-kf5q5                   0/1     CrashLoopBackOff   4 (53s ago)   2m21s
fluent-operator-749bfb89bd-kcc6m   1/1     Running            0             2m32s
username@hostname:~$ kubectl logs fluent-bit-bkzbj -n fluent
exec /fluent-bit/bin/fluent-bit-watcher: no such file or directory

[benjaminhuo]

@

just wanted to report I also got this. I remember before I didnt get this, so does anyone know what version we might need to roll back to in order to get it working again?

edit: nvm you said "it runs, showing proper output similar to v1.9.8" so that answers that question, anyways for thoroughness heres my output:

kubectl get pods -n fluent
NAME                               READY   STATUS             RESTARTS      AGE
fluent-bit-bkzbj                   0/1     CrashLoopBackOff   4 (46s ago)   2m21s
fluent-bit-kf5q5                   0/1     CrashLoopBackOff   4 (53s ago)   2m21s
fluent-operator-749bfb89bd-kcc6m   1/1     Running            0             2m32s
username@hostname:~$ kubectl logs fluent-bit-bkzbj -n fluent
exec /fluent-bit/bin/fluent-bit-watcher: no such file or directory

@wenchajun Would you take a look if you can reproduce this

[wenchajun]

After my testing, it seems that the underlying image is the cause here, and the exact cause is still being troubleshot. You can switch the image to kubesphere/fluent-bit:v1.9.4 for now.

[benjaminhuo]

I've revert #416 by https://github.com/fluent/fluent-operator/pull/439 which might cause this problem and @wenchajun will release v1.6.1 @ITler @perezjasonr