Helm Chart 1.0.2: fluent-bit: unknown configuration property 'DB.Sync'

[hostalp]

Describe the bug

When 1.0.2 is installed from Helm chart, fluent-bit pods error out with:

unknown configuration property 'DB.Sync'. The following properties are allowed: path, max_fields, max_entries, systemd_filter_type, systemd_filter, read_from_tail, lowercase, strip_underscores, db_sync, and db

And they don't work properly then. I quickly tested 1.0.1 and it also appears to be affected. No such issue with 1.0.0

To Reproduce

Deploy 1.0.2 as follows:

helm upgrade --install fluent-operator --create-namespace -n logging https://github.com/fluent/fluent-operator/releases/latest/download/fluent-operator.tgz -f fluent-operator-values.yaml

with fluent-operator-values.yaml:

containerRuntime: docker
Kubernetes: true

operator:
  resources:
    limits:
      cpu: 100m
      memory: 100Mi
    requests:
      cpu: 10m
      memory: 20Mi

fluentbit:
  resources:
    limits:
      cpu: 500m
      memory: 200Mi
    requests:
      cpu: 10m
      memory: 25Mi
  input:
    tail:
      memBufLimit: 10MB

fluentd:
  enable: true
  replicas: 1
  forward:
    host: "logging.svc"
  watchedNamespaces:
    - kube-system
  resources:
    limits:
      cpu: 1000m
      memory: 256Mi
    requests:
      cpu: 20m
      memory: 64Mi
  output:
    es:
      enable: true
      host: logs.local.loc
      port: 9200
      logstashPrefix: k8s_logs_dev
      buffer:
        enable: true
        type: file
        path: /buffers/es

Expected behavior

fluent-bit pods starting without issues

Your Environment

- Fluent Operator version: Helm Chart 1.0.2
- Container Runtime: docker, Kubernetes 1.20.11

How did you install fluent operator?

helm upgrade --install fluent-operator --create-namespace -n logging https://github.com/fluent/fluent-operator/releases/latest/download/fluent-operator.tgz -f fluent-operator-values.yaml

What happened?

No response

Your Error Log

[error] [config] systemd: unknown configuration property 'DB.Sync'. The following properties are allowed: path, max_fields, max_entries, systemd_filter_type, systemd_filter, read_from_tail, lowercase, strip_underscores, db_sync, and db.

Additional context

Related to https://github.com/fluent/fluent-bit/issues/5426 Until the fix is released you should probably avoid using fluent-bit 1.9.x

[wenchajun]

This looks like a problem with fluentbit, it might be better to change the version of fluentbit, kubesphere/fluent-bit:v1.8.11. Do we have to change the version of fluentbit? @benjaminhuo

[benjaminhuo]

We should verify that v1.8.11 doesn't have this problem and v1.9.3 introduce this, looks like is not enough.

If so, you can change the FB version to v1.8.3 temporarily and update the yaml and helm accordingly.

But we should keep finding the root cause of the error, maybe by checking the release notes between v1.8.11 and v1.9.3 to see what changes should we make to fix this or talk to fluentbit maintainers.

FluentBit new version like v1.9.3+ should be supported by fluent operator, we shouldn't change back to v1.8.11 and then do nothing to find the root cause

[hostalp]

I've already reinstalled 1.0.2 with fluent-bit image changed to 1.8.11 and it does start just fine. As for the issue that's behind this, have a look at https://github.com/fluent/fluent-bit/issues/5426

[benjaminhuo]

@hostalp Thanks for the info. @wenchajun would you help to set the fluentbit version back to 1.8.11 for both the operator and the walkthrough repo including update the release page stuff? and we'll upgrade to a higher version than v1.9.3 when the fix is included?