kubeblocks icon indicating copy to clipboard operation
kubeblocks copied to clipboard

[BUG] operator does not support sidecar injection

Open atobaum opened this issue 7 months ago • 6 comments

Describe the bug I'm using Istio in sidecar mode, configured to inject sidecar into the namespace where the MySQL and Redis pods reside. The problem is, Kubeblocks operator consistently removes the pod and recreates it again and again.

When I do not configure istio sidecar injection, everything works well.

I found the code where the kubeblocks deletes pod if the pod spec in InstanceSet and real pod spec are different. I suspect the kubeblocks operator deletes the pod because the injected pod spec is different from the spec defined in InstanceSet, and recreate it.

https://github.com/apecloud/kubeblocks/blob/5c004f5101a00886fb02ae34931a506bc172f32f/pkg/controller/instanceset/in_place_update_util.go#L261

https://github.com/apecloud/kubeblocks/blob/5c004f5101a00886fb02ae34931a506bc172f32f/pkg/controller/instanceset/in_place_update_util.go#L336-L342

I with if the operator provides some options ignoring sidecars when comparing pod specs to support sidecar injection.

To Reproduce

  1. Install Kubeblocks 0.9.4-beta15 on kubernetes 1.32
  2. Configure sidecar injection into MySql pods which provisioned by kubeblocks I installed istio and setup istio injection in kubeblocks namespace.
apiVersion: v1
kind: Namespace
metadata:
  labels:
    istio-injection: enabled
    name: kubeblocks-system
  name: kubeblocks-system
  1. Create cluster CR as follows:
apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
  name: apemysql-test
  namespace: kubeblocks-system
spec:
  affinity:
    podAntiAffinity: Preferred
    tenancy: SharedNode
  clusterDefinitionRef: apecloud-mysql
  clusterVersionRef: ac-mysql-8.0.30
  componentSpecs:
  - componentDefRef: mysql
    disableExporter: true
    enabledLogs:
    - auditlog
    - error
    - general
    - slow
    name: mysql
    replicas: 3
    resources:
      limits:
        cpu: "1"
        memory: 1Gi
      requests:
        cpu: "0.1"
        memory: 200Mi
    serviceAccountName: kb-apemysql-test
    volumeClaimTemplates:
    - name: data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 20Gi
  resources:
    cpu: "0"
    memory: "0"
  storage:
    size: "0"
  terminationPolicy: Delete
  1. One of the MySQL pod will be consistently deleted and recreated. Image

Expected behavior Kubebloks operator doesn't delete pods injected a sidecar.

Screenshots

Desktop (please complete the following information):

  • kubernetes 1.32
  • kubeblocks 0.9.4-beta15

Additional context Add any other context about the problem here.

atobaum avatar Apr 21 '25 01:04 atobaum

Hi @atobaum,

KubeBlocks renderes MySQL Cluster's PODs according to MySQL ClusterDefinition as specified in clusterDefinitionRef: apecloud-mysql. And KB does not allow sidecar injection so far.

It seems injecting sidecar is a must in your case. We will discuss whether to supoort the feature, and if yes, how.

shanshanying avatar Apr 21 '25 02:04 shanshanying

Injecting initContainers through webhook is not working in KubeBlocks too.

lancelot1989 avatar Apr 28 '25 06:04 lancelot1989

duplicate with https://github.com/apecloud/kubeblocks/issues/8736

wangyelei avatar May 20 '25 02:05 wangyelei

This issue has been marked as stale because it has been open for 30 days with no activity

github-actions[bot] avatar Jun 23 '25 00:06 github-actions[bot]

I bypassed this problem using Istio ambient mode.

atobaum avatar Jun 23 '25 00:06 atobaum

This issue has been marked as stale because it has been open for 30 days with no activity

github-actions[bot] avatar Aug 04 '25 00:08 github-actions[bot]