kube-arangodb Unable to deploy operator along with ArangoDeployment CRD

Hi everyone. Lately I've been trying to update my kube-arangodb from version 1.1.5 to 1.2.15 but I've enforced a problem. It seems that with that version, I'm unable to deploy arango-operator along with ArangoDeployment object. Everything works fine when I don't define ArangoDeployment.

POD logs are as follows:

I0722 14:35:33.727108       1 cmd.go:246] nice to meet you
2022-07-22T14:35:33Z INF Starting arangodb-operator (Community), version 1.2.15 build e9c43416 operator-id=9ljmm pod-name=arango-xxx-operator-74d4484bfc-9ljmm pod-namespace=xxx-9211-bump-arango-eks
2022-07-22T14:35:33Z INF Serving on 0.0.0.0:8528 operator-id=9ljmm
I0722 14:35:33.841831       1 leaderelection.go:243] attempting to acquire leader lease xxx-9211-bump-arango-eks/arango-deployment-operator...
I0722 14:35:33.947019       1 leaderelection.go:253] successfully acquired lease xxx-9211-bump-arango-eks/arango-deployment-operator
2022-07-22T14:35:33Z INF New leader detected identity=arango-xxx-operator-74d4484bfc-9ljmm-arango-xxx-operator-74d4484bfc-9ljmm lock-name=arango-deployment-operator namespace=xxx-9211-bump-arango-eks operator-id=9ljmm
2022-07-22T14:35:33Z INF Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"xxx-9211-bump-arango-eks", Name:"arango-deployment-operator", UID:"7f0524b8-af7e-4a1c-b2f2-00e5238e784c", APIVersion:"v1", ResourceVersion:"198934152", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' arango-xxx-operator-74d4484bfc-9ljmm-arango-xxx-operator-74d4484bfc-9ljmm became leader operator-id=9ljmm
2022-07-22T14:35:33Z INF Event(v1.ObjectReference{Kind:"Deployment", Namespace:"xxx-9211-bump-arango-eks", Name:"arango-xxx-operator", UID:"16df20b2-6b52-40ea-9961-ae14c29fbe25", APIVersion:"apps/v1", ResourceVersion:"198930520", FieldPath:""}): type: 'Normal' reason: 'Leader Election Won' Pod arango-xxx-operator-74d4484bfc-9ljmm is running as leader operator-id=9ljmm
2022-07-22T14:35:34Z DBG Waiting for arangodeployments.database.arangodb.com CRD to be ready -  namespace=xxx-9211-bump-arango-eks operator-id=9ljmm
2022-07-22T14:35:34Z DBG CRDs ready namespace=xxx-9211-bump-arango-eks operator-id=9ljmm
2022-07-22T14:35:34Z DBG ArangoDeployment added name=xxx-arangodb namespace=xxx-9211-bump-arango-eks operator-id=9ljmm
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x172cb00]

goroutine 110 [running]:
github.com/arangodb/kube-arangodb/pkg/deployment/resources/inspector.(*serviceMonitorsInspector).V1(0xc000e99a60)
	/home/ubuntu/.GIT/arango/kube-arangodb-release-manager/modules/community/pkg/deployment/resources/inspector/sm_v1.go:35
github.com/arangodb/kube-arangodb/pkg/deployment.(*Deployment).lookForServiceMonitorCRD(0xc000eba400)
	/home/ubuntu/.GIT/arango/kube-arangodb-release-manager/modules/community/pkg/deployment/deployment.go:598 +0x79
github.com/arangodb/kube-arangodb/pkg/deployment.(*Deployment).run(0xc000eba400)
	/home/ubuntu/.GIT/arango/kube-arangodb-release-manager/modules/community/pkg/deployment/deployment.go:352 +0x2f9
created by github.com/arangodb/kube-arangodb/pkg/deployment.New
	/home/ubuntu/.GIT/arango/kube-arangodb-release-manager/modules/community/pkg/deployment/deployment.go:268 +0xac5

Values which I'm using are:

operator:
  image: arangodb/kube-arangodb:1.2.15
  imagePullPolicy: IfNotPresent
  imagePullSecrets: []

  scope: namespaced
  
  architectures:
  - amd64

  debug: false

  args: []

  service:
    type: ClusterIP

  annotations: {}

  resources:
    limits:
      cpu: 1
      memory: 256Mi
    requests:
      cpu: 250m
      memory: 256Mi

  securityContext:
    runAsUser: 1001

  replicaCount: 1

  updateStrategy:
    type: Recreate

  allowChaos: false

  nodeSelector: {}
  
  enableCRDManagement: true

  features:
    deployment: true
    deploymentReplications: false
    storage: false
    backup: false
    apps: false
    k8sToK8sClusterSync: false

  images:
    base: alpine:3.11
    metricsExporter: arangodb/arangodb-exporter:0.1.7
    arango: arangodb/arangodb:latest

  tolerations: []

rbac:
  enabled: true
  extensions:
    monitoring: true
    acs: true
    at: true

and the definition of ArangoDeployment object is:

apiVersion: "database.arangodb.com/v1"
kind: "ArangoDeployment"
metadata:
  name: xxx-arangodb
spec:
  mode: Single
  environment: Production
  image: "arangodb/arangodb:latest"
  imagePullPolicy: "IfNotPresent"
  imagePullSecrets:
    
  externalAccess:
    type: None
  tls:
    caSecretName: None
  lifecycle:
    resources:
      limits:
        cpu: 100m
        memory: 32Mi
      requests:
        cpu: 100m
        memory: 24Mi
  bootstrap:
    passwordSecretNames:
      root: "xxx-arangodb-root"
  single:
    count: 1
    resources:
      limits:
        cpu: 4
        memory: 7Gi
      requests:
        cpu: 200m
        memory: 1Gi
        storage: 120Gi
    overrideDetectedTotalMemory: true
    args:
      - --server.maximal-threads=32
      - --server.minimal-threads=2
      - --log.level=info
      - --log.level=all=info
      - --log.level=requests=info
      - --log.level=startup=info
      - --log.level=performance=info
      - --log.time-format=utc-datestring-millis
      - --rocksdb.table-block-size=512000
      - --rocksdb.enforce-block-cache-size-limit=true
      - --rocksdb.enable-statistics=true

It looks like a problem with RBAC. Could anyone help?

Thanks in advance!

Jul 22 '22 14:07 billythekidpl

Hello!

It is an issue with namespaced mode - we will fix it in the next release (race inside client management creation).

The workaround is to use legacy mode.

Best Regards, Adam.

Jul 24 '22 17:07 ajanikow

Thanks a lot for your response! Could you link a GitHub issue or anything to track it?

Best, Maciej.

Jul 25 '22 11:07 billythekidpl