java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data

[Slyke]

Fresh install of ECK and Elasticsearch. Bare-metal K8s. NFS mounted volume for PV on nodes.

I occasionally get pods that do this, which is often fixed with an initContainer. Example with Grafana:

      initContainers:
      - name: setup-perms
        image: busybox:1.35.0
        command: ["/bin/sh", "-c", "chown -R 472:472 /var/lib/grafana && chmod -R 770 /var/lib/grafana"]
        volumeMounts:
        - mountPath: /certs
          name: grafana-certs-claim
        - mountPath: /var/lib/grafana
          name: grafana-data-claim

But not sure if an initContainer can be used here.

Using version 2.16.0 for CRDs and Operator.

$ kubectl get -n elastic-system pods
NAME                 READY   STATUS             RESTARTS      AGE
elastic-operator-0   1/1     Running            0             3h52m
es-es-default-0      0/1     CrashLoopBackOff   7 (64s ago)   12m

Deployment:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: es
  namespace: elastic-system
spec:
  version: 8.17.0
  nodeSets:
  - name: default
    count: 1
    config:
      node.store.allow_mmap: false
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
        namespace: elastic-system
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
        storageClassName: ""

PV:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-elasticsearch
  namespace: elastic-system
spec:
  storageClassName: ""
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  mountOptions:
    - hard
    - nfsvers=3
  nfs:
    server: 192.168.14.4
    path: "kubernetes/elastic-system/elasticsearch/data"
    readOnly: false

Error:

java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
        at java.base/sun.nio.fs.UnixException.asIOException(UnixException.java:115)
        at java.base/sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:502)
        at java.base/java.nio.file.Files.newDirectoryStream(Files.java:482)
        at java.base/java.nio.file.Files.list(Files.java:3796)
        at org.elasticsearch.xpack.security.cli.AutoConfigureNode.isDirEmpty(AutoConfigureNode.java:1153)
        at org.elasticsearch.xpack.security.cli.AutoConfigureNode.execute(AutoConfigureNode.java:167)
        at org.elasticsearch.server.cli.ServerCli.autoConfigureSecurity(ServerCli.java:185)
        at org.elasticsearch.server.cli.ServerCli.execute(ServerCli.java:91)
        at org.elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:55)
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:95)
        at org.elasticsearch.cli.Command.main(Command.java:52)
        at org.elasticsearch.launcher.CliToolLauncher.main(CliToolLauncher.java:65