MountVolume.NewMounter initialization failed for volume

[bthj]

A couple of weeks ago, one database server node in a default configuration using the Kubernetes Operator (3 of each) suddenly failed by getting stuck in the Pending state, with the event log message:

MountVolume.NewMounter initialization failed for volume "arangodb-local-storage-3d86b0-sm3rxsop3kpft8p2" : path "/var/lib/arango-storage/sm3rxsop3kpft8p2" does not exist FailedMount ...

I was unable to see what was causing this issue, but to have fewer moving parts, I re-deployed the database in a single node configuration while explicitly requesting the amount of storage as 80Gi and memory as 10Gi:

apiVersion: "database.arangodb.com/v1alpha"
kind: "ArangoDeployment"
metadata:
  name: dbgraph
spec:
  mode: Single
  single:
    resources:
      requests:
        storage: 80Gi
        memory: 10Gi
  environment: Production
  image: "arangodb/arangodb:3.4.6"
  externalAccess:
    type: "None"
  tls:
    caSecretName: "None"

This morning I saw that the database was down - the pod stuck in an initializing state with the event message:

MountVolume.NewMounter initialization failed for volume "arangodb-local-storage-..." : path "/var/lib/arango-storage/*storage-id*" does not exist

• but in fact I saw that storage with storage-id did exist in the GKE dashboard.

As I was using ArangoLocalStorage I've now recreated the Operator and database deployments without ArangoLocalStorage, in hope that this was an issue with that functionality - so now the database deployment is using the standard storage class.

Previously I was using version 0.3.11 of the Operator but am now using the latest version 0.4.1.

This is running on GKE in a cluster with a pool of n1-highmem-2 nodes.

Hopefully the removal of ArangoLocalStorage and upgrade to 0.4.1 resolves my issues but it would be great to hear if you see some other indicators from the above error messages?

Best regards /Björn