harbor-db not starting (initdb: error: could not create directory / Permission denied)

[bbqrob]

Using helm chart v1.7.3 (using application v2.3.3) as seen here, installed the chart into namespace harbor.

ls: cannot access '/var/lib/postgresql/data/pgdata': No such file or directory
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locales
  COLLATE:  en_US.UTF-8
  CTYPE:    en_US.UTF-8
  MESSAGES: C
  MONETARY: C
  NUMERIC:  C
  TIME:     C
The default text search configuration will be set to "english".

Data page checksums are disabled.

creating directory /var/lib/postgresql/data/pgdata/pg13 ... 2021-10-30T19:16:21.245845654+02:00 initdb: error: could not create directory "/var/lib/postgresql/data/pgdata": Permission denied

My values.yaml got the following snippet for database claim:

    database:
      existingClaim: "database-data-harbor-database-0"
      storageClass: ""
      subPath: ""
      accessMode: ReadWriteOnce
      size: 2Gi

PersistentVolume is configured as:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: database-data-harbor-database-0
  namespace: harbor
  labels:
    app: harbor-pg
    type: local
spec:
    capacity:
        storage: 2Gi
    accessModes:
        - ReadWriteOnce
    hostPath:
        path: "/var/kubelet/harbor/pg"

with claim:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: database-data-harbor-database-0
  namespace: harbor
  labels:
    app: harbor-pg
spec:
  storageClassName: ""
  capacity:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
  selector:
    matchLabels:
      app: harbor-pg

[bbqrob]

Adding

subPath: "data" 

simply created the directory in the filesystem, but didn't change anything to the permission issue.

[zyyw]

currently, we use fsGroup to change the Permission, does your storage provider/driver support fsGroup? alternately, you can use initContainer to update the file permission with root.

[timbrown5]

I managed to work around this with an init container, I used the following YAML

./harbor/kustomization.yaml:

helmCharts:
- name: harbor
  version: 1.8.1
  repo: https://helm.goharbor.io 
  releaseName: deployment-harbor
  namespace: harbor
  valuesFile: values.yaml

patchesStrategicMerge:
- postgres-patch.yaml

./harbor/postgres-patch.yaml:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: deployment-harbor-database
spec:
  replicas: 1
  selector:
    matchLabels:
      app: harbor
      component: database
      release: deployment-harbor
  serviceName: deployment-harbor-database
  template:
    spec:
      initContainers:
      - args:
        - -c
        - '[ -e /var/lib/postgresql/data/postgresql.conf ] && [ ! -d /var/lib/postgresql/data/pgdata
          ] && mkdir -m 0700 /var/lib/postgresql/data/pgdata && mv /var/lib/postgresql/data/*
          /var/lib/postgresql/data/pgdata/ || true'
        command:
        - /bin/sh
        image: goharbor/harbor-db:v2.4.1
        imagePullPolicy: IfNotPresent
        name: data-migrator
        volumeMounts:
        - mountPath: /var/lib/postgresql/data
          name: database-data
          subPath: null
      - args:
        - -c
        - chmod -R 700 /var/lib/postgresql/data/pgdata || true
        command:
        - /bin/sh
        image: goharbor/harbor-db:v2.4.1
        imagePullPolicy: IfNotPresent
        name: data-permissions-ensurer
        volumeMounts:
        - mountPath: /var/lib/postgresql/data
          name: database-data
          subPath: null
      # This is the only new bit (the other two init containers exist in the helm chart).
      - args:
        - -c
        - chown -R 999:999 /var/lib/postgresql/data || true
        command:
        - /bin/sh
        image: goharbor/harbor-db:v2.4.1
        name: data-ownership-ensurer
        volumeMounts:
        - mountPath: /var/lib/postgresql/data
          name: database-data
          subPath: null
      securityContext:
        fsGroup: 999
        runAsUser: 999

kustomize build --enable-helm --load-restrictor LoadRestrictionsNone ./harbor/ > harbor.yaml (customize version: v4.1.3)

I can see that this might only be an issue when using NFS, but If harbor is chmod-ing /var/lib/postgresql/data with 700, maybe it should chown it as well?

[timbrown5]

I created a PR but not sure how it will be received 😅

https://github.com/goharbor/harbor-helm/pull/1170

[QiuToo]

请将accessModes:的ReadWriteOnce修改为ReadWriteMany

[supernacho]

It works for me, updataing iniContainer data-migrator and data-permissions-ensurer to boot as root and owning postgres user in /var/lib/postgresql/data/ folder.

      initContainers:
        - name: data-migrator
          image: goharbor/harbor-db:v2.5.0
          command:
            - /bin/sh
          args:
            - '-c'
            - >-
              [ -e /var/lib/postgresql/data/postgresql.conf ] && [ ! -d
              /var/lib/postgresql/data/pgdata ] && chown -R postgres
              /var/lib/postgresql/data/ && mkdir -m 0700
              /var/lib/postgresql/data/pgdata && mv /var/lib/postgresql/data/*
              /var/lib/postgresql/data/pgdata/ || true
          resources: {}
          volumeMounts:
            - name: database-data
              mountPath: /var/lib/postgresql/data
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
          securityContext:
            runAsUser: 0
     - name: data-permissions-ensurer
          image: goharbor/harbor-db:v2.5.0
          command:
            - /bin/sh
          args:
            - '-c'
            - >-
              chown -R postgres /var/lib/postgresql/data/ && chmod -R 700
              /var/lib/postgresql/data/pgdata || true
         securityContext:
             runAsUser: 0

[PeterJRVV]

I am also using a NFS storage provider and run into the same issue. how can I apply the init container scripts withinn the values.yaml file. Should it go within this section

initContainer: migrator: {} # resources: # requests: # memory: 128Mi # cpu: 100m permissions: {} # resources: # requests: # memory: 128Mi # cpu: 100m

[marcusnh]

i'm experiencing this issue where I have permission denied for:

• harbor-redis
  

• harbor-trivy
  

• harbor-jobservice
  

• harbor-registry
  

is there a way to workaround this? Have tried: volumePermissions: enabled: true

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 30 days with no activity. If this issue is still relevant, please re-open a new issue.

[ksummersill2]

The helm package for goharbor/harbor-db:v2.10.0 is still having this issue. Much like @supernacho said i was able to resolve this by setting up the SecurityContext user, as the default within the container the user does not have permissions.

Example of the init containers

initContainers:
      # as we change the data directory to a sub folder to support psp, the init container here
      # is used to migrate the existing data. See https://github.com/goharbor/harbor-helm/issues/756
      # for more detail.
      # we may remove it after several releases
      - name: "data-migrator"
        image: goharbor/harbor-db:v2.10.0
        securityContext:
          runAsUser: 0  # Run as root user
        imagePullPolicy: IfNotPresent
        command: ["/bin/sh"]
        args: ["-c", "[ ! -d /var/lib/postgresql/data/pgdata ] && mkdir -p /var/lib/postgresql/data/pgdata && chmod 700 -R /var/lib/postgresql/data/pgdata && [ -e /var/lib/postgresql/data/postgresql.conf ] && mv /var/lib/postgresql/data/* /var/lib/postgresql/data/pgdata/ || chmod 0700 /var/lib/postgresql/data/pgdata && ls -l /var/lib/postgresql/data || true"]
        volumeMounts:
          - name: database-data
            mountPath: /var/lib/postgresql/data
            subPath: 
      # with "fsGroup" set, each time a volume is mounted, Kubernetes must recursively chown() and chmod() all the files and directories inside the volume
      # this causes the postgresql reports the "data directory /var/lib/postgresql/data/pgdata has group or world access" issue when using some CSIs e.g. Ceph
      # use this init container to correct the permission
      # as "fsGroup" applied before the init container running, the container has enough permission to execute the command
      - name: "data-permissions-ensurer"
        image: goharbor/harbor-db:v2.10.0
        securityContext:
          runAsUser: 0  # Run as root user
        imagePullPolicy: IfNotPresent
        command: ["/bin/sh"]
        args: ["-c", "chown -R 999:999 /var/lib/postgresql/data/pgdata && chmod -R 700 /var/lib/postgresql/data/pgdata || true"]
        volumeMounts:
          - name: database-data
            mountPath: /var/lib/postgresql/data
            subPath: 

Change the init containers to utilise runAsUser: 0 and also update the data-permissions-ensurer to include 999 to the pgdata path to give the database user the permission it needs.

[ksummersill2]

I also wanted to comment that there should be a couple of more init containers to fix permission issues. One for the Redis and one for the Trivy Containers. Once i did this i was able to have all the pods load up without any permission issues.

Trivy Init Container to allow permissions to 10000 user:

initContainers:
      - name: trivy-permission-fix
        securityContext:
          runAsUser: 0  # Run as root user
        image: busybox
        command: ['sh', '-c', 'chown -R 10000:10000 /home/scanner']
        volumeMounts:
        - name: data
          mountPath: /home/scanner

InitContainer for Redis

initContainers:
      - name: redis-permission-fix
        securityContext:
          runAsUser: 0  # Run as root user
        image: busybox
        command: ['sh', '-c', 'chown -R 999:999 /var/lib/redis']
        volumeMounts:
        - name: data
          mountPath: /var/lib/redis