Problems setting up backups to s3

[philmccOri]

Please, answer some short questions which should help us to understand your problem / question better?

• **Which image of the operator are you using? registry.opensource.zalan.do/acid/postgres-operator:v1.8.1
• **Where do you run it - cloud or metal? Kubernetes or OpenShift? Bare Metal K8s
• **Are you running Postgres Operator in production? no
• **Type of issue? question

I am attempting to set up backups/wal archiving to an AWS s3 bucket but I cannot get it to work.

The current problem is that the wal-e env directlry does not seem to be getting set up:

envdir "/run/etc/wal-e.d/env" wal-g backup-list chpst: fatal: unable to switch to directory: /run/etc/wal-e.d/env: file does not exist

I have been adding config values one at a time starting from the ones listed in the docs here: Using AWS S3 or compliant services Just adding those ones does not seem to do anything.

I have used a configmap and also set values in the helm chart:

apiVersion: v1
kind: ConfigMap
metadata:
  name: pod-env-overrides
  namespace: postgres-operator
data:
  # Any env variable used by spilo can be added
  AWS_REGION: eu-west-1
  kube_iam_role: pg-backup-role
  wal_s3_bucket: pg-backup-bucket
  AWS_ACCESS_KEY_ID: [VALUE]
  AWS_SECRET_ACCESS_KEY: [VALUE]
  additional_secret_mount: aws-bucket-key
  additional_secret_mount_path: /var/secrets/aws
  BACKUP_NUM_TO_RETAIN: "30"
  BACKUP_SCHEDULE: 0 02 * * *
  USE_WALG_BACKUP: "false"
  USE_WALG_RESTORE: "false"
  WAL_BUCKET_SCOPE_PREFIX: ""
  AWS_S3_FORCE_PATH_STYLE: "true"
  WALG_DISABLE_S3_SSE: "true"
  # STANDBY
  STANDBY_additional_secret_mount: aws-bucket-key
  STANDBY_additional_secret_mount_path: /var/secrets/aws
  STANDBY_AWS_REGION: eu-west-1
  STANDBY_AWS_ACCESS_KEY_ID: [VALUE]
  STANDBY_AWS_SECRET_ACCESS_KEY: [VALUE]
  STANDBY_WAL_S3_BUCKET: pg-backup-bucket 

And the relevant section in the helm release yaml:

configAwsOrGcp:
      AWS_REGION: eu-west-1
      kube_iam_role: pg-backup-role
      wal_s3_bucket: pg-backup-bucket
      additional_secret_mount: aws-bucket-key
      additional_secret_mount_path: /var/secrets/aws
      STANDBY_AWS_REGION: eu-west-1
      STANDBY_AWS_ACCESS_KEY_ID: [VALUE]
      STANDBY_AWS_SECRET_ACCESS_KEY: [VALUE]

It seems to be populating the env variables on the pods but not setting up that env directory or trying to run wal-e/g

[FactorT]

Oh, I spent a lot of time to force the operator to make backup to S3. We have S3 from minio. First you need to set parametr "pod_environment_configmap" in postgresql-operator-default-configuration.yaml For example - pod_environment_configmap: "$namespace$/postgres-pod-config"

Next you need to create configmap with name from parameter above.

For example:
apiVersion: v1
kind: ConfigMap
metadata:
  name: postgres-pod-config
  namespace: $namespace$

And to fill this configmap your s3 parameters; For example:

  AWS_ENDPOINT: "http://server:posrt"
  AWS_SECRET_ACCESS_KEY: "secret_key"
  AWS_ACCESS_KEY_ID: "access_key"
  AWS_REGION: "region"
  AWS_S3_FORCE_PATH_STYLE: "true"
  BACKUP_SCHEDULE: "*/5 * * * *"
  BACKUP_NUM_TO_RETAIN: "10"
  USE_WALG_BACKUP: "true"
  USE_WALG_RESTORE: "true"
  WALG_DISABLE_S3_SSE: "true"
  WALE_DISABLE_S3_SSE: "true"
  WALG_S3_PREFIX: "s3://bucket/prefix"
  WAL_S3_BUCKET : "bucket"

It's works for me.