postgres-operator icon indicating copy to clipboard operation
postgres-operator copied to clipboard

if you use etcd_host, one key is used for different namespaces

Open alexey-gavrilov-flant opened this issue 3 years ago • 2 comments

Please, answer some short questions which should help us to understand your problem / question better?

  • Which image of the operator are you using? e.g. registry.opensource.zalan.do/acid/postgres-operator:v1.8.2
  • Where do you run it - cloud or metal? Kubernetes or OpenShift? Bare Metal K8s
  • Are you running Postgres Operator in production? yes
  • Type of issue? Bug report

Some general remarks when posting a bug report:

  • Please, check the operator, pod (Patroni) and postgresql logs first. When copy-pasting many log lines please do it in a separate GitHub gist together with your Postgres CRD and configuration manifest.
  • If you feel this issue might be more related to the Spilo docker image or Patroni, consider opening issues in the respective repos.

there are two problems:

  1. the problem manifests itself with the same name and different namespace if use etcd_host
  2. deleting cr does not delete the entry in etcd about the psql cluster, which makes it impossible to create the same cluster over a new one.

operator env

  configGeneral:
    etcd_host: etcd.{{ .Release.Namespace }}.svc.{{ .Values.configKubernetes.cluster_domain }}

test psql-test-01 or psql-test-02 don't start for psql-test-03 I found a hack solution through an additional env

  env:
    - name: NAMESPACE
      value: /psql-test-03 

I think the first problem is related to this code

test.yaml

---
apiVersion: v1
kind: Namespace
metadata:
  name: psql-test-01
spec:
  finalizers:
  - kubernetes
---
apiVersion: v1
kind: Namespace
metadata:
  name: psql-test-02
spec:
  finalizers:
  - kubernetes
---
apiVersion: v1
kind: Namespace
metadata:
  name: psql-test-03
spec:
  finalizers:
  - kubernetes
---
apiVersion: acid.zalan.do/v1
kind: postgresql
metadata:
  name: test-psql
  namespace: psql-test-01
spec:
  allowedSourceRanges:
  - 0.0.0.0/0
  databases:
    catalogu: zalando
    init: zalando
  enableShmVolume: true
  numberOfInstances: 1
  patroni:
    pg_hba:
    - host      all         okagent all md5
    - local     all         all trust
    - hostssl   all         +zalandos 127.0.0.1/32 pam
    - host      all         all 127.0.0.1/32 md5
    - hostssl   all         +zalandos ::1/128 pam
    - host      all         all ::1/128 md5
    - local     replication standby trust
    - hostssl   replication standby all md5
    - hostnossl all         all all reject
    - hostssl   all         +zalandos all pam
    - hostssl   all         all all md5
  podAnnotations:
    threshold.extended-monitoring.flant.com/disk-bytes-critical: "70"
    threshold.extended-monitoring.flant.com/disk-bytes-warning: "50"
  postgresql:
    parameters:
      hot_standby_feedback: "on"
      max_standby_archive_delay: 300s
      max_standby_streaming_delay: 300s
      pg_stat_statements.max: "500"
      pg_stat_statements.save: "false"
      pg_stat_statements.track: top
      pg_stat_statements.track_utility: "true"
      track_io_timing: "on"
    version: "13"
  resources:
    limits:
      cpu: 2000m
      memory: 256Mi
    requests:
      cpu: 50m
      memory: 256Mi
  teamId: test
  users:
    foo_user: []
    zalando:
    - superuser
    - createdb
  volume:
    size: 10Gi
---
apiVersion: acid.zalan.do/v1
kind: postgresql
metadata:
  name: test-psql
  namespace: psql-test-02
spec:
  allowedSourceRanges:
  - 0.0.0.0/0
  databases:
    catalogu: zalando
    init: zalando
  enableShmVolume: true
  numberOfInstances: 1
  patroni:
    pg_hba:
    - host      all         okagent all md5
    - local     all         all trust
    - hostssl   all         +zalandos 127.0.0.1/32 pam
    - host      all         all 127.0.0.1/32 md5
    - hostssl   all         +zalandos ::1/128 pam
    - host      all         all ::1/128 md5
    - local     replication standby trust
    - hostssl   replication standby all md5
    - hostnossl all         all all reject
    - hostssl   all         +zalandos all pam
    - hostssl   all         all all md5
  podAnnotations:
    threshold.extended-monitoring.flant.com/disk-bytes-critical: "70"
    threshold.extended-monitoring.flant.com/disk-bytes-warning: "50"
  postgresql:
    parameters:
      hot_standby_feedback: "on"
      max_standby_archive_delay: 300s
      max_standby_streaming_delay: 300s
      pg_stat_statements.max: "500"
      pg_stat_statements.save: "false"
      pg_stat_statements.track: top
      pg_stat_statements.track_utility: "true"
      track_io_timing: "on"
    version: "13"
  resources:
    limits:
      cpu: 2000m
      memory: 256Mi
    requests:
      cpu: 50m
      memory: 256Mi
  teamId: test
  users:
    foo_user: []
    zalando:
    - superuser
    - createdb
  volume:
    size: 10Gi
---
apiVersion: acid.zalan.do/v1
kind: postgresql
metadata:
  name: test-psql
  namespace: psql-test-03
spec:
  allowedSourceRanges:
  - 0.0.0.0/0
  databases:
    catalogu: zalando
    init: zalando
  enableShmVolume: true
  numberOfInstances: 1
  env:
    - name: NAMESPACE
      value: /psql-test-03 
  patroni:
    pg_hba:
    - host      all         okagent all md5
    - local     all         all trust
    - hostssl   all         +zalandos 127.0.0.1/32 pam
    - host      all         all 127.0.0.1/32 md5
    - hostssl   all         +zalandos ::1/128 pam
    - host      all         all ::1/128 md5
    - local     replication standby trust
    - hostssl   replication standby all md5
    - hostnossl all         all all reject
    - hostssl   all         +zalandos all pam
    - hostssl   all         all all md5
  podAnnotations:
    threshold.extended-monitoring.flant.com/disk-bytes-critical: "70"
    threshold.extended-monitoring.flant.com/disk-bytes-warning: "50"
  postgresql:
    parameters:
      hot_standby_feedback: "on"
      max_standby_archive_delay: 300s
      max_standby_streaming_delay: 300s
      pg_stat_statements.max: "500"
      pg_stat_statements.save: "false"
      pg_stat_statements.track: top
      pg_stat_statements.track_utility: "true"
      track_io_timing: "on"
    version: "13"
  resources:
    limits:
      cpu: 2000m
      memory: 256Mi
    requests:
      cpu: 50m
      memory: 256Mi
  teamId: test
  users:
    foo_user: []
    zalando:
    - superuser
    - createdb
  volume:
    size: 10Gi
root@test-psql-0:/home/postgres# ETCDCTL_API=2 etcdctl --endpoints http://etcd.postgres-operator-bravo-shared.svc.cluster.local:2379 ls --recursive
/service
/service/test-psql
/service/test-psql/members
/service/test-psql/members/test-psql-0
/service/test-psql/initialize
/service/test-psql/config
/service/test-psql/leader
/service/test-psql/status
/psql-test-03
/psql-test-03/test-psql
/psql-test-03/test-psql/members
/psql-test-03/test-psql/members/test-psql-0
/psql-test-03/test-psql/initialize
/psql-test-03/test-psql/config
/psql-test-03/test-psql/leader
/psql-test-03/test-psql/status

alexey-gavrilov-flant avatar Aug 30 '22 15:08 alexey-gavrilov-flant

https://github.com/zalando/patroni/blob/master/docs/SETTINGS.rst#globaluniversal

alexey-gavrilov-flant avatar Aug 30 '22 15:08 alexey-gavrilov-flant