Data directory "/var/lib/postgresql/data" has wrong ownership in Kubernetes

Open Telokis opened this issue 6 years ago • 0 comments

I just tried installing the helm chart for PostDock and I can't access Postgres. Inside the logs of my node-0, I have the following error log:

Logs of the pod mysystem-db-node-0

>>> Setting up STOP handlers...
>>> STARTING SSH (if required)...
>>> SSH is not enabled!
>>> STARTING POSTGRES...
>>> SETTING UP POLYMORPHIC VARIABLES (repmgr=3+postgres=9 | repmgr=4, postgres=10)... 
>>> TUNING UP POSTGRES...
>>> Cleaning data folder which might have some garbage...
>>> Check all partner nodes for common upstream node...
>>>>>> Checking NODE=mysystem-db-node-0.mysystem-db...
psql: could not translate host name "mysystem-db-node-0.mysystem-db" to address: Name or service not known
>>>>>> Skipping: failed to get master from the node!
>>>>>> Checking NODE=mysystem-db-node-1.mysystem-db...
psql: could not connect to server: Connection refused
	Is the server running on host "mysystem-db-node-1.mysystem-db" (10.1.1.50) and accepting
	TCP/IP connections on port 5432?
>>>>>> Skipping: failed to get master from the node!
>>>>>> Checking NODE=mysystem-db-node-2.mysystem-db...
psql: could not connect to server: Connection refused
	Is the server running on host "mysystem-db-node-2.mysystem-db" (10.1.1.51) and accepting
	TCP/IP connections on port 5432?
>>>>>> Skipping: failed to get master from the node!
>>> Auto-detected master name: ''
>>> Setting up repmgr...
>>> Setting up repmgr config file '/etc/repmgr.conf'...
>>> Setting up upstream node...
>>> Sending in background postgres start...
>>> Waiting for local postgres server recovery if any in progress:LAUNCH_RECOVERY_CHECK_INTERVAL=30
>>> Recovery is in progress:
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 20
selecting default shared_buffers ... 400kB
selecting default timezone ... Etc/UTC
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
2019-11-13 09:54:41.251 UTC [143] FATAL:  data directory "/var/lib/postgresql/data" has wrong ownership
2019-11-13 09:54:41.251 UTC [143] HINT:  The server must be started by the user that owns the data directory.
child process exited with exit code 1
initdb: removing contents of data directory "/var/lib/postgresql/data"
running bootstrap script ... >>>>>> RECOVERY_WAL_ID is empty!
>>> Not in recovery state (anymore)
>>> Waiting for local postgres server start...
>>> Wait schema replication_db.public on mysystem-db-node-0.mysystem-db:5432(user: replica_user,password: *******), will try 9 times with delay 10 seconds (TIMEOUT=90)
psql: could not connect to server: Connection refused
	Is the server running on host "mysystem-db-node-0.mysystem-db" (10.1.1.49) and accepting
	TCP/IP connections on port 5432?
>>>>>> Host mysystem-db-node-0.mysystem-db:5432 is not accessible (will try 9 times more)
psql: could not connect to server: Connection refused
	Is the server running on host "mysystem-db-node-0.mysystem-db" (10.1.1.49) and accepting
	TCP/IP connections on port 5432?
>>>>>> Host mysystem-db-node-0.mysystem-db:5432 is not accessible (will try 8 times more)
psql: could not connect to server: Connection refused
	Is the server running on host "mysystem-db-node-0.mysystem-db" (10.1.1.49) and accepting
	TCP/IP connections on port 5432?
>>>>>> Host mysystem-db-node-0.mysystem-db:5432 is not accessible (will try 7 times more)
psql: could not connect to server: Connection refused
	Is the server running on host "mysystem-db-node-0.mysystem-db" (10.1.1.49) and accepting
	TCP/IP connections on port 5432?
>>>>>> Host mysystem-db-node-0.mysystem-db:5432 is not accessible (will try 6 times more)
psql: could not connect to server: Connection refused
	Is the server running on host "mysystem-db-node-0.mysystem-db" (10.1.1.49) and accepting
	TCP/IP connections on port 5432?
>>>>>> Host mysystem-db-node-0.mysystem-db:5432 is not accessible (will try 5 times more)
psql: could not connect to server: Connection refused
	Is the server running on host "mysystem-db-node-0.mysystem-db" (10.1.1.49) and accepting
	TCP/IP connections on port 5432?
>>>>>> Host mysystem-db-node-0.mysystem-db:5432 is not accessible (will try 4 times more)
psql: could not connect to server: Connection refused
	Is the server running on host "mysystem-db-node-0.mysystem-db" (10.1.1.49) and accepting
	TCP/IP connections on port 5432?
>>>>>> Host mysystem-db-node-0.mysystem-db:5432 is not accessible (will try 3 times more)
psql: could not connect to server: Connection refused
	Is the server running on host "mysystem-db-node-0.mysystem-db" (10.1.1.49) and accepting
	TCP/IP connections on port 5432?
>>>>>> Host mysystem-db-node-0.mysystem-db:5432 is not accessible (will try 2 times more)
psql: could not connect to server: Connection refused
	Is the server running on host "mysystem-db-node-0.mysystem-db" (10.1.1.49) and accepting
	TCP/IP connections on port 5432?
>>>>>> Host mysystem-db-node-0.mysystem-db:5432 is not accessible (will try 1 times more)
>>> Schema replication_db.public is not accessible, even after 9 tries!

A similar log is present for node-1 and node-2 as well.

I am running Kubernetes on my Windows 10 machine using Docker Desktop.

$ kubectl version
Client Version: version.Info {
  Major:"1",
  Minor:"16",
  GitVersion:"v1.16.0",
  GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77",
  GitTreeState:"clean",
  BuildDate:"2019-09-18T14:36:53Z",
  GoVersion:"go1.12.9",
  Compiler:"gc",
  Platform:"windows/amd64"
}
Server Version: version.Info {
  Major:"1",
  Minor:"14",
  GitVersion:"v1.14.7",
  GitCommit:"8fca2ec50a6133511b771a11559e24191b1aa2b4",
  GitTreeState:"clean",
  BuildDate:"2019-09-18T14:39:02Z",
  GoVersion:"go1.12.9",
  Compiler:"gc",
  Platform:"linux/amd64"
}

$ helm version
Client: &version.Version {
  SemVer:"v2.15.0",
  GitCommit:"c2440264ca6c078a06e088a838b0476d2fc14750",
  GitTreeState:"clean"
}
Server: &version.Version {
  SemVer:"v2.15.0",
  GitCommit:"c2440264ca6c078a06e088a838b0476d2fc14750",
  GitTreeState:"clean"
}

I didn't change anything in the Values.yml file. However, I manually changed the port for the pgpool service and set its type to LoadBalancer in order to be able to connect to it using Datagrip for debugging/tinkering.

Nov 13 '19 10:11 Telokis