Postgres cleanup script fails

[csvan]

Self-Hosted Version

24.4.1

CPU Architecture

x86_64

Docker Version

24.0.7

Docker Compose Version

2.24.0

Steps to Reproduce

Run the postgres cleanup script from https://develop.sentry.dev/self-hosted/troubleshooting/#postgres :

# Only keep the last 7 days of nodestore data. We heavily use performance monitoring.
docker compose run --rm -T web cleanup --days 7 -m nodestore -l debug
# This ensures pg-repack exists before running as the container gets recreated on upgrades
docker compose run --rm -T postgres bash -c "apt update && apt install -y --no-install-recommends postgresql-14-repack && su postgres -c 'pg_repack -E info -t nodestore_node'"

Expected Result

It should clean out the database and return 0

Actual Result

It fails with the following:

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:2 http://apt.postgresql.org/pub/repos/apt bullseye-pgdg InRelease [123 kB]
Get:3 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
Get:4 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Get:5 http://deb.debian.org/debian bullseye/main amd64 Packages [8,068 kB]
Get:6 http://apt.postgresql.org/pub/repos/apt bullseye-pgdg/main amd64 Packages [314 kB]
Get:7 http://apt.postgresql.org/pub/repos/apt bullseye-pgdg/14 amd64 Packages [2,582 B]
Get:8 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [271 kB]
Get:9 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [18.8 kB]
Fetched 9,005 kB in 2s (4,490 kB/s)
Reading package lists...
Building dependency tree...
Reading state information...
44 packages can be upgraded. Run 'apt list --upgradable' to see them.

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
  postgresql-14-repack
0 upgraded, 1 newly installed, 0 to remove and 44 not upgraded.
Need to get 107 kB of archives.
After this operation, 255 kB of additional disk space will be used.
Get:1 http://apt.postgresql.org/pub/repos/apt bullseye-pgdg/main amd64 postgresql-14-repack amd64 1.5.0-1.pgdg110+1 [107 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 107 kB in 0s (382 kB/s)
Selecting previously unselected package postgresql-14-repack.
(Reading database ... 12113 files and directories currently installed.)
Preparing to unpack .../postgresql-14-repack_1.5.0-1.pgdg110+1_amd64.deb ...
Unpacking postgresql-14-repack (1.5.0-1.pgdg110+1) ...
Setting up postgresql-14-repack (1.5.0-1.pgdg110+1) ...
update-alternatives: using /usr/lib/postgresql/14/bin/pg_repack to provide /usr/bin/pg_repack (pg_repack) in auto mode
Processing triggers for postgresql-common (244.pgdg110+1) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
ERROR: could not connect to database: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: No such file or directory
	Is the server running locally and accepting connections on that socket?

Event ID

No response

[hubertdeng123]

I believe this is happening since the su postgres -c... part of the command in the docs is not able to properly connect to postgres. Is your postgres instance otherwise healthy? Can you try to bring down your postgres container before running the command?

[csvan]

Apologies for the late reply @hubertdeng123 . Yes, the image is otherwise working fine (we are using vanilla on-prem so we are just using the defaul postgres image provided by Sentry). I tried stopping Sentry before running the cleanup, but the error is unfortunately still the same.

[azaslavsky]

Can you run through this script to troubleshoot? I'm curious where things are breaking down - is psql not seeing the socket, does not exist, or something else.

[getsantry[bot]]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀