400 on integration and session requests when self-hosting

[soundstep]

Description:

Client-side requests 400 when self-hosting.

Steps to reproduce:

1. visit https://fe-dora.onrender.com/welcome (this instance can have cold start or be interrupted)
2. check devtool request against https://fe-dora.onrender.com/api/auth/session

This is a reproduction of an issue I'm experiencing in GCP as well. I do not see this issue locally with the docker-compose local postgres and redis.

Any idea?

Expected behavior:

A 200 request.

Actual behavior:

A 400 response (https://fe-dora.onrender.com/api/auth/session):

{"message":"select * from \"Organization\" - relation \"Organization\" does not exist","stack":"error: select * from \"Organization\" - relation \"Organization\" does not exist\n    <internal/redacted>\n    <internal/redacted>\n    <internal/redacted>\n    <internal/redacted>\n    at Socket.emit (node:events:518:28)\n    at addChunk (node:internal/streams/readable:561:12)\n    at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)\n    at Readable.push (node:internal/streams/readable:392:5)\n    at TCP.onStreamRead (node:internal/stream_base_commons:189:23)\n    at TCP.callbackTrampoline (node:internal/async_hooks:130:17)"}

It seems it can't store a generated org, as I can also see this request when I try to set a github token, see the undefined (https://fe-dora.onrender.com/api/resources/orgs/undefined/integration):

{"message":"org_id must be a valid UUID","stack":"ValidationError: org_id must be a valid UUID\n    <internal/redacted>\n    <internal/redacted>\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)"}

Server Setup Information:

On render.com

postgres 14 redis and our Docker image:

FROM middlewareeng/middleware@sha256:8464f08370d0d47fad9c77aaa4d1feb5fd4492bfe25a48fb4d32cde745b58932

WORKDIR /app

COPY entrypoint.sh /app

ENTRYPOINT ["/app/entrypoint.sh"]

CMD ["/bin/bash", "-c", "/app/setup_utils/start.sh"]

Entrypoint:

#!/usr/bin/env bash

set -e

echo "Starting up application..."

exec "$@"

[soundstep]

Some more logs if that helps.

Docker logs:

Starting up application...
No /secretVol/secrets file
MHQ_EXTRACT_BACKEND_DEPENDENCIES
MHQ_EXTRACT_FRONTEND
MHQ_STARTING SUPERVISOR
config.ini not found. Running generate_config_ini.sh...
writing RSA key
Keys file created: /app/backend/analytics_server/mhq/config/config.ini
2025-09-30 10:06:01,832 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
2025-09-30 10:06:01,836 INFO supervisord started with pid 29
2025-09-30 10:06:02,851 INFO spawned: 'postgres' with pid 30
2025-09-30 10:06:02,859 INFO spawned: 'initialize_db' with pid 31
2025-09-30 10:06:02,923 INFO spawned: 'redis' with pid 32
2025-09-30 10:06:03,024 INFO spawned: 'backend' with pid 34
2025-09-30 10:06:03,125 INFO spawned: 'backend_sync' with pid 36
2025-09-30 10:06:03,231 INFO spawned: 'cron' with pid 39
2025-09-30 10:06:03,425 INFO spawned: 'frontend' with pid 41
2025-09-30 10:06:04,436 INFO success: initialize_db entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-09-30 10:06:13,709 INFO success: postgres entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2025-09-30 10:06:13,809 INFO success: redis entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2025-09-30 10:06:13,809 INFO success: backend entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2025-09-30 10:06:13,809 INFO success: backend_sync entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2025-09-30 10:06:13,810 INFO success: cron entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2025-09-30 10:06:13,810 INFO success: frontend entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
2025-09-30 10:06:21,329 WARN exited: initialize_db (exit status 1; not expected)

Postgres 14 logs:

[68dbd905.90d99-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=58842
[68dbd905.90d98-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=58834
[68dbd905.90d9a-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=58844
[68dbd905.90d9a-2] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authenticated: identity="postgres" method=md5 (/pgdata/dpg-d3afqfgdl3ps73epc3lg-a/pg_hba.conf:88)
[68dbd905.90d9a-3] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authorized: user=postgres database=doradb_b2nj SSL enabled (protocol=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256)
[68dbd905.90d9a-4] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  disconnection: session time: 0:00:03.096 user=postgres database=doradb_b2nj host=10.201.12.157 port=58844
[68dbd914.90da6-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=41108
[68dbd914.90da7-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=41112
[68dbd915.90da8-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=41114
[68dbd915.90da8-2] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authenticated: identity="postgres" method=md5 (/pgdata/dpg-d3afqfgdl3ps73epc3lg-a/pg_hba.conf:88)
[68dbd915.90da8-3] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authorized: user=postgres database=doradb_b2nj SSL enabled (protocol=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256)
[68dbd915.90da8-4] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  disconnection: session time: 0:00:03.109 user=postgres database=doradb_b2nj host=10.201.12.157 port=41114
[68dbd941.90de7-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=52658
[68dbd941.90de8-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=52664
[68dbd941.90de9-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=52666
[68dbd941.90de9-2] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authenticated: identity="postgres" method=md5 (/pgdata/dpg-d3afqfgdl3ps73epc3lg-a/pg_hba.conf:88)
[68dbd941.90de9-3] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authorized: user=postgres database=doradb_b2nj SSL enabled (protocol=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256)
[68dbd941.90de9-4] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  disconnection: session time: 0:00:03.008 user=postgres database=doradb_b2nj host=10.201.12.157 port=52666
[68dbd950.90df6-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=44582
[68dbd950.90df7-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=44592
[68dbd951.90df8-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=39754
[68dbd951.90df8-2] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authenticated: identity="postgres" method=md5 (/pgdata/dpg-d3afqfgdl3ps73epc3lg-a/pg_hba.conf:88)
[68dbd951.90df8-3] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authorized: user=postgres database=doradb_b2nj SSL enabled (protocol=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256)
[68dbd951.90df8-4] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  disconnection: session time: 0:00:03.012 user=postgres database=doradb_b2nj host=10.201.12.157 port=39754
[68dbd97d.90e37-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=52564
[68dbd97d.90e38-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=52576
[68dbd97d.90e39-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=52588
[68dbd97d.90e39-2] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authenticated: identity="postgres" method=md5 (/pgdata/dpg-d3afqfgdl3ps73epc3lg-a/pg_hba.conf:88)
[68dbd97d.90e39-3] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authorized: user=postgres database=doradb_b2nj SSL enabled (protocol=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256)
[68dbd97d.90e39-4] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  disconnection: session time: 0:00:03.005 user=postgres database=doradb_b2nj host=10.201.12.157 port=52588
[68dbd98c.90e47-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=46198
[68dbd98c.90e48-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=46214
[68dbd98d.90e49-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=50594
[68dbd98d.90e49-2] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authenticated: identity="postgres" method=md5 (/pgdata/dpg-d3afqfgdl3ps73epc3lg-a/pg_hba.conf:88)
[68dbd98d.90e49-3] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authorized: user=postgres database=doradb_b2nj SSL enabled (protocol=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256)
[68dbd98d.90e49-4] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  disconnection: session time: 0:00:03.030 user=postgres database=doradb_b2nj host=10.201.12.157 port=50594
[68d4fda2.8f-2995] user=,db=,app=,client=,LOG:  checkpoint starting: time
[68d4fda2.8f-2996] user=,db=,app=,client=,LOG:  checkpoint complete: wrote 1 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.002 s, sync=0.001 s, total=0.015 s; sync files=1, longest=0.001 s, average=0.001 s; distance=1 kB, estimate=1 kB
[68dbd9b9.90e87-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=47218
[68dbd9b9.90e88-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=47232
[68dbd9b9.90e89-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=47236
[68dbd9b9.90e89-2] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authenticated: identity="postgres" method=md5 (/pgdata/dpg-d3afqfgdl3ps73epc3lg-a/pg_hba.conf:88)
[68dbd9b9.90e89-3] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authorized: user=postgres database=doradb_b2nj SSL enabled (protocol=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256)
[68dbd9b9.90e89-4] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  disconnection: session time: 0:00:03.098 user=postgres database=doradb_b2nj host=10.201.12.157 port=47236
[68dbd9c8.90e97-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=49960
[68dbd9c8.90e96-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=49946
[68dbd9c9.90e98-1] user=[unknown],db=[unknown],app=[unknown],client=10.201.12.157,LOG:  connection received: host=10.201.12.157 port=49966
[68dbd9c9.90e98-2] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authenticated: identity="postgres" method=md5 (/pgdata/dpg-d3afqfgdl3ps73epc3lg-a/pg_hba.conf:88)
[68dbd9c9.90e98-3] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  connection authorized: user=postgres database=doradb_b2nj SSL enabled (protocol=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256)
[68dbd9c9.90e98-4] user=postgres,db=doradb_b2nj,app=[unknown],client=10.201.12.157,LOG:  disconnection: session time: 0:00:03.115 user=postgres database=doradb_b2nj host=10.201.12.157 port=49966

No Redis logs, not sure that's normal.