twenty icon indicating copy to clipboard operation
twenty copied to clipboard

Migration failing in v1.0, when do clear install via k8s/docker | Migration issue with already created empty databases

Open yeskiy opened this issue 4 months ago • 7 comments

Bug Description

When setting up a fresh installation of Twenty v1.0 using Docker or Kubernetes, the database migration process fails right at startup. Specifically, the app tries to access the core.keyValuePair table, but that table doesn’t exist yet. As a result, the migration fails and the app can’t start.

Running database setup and migrations...
query failed: SELECT "KeyValuePair"."id" AS "KeyValuePair_id", "KeyValuePair"."userId" AS "KeyValuePair_userId", "KeyValuePair"."workspaceId" AS "KeyValuePair_workspaceId", "KeyValuePair"."key" AS "KeyValuePair_key", "KeyValuePair"."value" AS "KeyValuePair_value", "KeyValuePair"."textValueDeprecated" AS "KeyValuePair_textValueDeprecated", "KeyValuePair"."type" AS "KeyValuePair_type", "KeyValuePair"."createdAt" AS "KeyValuePair_createdAt", "KeyValuePair"."updatedAt" AS "KeyValuePair_updatedAt", "KeyValuePair"."deletedAt" AS "KeyValuePair_deletedAt" FROM "core"."keyValuePair" "KeyValuePair" WHERE (("KeyValuePair"."type" = $1) AND ("KeyValuePair"."userId" IS NULL) AND ("KeyValuePair"."workspaceId" IS NULL)) -- PARAMETERS: ["CONFIG_VARIABLE"]
error: error: relation "core.keyValuePair" does not exist

Image

Expected behavior

Should create all necessary tables and schemas, so the app can finish starting up.

Technical inputs

  • Running twentycrm/twenty:v1.0 Docker image
  • Clean installation (not an upgrade)
  • PostgreSQL ver 16.4.0
  • Redis ver 8.0.2

P.S

Hey! I just want to say a big thank-you for this project! Awesome job to make it to 1.0! If any more details would help, I’m happy to assist further — just let me know.

UPDATE

I used the bitnami/postgresql image in order to start the PostgreSQL database. One of the "additionalities" of this approach - Bitnami image will take care of creating a database for you when PostgreSQL starts. This leads to an exact error:

services:
  server:
    image: twentycrm/twenty:v1.0
    ports:
      - "3000:3000"
    environment:
      NODE_PORT: 3000
      # -------------------------------------------------------------------------------------- #
      # will fail cause "twenty" schema will be created automatically when the database starts     
      PG_DATABASE_URL: postgres://postgres:${PG_DATABASE_PASSWORD:-postgres}@db:5432/twenty
      # -------------------------------------------------------------------------------------- #
      SERVER_URL: http://localhost:3000
      REDIS_URL: ${REDIS_URL:-redis://redis:6379}
# * redacted * #

  db:
    image: bitnami/postgresql:16
    volumes:
      - db-data:/bitnami/postgresql
    environment:
      # -------------------------------------------------------------------------------------- #
      POSTGRESQL_DATABASE: twenty
      # -------------------------------------------------------------------------------------- #
      POSTGRESQL_POSTGRES_PASSWORD: ${PG_DATABASE_PASSWORD:-postgres}
      POSTGRESQL_PASSWORD: ${PG_DATABASE_PASSWORD:-postgres}
    healthcheck:
      test: pg_isready -U postgres -h localhost -d postgres
      interval: 5s
      timeout: 5s
      retries: 10
    restart: always

yeskiy avatar Jun 28 '25 00:06 yeskiy

Hey! 👋 I ran into this exact issue while setting up the project locally and was able to get it working — so just sharing what worked for me in case it helps someone else:

✅ TL;DR — Solution Make sure your .env is pointing to the correct Postgres database.

By default, my .env had:

PG_DATABASE_URL=postgres://postgres:postgres@localhost:5432/default

But the application expects the core schema to exist in a database called core, not default.

So I had to:

  1. Create the core database manually:
createdb core
  1. Create the core schema inside that database:
psql -U postgres -d core -c "CREATE SCHEMA core;"
  1. Update .env to:
PG_DATABASE_URL=postgres://postgres:postgres@localhost:5432/core
  1. Then rerun the migration:
npx nx run twenty-server:database:migrate

Once I did that, everything worked as expected. ✅

rajatsingh23 avatar Jun 28 '25 12:06 rajatsingh23

@rajatsingh23 Thanks! Found pretty same solution, just absolutely sideways 😊 I used the Bitnami PostgreSQL package, which can create databases when starting. It looks like it's not okay for the current migration implementation.

yeskiy avatar Jun 28 '25 12:06 yeskiy

I encountered the very same issue while trying to create a twenty integration for .NET Aspire. The postgres integration for it would usually create the database for you, leading to the migration failing. For now my workaround is to just stop using a preconfigured database and simply pass the name to twenty, so twenty migrates everything. A better developer experience would be great if this scenario could be supported though, as the individual database would then appear in the developer dashboard.

My guess would be the packages/twenty-docker/twenty/entrypoint.sh needs some adjustments for it. Instead of only checking pg_database, maybe also check in pg_tables?

WhatzGames avatar Jun 29 '25 16:06 WhatzGames

Fresh installation same identical issue. Any news on this?

mariushosting avatar Jun 30 '25 09:06 mariushosting

i used

PG_DATABASE_URL: postgresql://twenty:password@twenty-db-test:5432/default

and it magically worked. make sure you're both updating worker and server if you're using docker-compose

argestes avatar Jun 30 '25 14:06 argestes

i used

PG_DATABASE_URL: postgresql://twenty:password@twenty-db-test:5432/default

and it magically worked. make sure you're both updating worker and server if you're using docker-compose

Did you create the database first? This issue is pretty much about that. Create the database first, then run twenty. That should give you the same results as we got.

WhatzGames avatar Jun 30 '25 15:06 WhatzGames

No I'm running docker compose with clean db. I'm not using dev version

------ Original Message ------ From "Obed Kooijman" @.> To "twentyhq/twenty" @.> Cc "Yiğit Ünlü" @.>; "Comment" @.> Date 6/30/2025 6:09:20 PM Subject Re: [twentyhq/twenty] Migration failing in v1.0, when do clear install via k8s/docker | Migration issue with already created empty databases (Issue #12936)

WhatzGames left a comment (twentyhq/twenty#12936) https://github.com/twentyhq/twenty/issues/12936#issuecomment-3019544481

i used

PG_DATABASE_URL: @.***:5432/default

and it magically worked. make sure you're both updating worker and server if you're using docker-compose

Did you create the database first? This issue is pretty much about that. Create the database first, then run twenty. That should give you the same results as we got.

— Reply to this email directly, view it on GitHub https://github.com/twentyhq/twenty/issues/12936#issuecomment-3019544481, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAP5EZBLVLDX5OOUH4FDPX33GFHKBAVCNFSM6AAAAACAKGQXOGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMJZGU2DINBYGE. You are receiving this because you commented.Message ID: @.***>

argestes avatar Jun 30 '25 17:06 argestes

Facing the same issue. With a fresh database.

mateusztylec avatar Jul 01 '25 09:07 mateusztylec

Alright, I investigated the issue with setting up a fresh instance.

First, there is a script located at packages/twenty-server/scripts/setup-db.ts that should set up all schemas, extensions, and so on. One of these is the core schema, which was not created when I followed the guide in the official documentation.

It appears that this file was not executed, even though it should have been according to packages/twenty-docker/twenty/entrypoint.sh.

I then accessed the twenty container and attempted to run it manually with yarn database:init:prod, but I encountered the following error:

/app/packages/twenty-server $ yarn database:init:prod
npm error code EAI_AGAIN
npm error syscall getaddrinfo
npm error errno EAI_AGAIN
npm error request to https://registry.npmjs.org/ts-node failed, reason: getaddrinfo EAI_AGAIN registry.npmjs.org
npm error A complete log of this run can be found in: /home/node/.npm/_logs/2025-07-01T09_29_38_191Z-debug-0.log
/app/packages/twenty-server $ cat /home/node/.npm/_logs/2025-07-01T09_29_38_191Z-debug-0.log
0 verbose cli /usr/local/bin/node /usr/local/lib/node_modules/npm/bin/npm-cli.js
1 info using [email protected]
2 info using [email protected]
3 silly config load:file:/usr/local/lib/node_modules/npm/npmrc
4 info config found workspace root at /app
5 silly config load:file:/app/.npmrc
6 silly config load:file:/home/node/.npmrc
7 silly config load:file:/usr/local/etc/npmrc
8 verbose title npm exec ts-node ./scripts/setup-db.ts
9 verbose argv "exec" "--" "ts-node" "./scripts/setup-db.ts"
10 verbose logfile logs-max:10 dir:/home/node/.npm/_logs/2025-07-01T09_29_38_191Z-
11 verbose logfile /home/node/.npm/_logs/2025-07-01T09_29_38_191Z-debug-0.log
12 silly logfile done cleaning log files
13 silly packumentCache heap:4345298944 maxSize:1086324736 maxEntrySize:543162368
14 http fetch GET https://registry.npmjs.org/npm attempt 1 failed with EAI_AGAIN
15 http fetch GET https://registry.npmjs.org/ts-node attempt 1 failed with EAI_AGAIN
16 http fetch GET https://registry.npmjs.org/npm attempt 2 failed with EAI_AGAIN
17 http fetch GET https://registry.npmjs.org/ts-node attempt 2 failed with EAI_AGAIN
18 http fetch GET https://registry.npmjs.org/npm attempt 3 failed with EAI_AGAIN
19 http fetch GET https://registry.npmjs.org/ts-node attempt 3 failed with EAI_AGAIN
20 verbose type system
21 verbose stack FetchError: request to https://registry.npmjs.org/ts-node failed, reason: getaddrinfo EAI_AGAIN registry.npmjs.org
21 verbose stack     at ClientRequest.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/minipass-fetch/lib/index.js:130:14)
21 verbose stack     at ClientRequest.emit (node:events:518:28)
21 verbose stack     at emitErrorEvent (node:_http_client:104:11)
21 verbose stack     at _destroy (node:_http_client:898:9)
21 verbose stack     at onSocketNT (node:_http_client:918:5)
21 verbose stack     at process.processTicksAndRejections (node:internal/process/task_queues:91:21)
22 error code EAI_AGAIN
23 error syscall getaddrinfo
24 error errno EAI_AGAIN
25 error request to https://registry.npmjs.org/ts-node failed, reason: getaddrinfo EAI_AGAIN registry.npmjs.org
26 verbose cwd /app/packages/twenty-server
27 verbose os Linux 5.4.0-216-generic
28 verbose node v22.16.0
29 verbose npm  v10.9.2

So it seems like container didn't have access to the internet. (DNS set to 127.0.0.11)

/app/packages/twenty-server $ ping -c3 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=42 time=8.226 ms
64 bytes from 8.8.8.8: seq=1 ttl=42 time=8.317 ms
64 bytes from 8.8.8.8: seq=2 ttl=42 time=8.647 ms

--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 8.226/8.396/8.647 ms
/app/packages/twenty-server $ nslookup registry.npmjs.org
Server:		127.0.0.11
Address:	127.0.0.11:53

** server can't find registry.npmjs.org: SERVFAIL

** server can't find registry.npmjs.org: SERVFAIL

So i added DNS records to my docker compose

    dns:
      - 8.8.8.8
      - 1.1.1.1

The issue with internet access has been resolved.

I restarted the containers, but this did not immediately fix the problem. I accessed the twenty container, ran yarn database:init:prod again, restarted the container, and everything seems to be working fine now.

To summarize, the problem was that packages/twenty-server/scripts/setup-db.ts did not run when the container started. This is why we all experienced issues with the missing core schema.

mateusztylec avatar Jul 01 '25 10:07 mateusztylec

Closing this issue due to inactivity. Please reopen it if needed!

Bonapara avatar Jul 21 '25 12:07 Bonapara