[BUG]: Not able to locate migrations when using vercel postgres

[Aaditya-23]

What version of drizzle-orm are you using?

0.26.3

What version of drizzle-kit are you using?

0.18.1

Describe the Bug

I am using drizzle with remix. In my development env the application works fine. But when I deploy it on vercel I get the following log

{"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"Error: Can't find meta/_journal.json file","reason":{"errorType":"Error","errorMessage":"Can't find meta/_journal.json file","stack":["Error: Can't find meta/_journal.json file","

you can clone the repo from here: https://github.com/Aaditya-23/remix-drizzle

my drizzle.config.ts looks like this.

import type { Config } from "drizzle-kit";
 
export default {
  schema: "./app/db/schema.server.ts",
  out: "./drizzle",
} satisfies Config; 

file from where I am exporting the client looks like this.

import { drizzle } from "drizzle-orm/vercel-postgres";
import { migrate } from "drizzle-orm/vercel-postgres/migrator";
import { sql } from "@vercel/postgres";
import * as schema from "./schema.server";

export const db = drizzle(sql, { schema });
migrate(db, { migrationsFolder: "drizzle" });

[Aaditya-23]

any updates or do you need something else? Would be appreciated if somebody helps me out.

[mharrvic]

was able to run migrations with vercel Postgres here https://github.com/mharrvic/serverless-llm-playground/pull/1

[Aaditya-23]

was able to run migrations with vercel Postgres here mharrvic/serverless-llm-playground#1

I am using remix while you are using Next. Is this a remix related problem? because you are stating it works fine with next

[ESchouten]

Same issue in Sveltekit

EDIT: @Aaditya-23 I have found the culprit. Migrate should only be called in the build/deployment phase. When the migrate(...) function is called in the same file as the database initialisation (export const db = drizzle(...)) this is done with every request since the service is started and stopped each request. Extract this migrate call to a separate .ts file to prevent it from being called on every request.

[Aaditya-23]

@ESchouten you are right and this works. I think they should update their docs because I picked this up from the docs.

[jscul]

I am still getting this error.

Error: Can't find meta/_journal.json file
    at Object.readMigrationFiles (/home/jscul/Software/blog-tutorial/node_modules/src/migrator.ts:40:9)
    at migrate (/home/jscul/Software/blog-tutorial/node_modules/src/node-postgres/migrator.ts:9:21)
    at _migrate (/home/jscul/Software/blog-tutorial/app/db/migrate.ts:6:10)
    at Object.<anonymous> (/home/jscul/Software/blog-tutorial/app/db/migrate.ts:9:1)
    at Module._compile (node:internal/modules/cjs/loader:1218:14)
    at Module.m._compile (/home/jscul/Software/blog-tutorial/node_modules/ts-node/src/index.ts:1618:23)
    at Module._extensions..js (node:internal/modules/cjs/loader:1272:10)
    at Object.require.extensions.<computed> [as .ts] (/home/jscul/Software/blog-tutorial/node_modules/ts-node/src/index.ts:1621:12)
    at Module.load (node:internal/modules/cjs/loader:1081:32)
    at Function.Module._load (node:internal/modules/cjs/loader:922:12)

I get this even when running the migration separate from Remix.

[wojtekKrol]

any updates?

[dewodt]

Same issue in Sveltekit

EDIT: @Aaditya-23 I have found the culprit. Migrate should only be called in the build/deployment phase. When the migrate(...) function is called in the same file as the database initialisation (export const db = drizzle(...)) this is done with every request since the service is started and stopped each request. Extract this migrate call to a separate .ts file to prevent it from being called on every request.

Works 100%, Thank you!

[do4Mother]

to solve this i use lodash to prevent call migrate function multiple times. here is example

/**
 * Migrate the database on startup. prevent call multiple times.
 */
once(() => {
  console.log("Migrating database...");
  migrate(db, { migrationsFolder: "drizzle" });
})();

[fredrik-sogaard]

~~I'm getting this as well when running migrations via drizzle-kit~~ I deleted my drizzle folder and tried again, and it worked.

[lightbluepoppy]

Same issue in Sveltekit

EDIT: @Aaditya-23 I have found the culprit. Migrate should only be called in the build/deployment phase. When the migrate(...) function is called in the same file as the database initialisation (export const db = drizzle(...)) this is done with every request since the service is started and stopped each request. Extract this migrate call to a separate .ts file to prevent it from being called on every request.

In my case this did not solve the error. EDIT: Solved. Code below is the one that is successful. It turns out that I had not generated a migration before executing migrate.ts.

// ./migrate.ts
import { migrate } from "drizzle-orm/neon-http/migrator"
import * as server from "./db/server.ts"
const main = async () => {
    try {
        await migrate(server.db, { migrationsFolder: "./db/migrations" })
        console.log("Migration complete")
    } catch (error) {
        console.log(error)
    }
    process.exit(0)
}
main()

// ./drizzle.config.ts
import type { Config } from "drizzle-kit"

export default {
    schema: "./db/schema/schema.ts",
    out: "./drizzle",
    driver: "pg",
} satisfies Config

// ./db/server.ts
import "dotenv/config"
import { drizzle } from "drizzle-orm/neon-http"
import { neon, neonConfig } from "@neondatabase/serverless"

neonConfig.fetchConnectionCache = true
const sql = neon(process.env.DATABASE_URL!)
export const db = drizzle(sql)

~/Code/Project/project master* ❯ pnpm migrate                                                              05:05:48

> my-qwik-basic-starter@ migrate /Users/home/Code/Project/project
> tsx migrate.ts

Error: Can't find meta/_journal.json file
    at readMigrationFiles (/Users/home/Code/Project/project/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected][email protected]/node_modules/src/migrator.ts:40:9)
    at migrate (/Users/home/Code/Project/project/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected][email protected]/node_modules/src/neon-http/migrator.ts:18:21)
    at main (/Users/home/Code/Project/project/migrate.ts:12:15)
    at <anonymous> (/Users/home/Code/Project/project/migrate.ts:19:1)
    at Object.<anonymous> (/Users/home/Code/Project/project/migrate.ts:19:6)
    at Module._compile (node:internal/modules/cjs/loader:1241:14)
    at Object.S (/Users/home/Library/pnpm/global/5/.pnpm/[email protected]/node_modules/tsx/dist/cjs/index.cjs:1:1292)
    at Module.load (node:internal/modules/cjs/loader:1091:32)
    at Module._load (node:internal/modules/cjs/loader:938:12)
    at cjsLoader (node:internal/modules/esm/translators:284:17)

~/Code/Project/project master* ❯ pnpm drizzle-kit generate:pg                                                 18:55:17
drizzle-kit: v0.20.6
drizzle-orm: v0.29.1

No config path provided, using default 'drizzle.config.ts'
Reading config file '/Users/orcinus/Code/Project/project/drizzle.config.ts'
11 tables
tag 5 columns 0 indexes 1 fks
tag_tree 2 columns 0 indexes 2 fks
task 9 columns 0 indexes 1 fks
task_dependency 2 columns 0 indexes 2 fks
task_instance 6 columns 0 indexes 1 fks
task_instance_dependency 2 columns 0 indexes 2 fks
task_statistics 6 columns 0 indexes 1 fks
task_instance_time_entry 5 columns 0 indexes 1 fks
task_instance_tree 2 columns 0 indexes 2 fks
task_tree 2 columns 0 indexes 2 fks
user 3 columns 0 indexes 0 fks

[✓] Your SQL migration file ➜ db/migrations/0000_silent_black_cat.sql 🚀

[email protected] | ISC | deps: 6 | versions: 33
[email protected] | MIT | deps: 14 | versions: 325
node v20.8.1

[kelcho-spense]

Hey, this issue arises because for some reson drizzle kit cant find your migration folder await migrate(server.db, { migrationsFolder: "./db/migrations" }) to resolve this i used the __dirname to fix this
await migrate(db, {migrationsFolder: __dirname + '/migrations'});

[CarolinaSanches24]

🚀 Migration started... ❌ Migration failed: Error: Can't find meta/_journal.json file

também estou com mesmo erro import {migrate} from "drizzle-orm/mysql2/migrator"; import { db } from "../db";

const main = async () => { console.log("🚀 Migration started..."); await migrate(db, { migrationsFolder: "./migrations" }); console.log("✅ Migration completed."); };

main().catch((error) => { console.error("❌ Migration failed:", error); process.exit(1); });

import { Config } from 'drizzle-kit' import { env } from './api/src/utils/dotenv/env'

export const dbUrl = mysql://${env.variables.DB_USER}:${env.variables.DB_PASSWORD}@${env.variables.DB_HOST}:${env.variables.DB_PORT}/${env.variables.DB_NAME}

export default { schema: "./api/src/services/schemas/user/userShema.ts*", out: "./drizzle", driver: 'mysql2', dbCredentials: { connectionString: String(dbUrl), }, verbose:true, strict:true, } as Config; import { drizzle } from "drizzle-orm/mysql2"; import mysql from "mysql2/promise"; import { env } from "../../../utils/dotenv/env";

const poolConnection = mysql.createPool({ host: env.variables.DB_HOST, user: env.variables.DB_NAME, password: env.variables.DB_PASSWORD, database: env.variables.DB_NAME, port: parseInt(env.variables.DB_PORT), });

export const db = drizzle(poolConnection);

[haexyh]

I got this error because I didn't ran 'drizzle-kit generate: