Prisma broken for next when generating prisma client in non-node_modules dirs

[t3dotgg]

My bad :(

Broke by my PR here: https://github.com/t3-oss/create-t3-turbo/pull/37

Error here, unable to find schema: https://twitter.com/t3dotgg/status/1582936894991609857

Tried solution here: https://github.com/prisma/prisma/issues/10433#issuecomment-1264687986

Gets further, but new error:

@acme/nextjs:dev: prisma:query SELECT `main`.`Post`.`id`, `main`.`Post`.`title`, `main`.`Post`.`content` FROM `main`.`Post` WHERE 1=1 LIMIT ? OFFSET ?
@acme/nextjs:dev: prisma:query SELECT 1
@acme/nextjs:dev: prisma:query SELECT name FROM sqlite_schema
@acme/nextjs:dev:       WHERE type='table'
@acme/nextjs:dev:       ORDER BY name;
@acme/nextjs:dev:       
@acme/nextjs:dev: failed to fetch, PrismaClientKnownRequestError: 
@acme/nextjs:dev: Invalid `prisma.post.findMany()` invocation:
@acme/nextjs:dev: 
@acme/nextjs:dev: 
@acme/nextjs:dev: The table `main.Post` does not exist in the current database.
@acme/nextjs:dev:     at RequestHandler.handleRequestError (webpack-internal:///(api)/../../packages/db/.generated/client/runtime/index.js:31874:19)
@acme/nextjs:dev:     at RequestHandler.request (webpack-internal:///(api)/../../packages/db/.generated/client/runtime/index.js:31853:18)
@acme/nextjs:dev:     at async PrismaClient._request (webpack-internal:///(api)/../../packages/db/.generated/client/runtime/index.js:32853:24)
@acme/nextjs:dev:     at async eval (webpack-internal:///(api)/../../packages/api/src/router/post.ts:19:26)
@acme/nextjs:dev:     at async resolveMiddleware (file:///Users/theobrowne/Code/contrib/npm-turbo-t3/node_modules/@trpc/server/dist/router-28a2c129.mjs:190:30)
@acme/nextjs:dev:     at async callRecursive (file:///Users/theobrowne/Code/contrib/npm-turbo-t3/node_modules/@trpc/server/dist/router-28a2c129.mjs:226:32)
@acme/nextjs:dev:     at async resolve (file:///Users/theobrowne/Code/contrib/npm-turbo-t3/node_modules/@trpc/server/dist/router-28a2c129.mjs:254:24)
@acme/nextjs:dev:     at async file:///Users/theobrowne/Code/contrib/npm-turbo-t3/node_modules/@trpc/server/dist/resolveHTTPResponse-f2b378b6.mjs:374:32
@acme/nextjs:dev:     at async Promise.all (index 0)
@acme/nextjs:dev:     at async resolveHTTPResponse (file:///Users/theobrowne/Code/contrib/npm-turbo-t3/node_modules/@trpc/server/dist/resolveHTTPResponse-f2b378b6.mjs:371:28) {
@acme/nextjs:dev:   code: 'P2021',
@acme/nextjs:dev:   clientVersion: '4.5.0',
@acme/nextjs:dev:   meta: { table: 'main.Post' }
@acme/nextjs:dev: }

assumption is that it can't actually find the db in next, which is weird because it can totally find it in prisma studio? idk.

[t3dotgg]

Update: The solution @roxkstar74 posted in the Prisma issue appears to work if your db url is

1. NOT a file path
2. An environment variable in BOTH apps/nextjs AND packages/db

I am so tired.

[juliusmarminge]

Will this be fixed if we build the packages instead of direct imports?

[juliusmarminge]

Update: The solution @roxkstar74 posted in the Prisma issue appears to work if your db url is

1. NOT a file path
2. An environment variable in BOTH apps/nextjs AND packages/db

I am so tired.

1. tbh that's not 100% out there, file path is relative and db url isn't. if there are issues resolving the path that issue goes away

2. that one is.. weird to say the least

[subaanqasim]

I spent a full day debugging this issue a few days ago, so I thought I'd give my two cents into how I've solved it (for now at least, since it's kinda hacky). Apologies for the long post, but I thought I'd explain my thought processes in case someone with a bigger brain can produce a legit solution based off the information below.

TLDR:

1. To fix nextjs ENOENT error:

// .npmrc

public-hoist-pattern[]=*prisma*

2. To fix prisma client type inference issues:

// schema.prisma

generator client {
  provider        = "prisma-client-js"
  output          = "../../../node_modules/.prisma/client"
}

// packages/db/index.ts

import { PrismaClient } from "@prisma/client";

export const prisma = new PrismaClient({
  log:
    process.env.NODE_ENV === "development"
      ? ["query", "error", "warn"]
      : ["error"],
});

3. Ensure proper caching of prisma client following the above changes, in turbo.json:

"pipeline": {
    "db-generate": {
      "outputs": ["../../node_modules/.prisma/client"]
    }
}

---

Context: the repo I'm working with is essentially a create-t3-turbo monorepo - 2 next apps (with next-auth, but no expo), with prisma and tRPC as shared packages.

From my understanding, the ENOENT in the nextjs app is due to it trying to import the prisma client from the node_modules at the root of the monorepo. Adding public-hoist-pattern[]=*prisma* to .npmrc seems to fix that issue by seemingly hoisting the prisma client generation to the root (from https://github.com/prisma/prisma/issues/12921#issuecomment-1152901509). HOWEVER, this will only work if PrismaClient is imported from @prisma/client in db/index.ts and not from a custom generator path.

However, this then results in an issue for the tRPC api package - the imported prisma client in api/src/context.ts from will be of type any. This is despite instances of prisma imported from @acme/db in the next app resolving the types properly (currently, you'll notice that prisma is type any in the next app and api package in create-t3-turbo due to no hoisting). If you set a custom generator path for your prisma schema and create your client from that in db/index.ts then the prisma type issues fix, but you get the ENOENT error again in the next app due to the custom path of the client not being hoisted up to the root node_modules. Humongous pain.

Importing from @prisma/client just exports the default schema generation path (node_modules/.prisma/client). In theory, according to this explanation of how pnpm and prisma client imports works, we should be able to generate the client into the root by having output = "../../../node_modules/.prisma/client" in schema.prisma and import PrismaClient from @prisma/client in db/index.ts. This is because we're now hoisting prisma to root and generating the client in the root, so all the prisma stuff is sitting together.

Then, to ensure proper caching of the prisma client, change the db-generate output in turbo.json to the root:

"pipeline": {
    "db-generate": {
      "outputs": ["../../node_modules/.prisma/client"]
    }
}

This is what ultimately worked for me - no ENOENT errors, proper type inference of the prisma client in all packages and apps, and proper caching by turborepo.

Also, with this solution, it might be worth locking pnpm version to <7.13.5 in Vercel builds using the ENABLE_EXPERIMENTAL_COREPACK=1 env variable due to this potential issue.

[ernestoresende]

I would like to cross-check if #46 and f30c8af893f11f0145c92b4845dd9bb14cfe4df4 actually solves this completely. It seems to solve all issues brought by @subaanqasim above:

1. Hoisting the Prisma Client to the root of the project removes ENOENT errors from Next.js;
2. Caching behavior from Turbo is preserved, even without specifying the outputs option on the pipeline config (see https://github.com/t3-oss/create-t3-turbo/pull/46#discussion_r1001537019);
3. Types are preserved when using Prisma through the trpc context handler:

[JesseKoldewijn]

Was the planned outcome to get the generated client in the node modules or as @TheoBr was working on in the db package still? Because I sorta half-assed reproduced his outcome just now just using the already provided sqlite db and everything seems to work just fine with the generated client on the "packages/db/prisma/generated" path.

[juliusmarminge]

Was the planned outcome to get the generated client in the node modules or as @TheoBr was working on in the db package still?

As long as they're cacheable it doesnt matter i dont think

[JesseKoldewijn]

Was the planned outcome to get the generated client in the node modules or as @TheoBr was working on in the db package still?

As long as they're cacheable it doesnt matter i dont think

Current env I got is:

DATABASE_URL="file:./prisma/prisma/db.sqlite"

generated prisma client path:

./generated (defined as output in the schema.prisma file)

db-generate script in turbo:

"outputs": ["./packages/db/prisma"],

[JesseKoldewijn]

It should be cachable if it's correctly outside the node_modules folder, right?

[juliusmarminge]

It should be cachable if it's correctly outside the node_modules folder, right?

You dont get nextjs errors with that?

[JesseKoldewijn]

It should be cachable if it's correctly outside the node_modules folder, right?

You dont get nextjs errors with that?

I haven't tried that yet

[JesseKoldewijn]

oh yeah lol, I got a ENOENT no such file error on the trpc API file pointing towards the prisma schema.

[juliusmarminge]

oh yeah lol, I got a ENOENT no such file error on the trpc API file pointing towards the prisma schema.

Yep same error as Theo screenshotted up top

[JesseKoldewijn]

yup, unfortunatly.

[nickyhajal]

I'm hitting this as well - has there been any clean solution here?

[JesseKoldewijn]

I'm hitting this as well - has there been any clean solution here?

Haven't found one as of yet unfortunatly.

[adambarito]

Need to sleep and come back to take a look, but these are the first set of changes that let me run dev and get through a clean build with no ENOENT issues and prisma type inference carrying through (including adding and testing a new model in prisma.schema).

/.env ENABLE_EXPERIMENTAL_COREPACK=1

/next.config.mjs:

defineNextConfig({
  webpack: config => {
    config.externals = [...(config.externals || []), '@prisma/client'];
    return config;
  }
})

/.npmrc (I removed node-linker=hoisted and public-hoist-patter[]=*prisma*

strict-peer-dependencies=true
auto-install-peers=true
link-workspace-packages = true
shamefully-hoist = true
shared-workspace-shrinkwrap = true
access = public
enable-pre-post-scripts = true

/turbo.json:

"db-generate": {
      "outputs": ["../../node_modules/.prisma/client"]
    },

/packages/db/prisma/schema.prisma

generator client {
    provider = "prisma-client-js"
    output ="../../../node_modules/.prisma/client"
}

/packages/db/.env

# Prisma
DATABASE_URL=<INSERT_DATABASE_URL>

Everything else matches current 'create-t3-turbo-auth', including a .env matching the example in /apps/nextjs. Can do a better job referencing later.

[nickyhajal]

Using the changes from @subaanqasim got things running for me (very appreciated) locally.

When I run the deploy, I see this:

@padelstripes/db:db-generate: > @padelstripes/[email protected] db-generate /vercel/path0/packages/db
@padelstripes/db:db-generate: > prisma generate
@padelstripes/db:db-generate: 
@padelstripes/db:db-generate: Prisma schema loaded from prisma/schema.prisma
@padelstripes/db:db-generate: 
@padelstripes/db:db-generate: ✔ Generated Prisma Client (4.5.0 | library) to ./../../node_modules/.prisma/client in 255ms
@padelstripes/db:db-generate: You can now start using Prisma Client in your code. Reference: https://pris.ly/d/client
@padelstripes/db:db-generate: ```
@padelstripes/db:db-generate: import { PrismaClient } from '../../node_modules/.prisma/client'

However, the API calls return:

ERROR	Error: @prisma/client did not initialize yet. Please run "prisma generate" and try to import it again.
In case this error is unexpected for you, please report it in https://github.com/prisma/prisma/issues

Seems like either the db package is looking in the wrong place, or the generation output is going maybe two levels up from the root and out of the project?

Anyone hit on anything like this?

[subaanqasim]

Prisma was working fine in my api routes in my deployments until this same issue hit me in the face yesterday (might have started earlier, but haven't been working over on it over the past week to notice).

I have all of my dependency versions (and pnpm version) controlled, so it can't have been an update in a dependency, so I'm assuming Vercel have changed the way pnpm monorepos resolve dependencies.

[juliusmarminge]

I don't know what you're struggling with. Why are you trying to hoist it down to the root node modules?

Last time I checked, prisma worked fine and was cacheable out-of-the-box with the setup on main?

[jadamita]

Any update on this?

[juliusmarminge]

Any update on this?

Use the repo as it is and it works

[juliusmarminge]

Closing this as not planned.

Prisma is cached correctly as it is today so I don't see the need to fix this

[JesseKoldewijn]

@juliusmarminge You managed to get both the prisma client and generated db cached? awesome! 😄

[mbifulco]

Hey gang - coming back to this issue since I've been wrestling it for a few days here. I'm building an app with create-t3-turbo, and have been trying to keep my changes roughly in sync with what's happened in this repo since I started with it in December.

I was getting ENOENT errors from my nextjs project when trying to import prisma -- and found my (current) fix in this comment - https://github.com/prisma/prisma/issues/12921#issuecomment-1414839304 - the prisma team have an experimental webpack plugin which seems to address this issue. The prisma issue was linked earlier here, but there's been some activity since then. Figured a bump might help.

[JesseKoldewijn]

Hey gang - coming back to this issue since I've been wrestling it for a few days here. I'm building an app with create-t3-turbo, and have been trying to keep my changes roughly in sync with what's happened in this repo since I started with it in December.

I was getting ENOENT errors from my nextjs project when trying to import prisma -- and found my (current) fix in this comment - prisma/prisma#12921 (comment) - the prisma team have an experimental webpack plugin which seems to address this issue. The prisma issue was linked earlier here, but there's been some activity since then. Figured a bump might help.

As far as I know the ENOENT errors were fixed in this repo already, so not really sure what the issue is in your case man.

[mbifulco]

Yeah, fair enough - I figured it's worth mentioning here in case someone else is in a similar situation.