NextJS monoerpo "schema.prisma1 could not be found"

[maxtechera]

Bug description

I'm having the issue with prisma files not found I cannot provide a reproduction right now.

I'm using Next 13 with the app dir inside a monorepo and having db package. I'm using the monorepo workaround which works almost everytime but it seems it hits some race conditions at times.

When going to a route the "compiling" messages shows and as soon as it's ready the page loads and throws the error. I can see that during compilation the .next/server/app/ get's created and before the prisma files get copied the app tries to load and fails.

Most times a refresh solves the issue and I can see the prisma files being copied into the newly created .next folder, where some other times it wont.

This happens a lot more consistently when doing client side navigation to a route or requesting an API which is not compiled yet, and needs to be compiled on the fly.

The compilation most times shows a success message after copying the files. I've checed the Prisma plugin code and can verify that when the error happens it doesn't get to the copy file code.

This usually happens a lot more on API routes when requested. i.e The next-auth route which get's requested from the client and/or the server.

How to reproduce

Expected behavior

I expect the compilation to complete correctly 100% of the time after the Prisma plugin copies the files into the correct .next/<path_to_route folder

Prisma information

generator client {
  provider             = "prisma-client-js"
  output               = "../generated/prisma-client"
  referentialIntegrity = "prisma"
}

// Add your code using Prisma Client

Environment & setup

• OS: macOS
• Database: PostgreSQL
• Node.js version: v18.13.0

Prisma Version

prisma                  : 4.14.0
@prisma/client          : 4.13.0
Current platform        : debian-openssl-3.0.x
Query Engine (Node-API) : libquery-engine d9a4c5988f480fa576d43970d5a23641aa77bc9c (at ../../node_modules/.pnpm/@[email protected]/node_modules/@prisma/engines/libquery_engine-debian-openssl-3.0.x.so.node)
Migration Engine        : migration-engine-cli d9a4c5988f480fa576d43970d5a23641aa77bc9c (at ../../node_modules/.pnpm/@[email protected]/node_modules/@prisma/engines/migration-engine-debian-openssl-3.0.x)
Format Wasm             : @prisma/prisma-fmt-wasm 4.14.0-67.d9a4c5988f480fa576d43970d5a23641aa77bc9c
Default Engines Hash    : d9a4c5988f480fa576d43970d5a23641aa77bc9c
Studio                  : 0.484.0

[Jolg42]

@maxtechera Does this issue persist after upgrading to the latest version, 4.15.0? If you could share a reproduction, it would be the best but as you said you can't right now, maybe share screenshots or error messages you are seeing. Which Next.js version are you using?

Sidenote: I noticed the version for prisma and @prisma/client are different, we recommend keeping them in sync.

[millsp]

Hey @maxtechera if you are using the import/export syntax, you might be interested is trying our ESM branch. How is this related? It happens that with ESM, Next.js is able to copy the files on your behalf, so you can even remove the workaround plugin. https://github.com/prisma/prisma/issues/5030#issuecomment-1565181377

[maxtechera]

@Jolg42 - I've updated to the latest version but nothing has changed. It really seems like a race condition where NextJS evaluates the module/route/component as soon as the JS is compiled but before the webpack compile step finishes and thus the plugin doesn't have enough time to copy the prisma files. Once this happens the only way to fix the errored state is to manually copy the Prisma files into the .next folder OR restart the dev server.

@millsp - Exciting things with ESM branch, I've tried this as well but I'm not getting a copy file behaviour, do I need to update any other configuration on the NextJS app? I only update the prisma and @prisma/client packages. Tried both with and without the plugin. With the plugin I'm getting the same behaviour as with the main version.

[cyrus-za]

I am running into the exact same issue, even after using the webpack plugin at https://www.prisma.io/docs/guides/other/troubleshooting-orm/help-articles/nextjs-prisma-client-monorepo

It seems like this discussion and specifically this comment is related: https://github.com/prisma/prisma/issues/12921#issuecomment-1591727303

EDIT: my bad I see this issue was created as a follow up to that conversation

PS. I am using latest version of next and prisma with turborepo+yarn workspace but not building the app from turborepo.

prisma is located inside the app directory itself, not its own package

[beefancohen]

running into this non-stop :/

[maxtechera]

I'm running into this all the time as well, my temporary solution is to copy/paste the files where they need to be, and then try not to stop the dev server at all and eventually the files will be everywhere they need to be for that session.

I notice that sometimes this happens more, and I'm inclined to think there is a race condition. Maybe there is a different Webpack hook we can use in the plugin to copy the files earlier rather than wait until the compilation is done. I believe the compilation get's done and Next tries to access the file before the copy operation completes.

[jkomyno]

Hey @maxtechera, thanks for the info you provided, but we weren't able to reproduce this problem locally. Could you please share a minimally reproducible project that we can clone and run locally to observe this behavior? The problem may be due to your project structure, or something related that we would like to investigate. Thanks!

[relsunkaev]

I was able to fix this issue by having the db package's package.json "main" point to "index.js" instead of "index.ts".

[beefancohen]

@maxtechera have you been able to resolve this at all?

[jkomyno]

Hi @maxtechera, as others have shared, this issue you're experiencing may not be caused by Prisma directly, but rather by a possible misconfiguration of the package.json settings. Could you please see if @relsunkaev's solution solves this issue, and if not, reply with more information to this comment? Thank you!

[prceasar]

@jkomyno I work with @maxtechera and can confirm that this issue is happening even with the db package.json pointing main to the index.js file.

[jthomaschewski]

I had the same issue when using pnpm as my package manager. Changing the main entry in the package.json did not resolve the problem.

What finally worked for me was to enable module hoisting for the prisma packages by adding a .npmrc file to the project root with the following content:

public-hoist-pattern[]=*prisma*
public-hoist-pattern[]=*eslint*
public-hoist-pattern[]=*prettier*

(The eslint & prettier patterns are enabled by default if no .npmrc exists, so they should be added to avoid breaking eslint/prettier)

Also the monorepo workaround plugin is not needed anymore (it might even break the setup)

[jkomyno]

Hey @jthomaschewski, your hoisting workaround is indeed interesting, but I really wonder why that's needed in the first place.

Could you please share a minimally reproducible example (could be just a pnpm monorepo with a couple of packages and a README telling us which commands to run) so that we have something we can run to investigate this issue and improve the DX for all other users? Thanks!

[prceasar]

@jkomyno FYI, Our project is also using PNPM, without hoisting.

[arrocke]

We are also experiencing this in an nx monorepo. You see this at arrocke/gloss-translation#db-package. There are instructions for setting things up in docs/contributing. Then you can load the site and you should see the error.

I put some logging in the prisma library code right before the schema file is read, and in the webpack plugin before and after when the file is copied. There is definitely a race condition, but I don't know the cause

[kduprey]

I am experiencing the same thing with a pnpm monorepo, db in a separate package. Major blocker for the project at hand.

[beefancohen]

i fixed this by not outputting prisma generated files to a custom directory, but not sure what the root of the problem was

[kduprey]

@beefancohen - Did you just remove the package from the external package directory, and only use in the NextJS application directory?

[beefancohen]

no, we have a turborepo monorepo with an app and a database package and use pnpm. we used to output prisma to packages/database/generated and it was "causing" all of these problems -- when we moved it back to its default location in node_modules, everything seemed to go away.

[kduprey]

@beefancohen - Do you have a repo that I can take a look at and compare? I just made those changes myself in my application, but still getting the same missing file error of schema.prisma1.

[rodrigoehlers]

I have a similar setup and tried using the official plugin to workaround this issue. However, I added a few breakpoints to the plugin and noticed that there is a race-condition. The plugin checks whether the files are present. It see they are but afterwards Next.js removes the whole dir and adds just the ones it needs, therefore removing both the schema and the runtime.

I'm not sure but this could also be related to SWC being used.

[liamross]

@rodrigoehlers This helped me fix it!

next.config.mjs

import {PrismaPlugin} from '@prisma/nextjs-monorepo-workaround-plugin';

// IMPORTANT: Initialize it here before to avoid race condition!!!
const prismaPlugin = new PrismaPlugin();

/** @type {import('next').NextConfig} */
const nextConfig = {
  webpack: (config, {isServer}) => {
    if (isServer) config.plugins = [...config.plugins, prismaPlugin];
    return config;
  },
  // ...
}

And my prisma generator:

generator client {
    provider = "prisma-client-js"
}

[jhylmb]

@liamross This not fix the problem for me, Thanks anyway.

[jhylmb]

I think I found a temporary solution.

reason

When non-compiled path (eg, apps/<project-name>/app/post/new/page.tsx) compiled, the workaround-plugin tries to copy schema.prisma and libquery_engine-xxx files from the original path (eg, packages/db/node_modules/.prisma/client) to target path (eg, apps/<project-name>/.next/server/app/post/new) using fromDestPrismaMap.

But during compile process, the workaround-plugin once set the desired destination path correctly then it overrides the desired destination path with already compiled path(eg, apps/<project-name>/.next/server/app) to fromDestPrismaMap

• desired fromDestPrismaMap

{ 
   "packages/db/node_modules/.prisma/client/schema.prisma": 
        "apps/<package-name>/.next/server/app/post/new/schema.prisma1"
}

• overrided fromDestPrismaMap

{ 
   "packages/db/node_modules/.prisma/client/schema.prisma": 
        "apps/<package-name>/.next/server/app/schema.prisma1"
}

So, we need to workaround the override situation.

temporary solution

change this: https://github.com/prisma/prisma/blob/841208dd871434c3201920ff10d407a52c868981/packages/nextjs-monorepo-workaround-plugin/index.js#L69-L91

to:

prismaFiles.forEach(async (f) => {
  const from = path.join(prismaDir, f)

  // if we have multiple schema.prisma files, we need to rename them
  if (f === 'schema.prisma' && fromDestPrismaMap[from] === undefined) {
    f += ++schemaCount
  }

  // if we already have renamed it, we need to get its "renamed" name
  if (f.includes('schema.prisma') && fromDestPrismaMap[from] !== undefined) {
    f = path.basename(fromDestPrismaMap[from])
  }

  if (f.includes('schema.prisma')) {
    // update "schema.prisma" to "schema.prisma{number}" in the sources
    const newSourceString = oldSourceContents.replace(/schema\.prisma/g, f)
    const newRawSource = new sources.RawSource(newSourceString)
    compilation.updateAsset(assetName, newRawSource)
  }

  const dest = path.join(assetDir, f)

  if (process.env.NODE_ENV === "development") {
    // check if the destination file already exists
    // if it does, it causes the override issue, otherwise we need to copy the required files 
    if ((await fs.access(dest).catch(() => false)) === false) {
      fromDestPrismaMap[from] = dest
    }
  } else {
    fromDestPrismaMap[from] = dest
  }
})

Hope this helps and somebody who better understands then me will PR the right solution as soon as possible.

[raivan0830]

good!!

[jthomaschewski]

Hey @jthomaschewski, your hoisting workaround is indeed interesting, but I really wonder why that's needed in the first place.

Could you please share a minimally reproducible example (could be just a pnpm monorepo with a couple of packages and a README telling us which commands to run) so that we have something we can run to investigate this issue and improve the DX for all other users? Thanks!

Sorry for not responding yet, I tried to create a reproducible example but oddly, I couldn't replicate the faulty behavior anymore while using the default client output directory (.prisma/client within node_modules).

I created an example repo which reproduces the issue from OP (custom client output directory while using the workaround plugin):

https://github.com/jthomaschewski/prisma-reproduce-issue-19784

It contains multiple pages (P1, P2 ... P5) which all trigger a render of an api route using prisma.
When clicking the links one after another, it usually trigger the race condition and the error should appear on the terminal.

-- One more thing regarding the hoisting workaround: The official turbo repo documentation does suggest this workaround when using Prisma within a turborepo monorepo.
See https://turbo.build/repo/docs/handbook/tools/prisma#2-add-a-new-database-package

[srosato]

I had a similar issue that I ended up resolving with patching the webpack plugin. See https://github.com/prisma/prisma/issues/20276#issuecomment-1728717267

[divx-alisson-ninomiya]

@srosato I'm taking no one wants to solve this problem at the prisma team so I wanted to reach out to you about the patch you made last week... I tried using the same patch in my environment but it didn't quite work, so can I check with you if my dependencies are set correctly?

(I had to use yarn patch because of my team, but) After the patch, at the top level package.json I have the "^5.3.1" version of @prisma/nextjs-monorepo-workaround-plugin set as a devDependency, while the patched version is set as a resolution. The child packages/services I have all list "@prisma/nextjs-monorepo-workaround-plugin": "*" as a peerDependency. Is this right or do you feel there is something off?

[toniopelo]

Waow, this one is really painful. I've banged my head on all the walls I could find for the last two days. I wasn't sure how to search for a similar issue, I finally found this thread.

On my side, switching to client component instead of server component seems to lower the frequency of errors (to none for now, but it might just be because I only have a single route handler at this point, so there are a lot less possibilities for this bug to happen ? The randomness doesn't help to track down real solutions).

Anyway, I can confirm that when I look to the .next folder, the problem arises when navigating to a page where the prisma client is not populated. When switching to client components, I don't have errors and prisma files are not copied on any routes anymore, this probably has to do with stuff that are included in the pages bundle when using server component vs client components (even though I import types from prisma client in these client components and use the exact same code).

If anyone found a working solution I'm all hears of course, it seems like all the provided ones above have been proven inconclusive for others. I'm not a savvy webpack hacker, so I can't really deep dive into this but I'll post back if I find anything of course.

[toniopelo]

I finally dove into this and created a PR that you can see mentioned above. @jhylmb Thanks, you were somehow on the right track and that helped me, but your version still has the bug if my understanding is right (see my PR description for the details).

In the meantime, while the PR is not merged, I created a repo with my patched files to use as a package directly, so I don't have to deal with monkey patching the node_modules. Of course, use at your own risks since it's not validated by the prisma team, but if it helps, here is what I put in my package.json:

  "devDependencies": {
    ...
    "@prisma/nextjs-monorepo-workaround-plugin": "toniopelo/prisma-nextjs-monorepo-workaround-plugin",
    ...
  }