[Bug]: Plugin "@netlify/plugin-nextjs" internal error: ENOENT: no such file or directory with runtime 5.0.0-beta.7

[caomicc]

[jesseday]

11:15:27 AM: Error: ENOENT: no such file or directory, open "/opt/build/repo/apps/web/required-server-files.json"

I believe the file does exist at /opt/build/repo/apps/web/.next/required-server-files.json

I don't have a solution yet, but maybe someone working on the plugin knows where this bug is. Will dig a bit

[jesseday]

Based on testing in the codebase that @caomicc originally posted it appears that there's a bug in the 5.x-beta.7 and 8 version of the @netlify/plugin-nextjs that it assumes the wrong path. From a team member

It assumes that standalone files should live in .next/standalone/.next/ but the correct path is .next/standalone/apps/web/.next/ in monorepo configuration. I've tired many different configuration options without any success.

[SilverFox70]

We are now experiencing the same issue with it not being able to find the required-server-files.json. We tried pinning the version to "5.0.0-beta.4" but that did not help. We have also tried version "5.0.0-rc.0" with no luck.

[MarcL]

Hey @jesseday and @SilverFox70.

We just released an early access version that should fix many issues. Can you try installing @netlify/plugin-nextjs@rc and let us know if you still have the problems?

Thanks!

[caomicc]

@MarcL Hi, I too am still having this issue.

Once trying this update to rc, which installs ^5.0.0-rc.4, I can no longer install any packages and am getting a new error:

9:41:45 AM: npm ERR! code ERR_INVALID_ARG_TYPE
9:41:45 AM: npm ERR! The "from" argument must be of type string. Received undefined
9:41:45 AM: npm ERR! A complete log of this run can be found in: /opt/buildhome/.npm/_logs/2024-03-13T13_41_08_575Z-debug-0.log
9:41:45 AM: Error during npm install
9:41:45 AM: Failing build: Failed to install dependencies
9:41:45 AM: Failed during stage "Install dependencies": dependency_installation script returned non-zero exit code: 1

Reverting to ^5.0.0-beta.9 afterwards does continue to install dependencies however fails on the same spot in the build.

[MarcL]

@caomicc @jesseday @SilverFox70 - We've released v5 now and started rolling it out. Can you remove your pinned dependencies and build and redeploy and check it now?

[dragons-library]

I ran into something similar, and I think I have a workaround.

TL;DR: instead of reading from .next/, read from .netlify/.next/. This feels like a temporary workaround and probably not the ideal solution.

  const path = require('path');

  module.exports = {
    async onPostBuild({ constants, inputs }) {
      const { PUBLISH_DIR } = constants; // where PUBLISH_DIR === '.next'
      const mediaDir = 'static/media';
<     const cwd = path.join(PUBLISH_DIR, mediaDir);
>     const cwd = path.join('.netlify', PUBLISH_DIR, mediaDir);
      // ...
    }
  };

After upgrading to v5 (currently on 5.1.2), I noticed our static assets weren't being uploaded to our CDN. We use a custom onPostBuild hook to grab all the nextJS assets and push them to the CDN.

Some digging, I found that .next directory now looks more like the built public directory. More digging and I found that .netlify/.next looks like the original .next output.

.next/
  _next/
  assets/
  audio/
  cms/
  downloads/
  img/
  android-chrome-192x192.png
  android-chrome-256x256.png
  apple-touch-icon.png
  browserconfig.xml
  favicon-16x16.png
  favicon-32x32.png
  favicon-512x512.png
  favicon.ico
  robots.txt
  site.webmanifest
  sitemap-0.xml
  sitemap.xml

.netlify/.next/
  cache/
  server/
  standalone/
  static/
  BUILD_ID
  build-manifest.json
  export-marker.json
  images-manifest.json
  next-minimal-server.js.nft.json
  next-server.js.nft.json
  package.json
  prerender-manifest.js
  prerender-manifest.json
  react-loadable-manifest.json
  required-server-files.json
  routes-manifest.json
  trace

FWIW, when I inspect the root .next directory outside of the plugin, it looks correct, but when debugging inside of the plugin it is as noted above.

[dragons-library]

Root issue seems to stem from this behavior in the v5 plugin...

var onPostBuild = async (options) => {
  await tracer.withActiveSpan("onPostBuild", async () => {
    await publishStaticDir(new PluginContext(options));
  });
};
...
var onEnd = async (options) => {
  await tracer.withActiveSpan("onEnd", async () => {
    await unpublishStaticDir(new PluginContext(options));
  });
};

...

var publishStaticDir = async (ctx) => {
  try {
    await rm(ctx.tempPublishDir, { recursive: true, force: true });
    await mkdir(basename(ctx.tempPublishDir), { recursive: true });
    await rename(ctx.publishDir, ctx.tempPublishDir);
    await rename(ctx.staticDir, ctx.publishDir);
  } catch (error) {
    ctx.failBuild("Failed publishing static content", error instanceof Error ? { error } : {});
  }
};
var unpublishStaticDir = async (ctx) => {
  try {
    if (existsSync(ctx.tempPublishDir)) {
      await rename(ctx.publishDir, ctx.staticDir);
      await rename(ctx.tempPublishDir, ctx.publishDir);
    }
  } catch {
  }
};

So any plugin with an onPostBuild (or onSuccess/onEnd) event hook that runs after @netlify/plugin-nextjs will have the "wrong" .next directory contents.

Knowing this, a better workaround (instead of reading .netlify/.next) is probably to move @netlify/plugin-nextjs to the end of your plugins in netlify.toml:

-[[plugins]]
-  package = "@netlify/plugin-nextjs"

 [[plugins]]
   package = "@newrelic/netlify-plugin"

 [[plugins]]
   package = "./plugins/my-custom-plugin"
 
+[[plugins]]
+  package = "@netlify/plugin-nextjs"

[ascorbic]

@dragons-library yes, I'd recommend just ensuring the Netlify plugin runs last, rather than relying on the internals of our build process.

Everyone else: in 5.1.2 there are some improvements to finding the build output on sites with custom setups, so if you have issues that relate to that then I'd recommend trying again to see if it now works