[Urgent] Module build failed: UnhandledSchemeError: Reading from "node:fs/promises" is not handled by plugins (Unhandled scheme).

[ruai0511]

System Info

Langchain version: 0.0.72 Next.js version: v13.4.1

Who can help?

@hwchase17 @vowelparrot

Information

• [ ] The official example notebooks/scripts
• [X] My own modified scripts

Related Components

• [ ] LLMs/Chat Models
• [X] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [X] Vector Stores / Retrievers
• [ ] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [ ] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

I'm using next.js and want to integrate langchain with Chroma. The error happened when I try to import { Chroma } from "langchain/vectorstores" in a my next.js file.

When I run npm run dev, the error message is in the screenshot below:

These are all the imports I'm doing: I checked and the error message is caused by the first line.

Below is my next.config.js

Expected behavior

I only expect it to compile successfully

[amirgamil]

i'm able to reproduce this ^

[oalexdoda]

I confirm this when using edge runtime on Next.js. Works without it.

[ludibel]

This config worked for me in next.config.js

/** @type {import('next').NextConfig} */
const nextConfig = {
    reactStrictMode: true,
    swcMinify: true,
    webpack: (config, { webpack }) => {
      config.experiments = { ...config.experiments, topLevelAwait: true };
      config.externals["node:fs"] = "commonjs node:fs";
      config.resolve.fallback = {
        ...config.resolve.fallback,
        fs: false,
    };
      config.plugins.push(

        new webpack.NormalModuleReplacementPlugin(
          /^node:/,
          (resource) => {
            resource.request = resource.request.replace(/^node:/, '');
          },
        ),
      );
  
      return config;
    }
  }
  
  module.exports = nextConfig;
/

[vowelparrot]

Thank you for raising this issue! @dev2049 linked this issue to the JS package issues board since it seems to be related to the langchainjs functionality in NextJS

https://github.com/hwchase17/langchainjs/issues/1211

[jacoblee93]

Hey @ruai0511, we've deprecated importing from langchain/vectorstores and langchain/embeddings in favor of new entrypoints to avoid issues like this (some other vector stores are Node only and use fs).

In your case the imports would be import { Chroma } from "langchain/vectorstores/chroma" and import { OpenAIEmbeddings} from "langchain/embeddings/openai" - can you try changing to that (and perhaps updating your LangChainJS version)?

[dosubot[bot]]

Hi, @ruai0511! I'm Dosu, and I'm helping the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

Based on the information provided, it seems that the issue you reported about a module build failure in Next.js when importing a module from "langchain/vectorstores" has been resolved. Several users, including amirgamil, altechzilla, and ludibel, were able to reproduce the error and shared their experiences and configurations that worked for them. Additionally, jacoblee93 mentioned that the deprecated imports from langchain/vectorstores and langchain/embeddings have alternative imports to try.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your contribution, and please don't hesitate to reach out if you have any further questions or concerns!

[sarmstead]

This config worked for me in next.config.js

/** @type {import('next').NextConfig} */
const nextConfig = {
    reactStrictMode: true,
    swcMinify: true,
    webpack: (config, { webpack }) => {
      config.experiments = { ...config.experiments, topLevelAwait: true };
      config.externals["node:fs"] = "commonjs node:fs";
      config.resolve.fallback = {
        ...config.resolve.fallback,
        fs: false,
    };
      config.plugins.push(

        new webpack.NormalModuleReplacementPlugin(
          /^node:/,
          (resource) => {
            resource.request = resource.request.replace(/^node:/, '');
          },
        ),
      );
  
      return config;
    }
  }
  
  module.exports = nextConfig;
/

I'm working on a Next.js project and this config worked for me as well. Thank you, @ludibel!

In case anyone is wondering what's going on here...

• The config.externals key tells Webpack to skip bundling a particular package. Basically, at the time of writing, Webpack can't figure out how to handle the new node: URI scheme. So we are bypassing bundling those imports altogether.
  • According to the docs: "The externals configuration option provides a way of excluding dependencies from the output bundles. Instead, the created bundle relies on that dependency to be present in the consumer's (any end-user application) environment"
• config.resolve.fallback is yet another bypassing measure.
  • According to the docs, here we are telling Webpack to "redirect module requests when normal resolving fails." So, don't keep trying to resolve and thereby fail at the build step. Instead, move along, Webpack, and continue with our build!
• Finally, the config.plugins.push key is adding a specific plugin to our Webpack config.
  • The plugin we are using here is the NormalModuleReplacementPlugin which allows us to take over module resolution behavior. NormalModuleReplacementPlugin() takes two parameters: resourceRegExp and newResource.
  • The resourceRegExp is a regular expression to match the resource we are looking for (in this case we are looking for resources that start with node:, as would be the case, for example, with a file that contains import { readFile } from "node:fs/promises";).
  • newResource is being used here as a function. As such, we can "overwrite the request attribute of the supplied resource". See the plugin docs for more on this.

Hope this helps somebody!

[mariosknl]

I'm having the same issue and the next.config provided doesn't solve it. I'm working on a NextJS 14.0.3 project using PayloadCMS. My Node version is 18.19.0. I'm seeing this error when I'm trying to initialize Mailtrap for my email system. Any idea how to solve this error please?

[sarmstead]

I'm having the same issue and the next.config provided doesn't solve it. I'm working on a NextJS 14.0.3 project using PayloadCMS. My Node version is 18.19.0. I'm seeing this error when I'm trying to initialize Mailtrap for my email system. Any idea how to solve this error please?

Screenshot 2024-02-10 at 08 33 49

@mariosknl, that looks similar to what I was experiencing. I'm curious though ... do you know where in your code you may be using the Node fs module? And is there any additional file information you can provide (a link to your repo would be even better if it's public)?