TurboPack build fails: Cannot resolve 'fs' in pdf-parse next 15

[rossanodr]

Checked other resources

• [x] I added a very descriptive title to this issue.
• [x] I searched the LangChain.js documentation with the integrated search.
• [x] I used the GitHub search to find a similar question and didn't find it.
• [x] I am sure that this is a bug in LangChain.js rather than my code.
• [x] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

import { WebPDFLoader } from "@langchain/community/document_loaders/web/pdf";
 await new WebPDFLoader(file, {
        splitPages: false,
        parsedItemSeparator: "",

      }).load();

or

import { PDFLoader } from "@langchain/community/document_loaders/fs/pdf";

await new PDFLoader(file, {
        splitPages: false,
        parsedItemSeparator: "",
      }).load();

Error Message and Stack Trace (if applicable)

× ./node_modules/pdf-parse/lib/pdf.js/v1.10.100/build/pdf.js:17378:10
Module not found: Can't resolve 'fs'
17378 | var fs = require('fs');
| ^^^^^^^^^^^^^
17379 | var http = require('http');
17380 | var https = require('https');
17381 | var url = require('url');

Description

I am trying to build a Next.js project with TurboPack enabled and using WebPDFLoader or PDFLoader. I expect the build to succeed, as it does when using Webpack with appropriate fallback settings. Instead, the build fails due to a module resolution error for fs, http, and other Node-specific modules. If I don't use the --turbo tag it works

System Info

 "next": "15.0.3-canary.2",

devDependencies: { '@faker-js/faker': '^7.6.0', '@jest/globals': '^29.5.0', '@langchain/anthropic': '', '@langchain/aws': '', '@langchain/cerebras': '', '@langchain/cohere': '', '@langchain/core': 'workspace:', '@langchain/google-genai': '', '@langchain/google-vertexai': '', '@langchain/google-vertexai-web': '', '@langchain/groq': '', '@langchain/mistralai': '', '@langchain/ollama': '', '@langchain/scripts': '>=0.1.0 <0.2.0', '@swc/core': '^1.3.90', '@swc/jest': '^0.2.29', '@tsconfig/recommended': '^1.0.2', '@types/handlebars': '^4.1.0', '@types/html-to-text': '^9', '@types/js-yaml': '^4', '@types/jsdom': '^21.1.1', '@types/uuid': '^9', '@types/ws': '^8', '@typescript-eslint/eslint-plugin': '^5.58.0', '@typescript-eslint/parser': '^5.58.0', axios: '^0.26.0', cheerio: '1.0.0-rc.12', dotenv: '^16.0.3', dpdm: '^3.12.0', eslint: '^8.33.0', 'eslint-config-airbnb-base': '^15.0.0', 'eslint-config-prettier': '^8.6.0', 'eslint-plugin-import': '^2.27.5', 'eslint-plugin-jest': '^27.6.0', 'eslint-plugin-no-instanceof': '^1.0.1', 'eslint-plugin-prettier': '^4.2.1', handlebars: '^4.7.8', jest: '^29.5.0', 'jest-environment-node': '^29.6.4', openai: '^4.41.1', peggy: '^3.0.2', prettier: '^2.8.3', 'release-it': '^17.6.0', rimraf: '^5.0.1', rollup: '^3.19.1', 'ts-jest': '^29.1.0', typeorm: '^0.3.20', typescript: '~5.1.6', wikipedia: '^2.1.2' }, peerDependencies: { '@langchain/anthropic': '', '@langchain/aws': '', '@langchain/cerebras': '', '@langchain/cohere': '', '@langchain/core': '>=0.2.21 <0.4.0', '@langchain/google-genai': '', '@langchain/google-vertexai': '', '@langchain/google-vertexai-web': '', '@langchain/groq': '', '@langchain/mistralai': '', '@langchain/ollama': '', axios: '', cheerio: '', handlebars: '^4.7.8', peggy: '^3.0.2', typeorm: '' }

[dosubot[bot]]

There are similar issues related to the "Module not found: Can't resolve 'fs'" error when using LangChain.js with Next.js. Here are some relevant threads and solutions:

1. Module not found: Error: Can't resolve 'fs' when using langchain document loader: This discussion suggests modifying the next.config.js file to disable the 'fs' module in the client-side code. You can do this by setting fs to false in the resolve.fallback section of the webpack configuration. Here's an example:

   /** @type {import('next').NextConfig} */
   const nextConfig = {
     webpack: (config, { isServer }) => {
       if (!isServer) {
         config.resolve.fallback = {
           ...config.resolve.fallback,
           fs: false,
           "node:fs/promises": false,
           module: false,
           perf_hooks: false,
         };
       }
       return config;
     },
   }
   
   module.exports = nextConfig;
   

2. Getting Module build failed: UnhandledSchemeError: Reading from "node:fs/promises" is not handled by plugins (Unhandled scheme). in building React app with Langchain JS: This issue also deals with similar module resolution problems and suggests creating a mock for the 'fs' module. You can create a mocks/fs.js file with module.exports = {}; and then alias it in your Next.js config:

   const path = require('path');
   module.exports = {
     webpack: (config, { isServer }) => {
       if (!isServer) {
         config.resolve.alias['fs'] = path.resolve(__dirname, './mocks/fs.js');
       }
       return config;
     },
   }
   

These solutions should help resolve the module resolution errors you're encountering when building your Next.js project with TurboPack.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[jacoblee93]

Easiest thing to do might be to enable node compatibility if possible - the web loader should work with turbopack but bundlers can be tricky :(

[rossanodr]

even with "moduleResolution": "node" I can't make it work :(

Easiest thing to do might be to enable node compatibility if possible - the web loader should work with turbopack but bundlers can be tricky :(

[chidii03]

nice