plugin-cloud-storage: Can't resolve 'fs' error with cloud-storage-plugin s3 adapter

[haliacmon]

Link to reproduction

https://github.com/haliacmon/payload-reproduction

Describe the Bug

Getting Module not found: Error: Can't resolve 'fs' error with plugin-cloud-storage and s3-adapter

ERROR in ./node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected]/node_modules/@payloadcms/plugin-cloud-storage/dist/adapters/s3/handleUpload.js 140:27-40
Module not found: Error: Can't resolve 'fs' in '...\node_modules\.pnpm\@[email protected]_@[email protected]_@[email protected][email protected]\node_modules\@payloadcms\plugin-cloud-storage\dist\adapters\s3'

To Reproduce

1- npx create-payload-app@latest reproduce-error-s3 -t blank --use-pnpm using postgress 2- pnpm add @payloadcms/plugin-cloud-storage 3- pnpm add @aws-sdk/client-s3 @aws-sdk/lib-storage aws-crt 4- optional. add s3 related variables to .env. Not necessary to reproduce the error. 5- adding build config for plugin-cloud-storage & s3-adapter by instructions. (error happens after step 5) https://github.com/payloadcms/payload/tree/c10db332cdeb3f702d378b0649be20eb789c13e1/packages/plugin-cloud-storage#s3-adapter

//payload.config.ts
...
import { cloudStorage } from '@payloadcms/plugin-cloud-storage'
import { s3Adapter } from '@payloadcms/plugin-cloud-storage/s3'

const adapter = s3Adapter({
  config: {
    credentials: {
      accessKeyId: process.env.S3_ACCESS_KEY_ID,
      secretAccessKey: process.env.S3_SECRET_ACCESS_KEY,
    },
    region: process.env.S3_REGION,
  },
  bucket: process.env.S3_BUCKET,
})
...

plugins: [
...
    cloudStorage({
      collections: {
        "my-collection-slug": {
          adapter: adapter,
        },
      },
    }),
  ],
...

Payload Version

2.1.1

Adapters and Plugins

db-postgres, plugin-cloud-storage, S3 Adapter

[denolfe]

@haliacmon Can you give the beta package a try? pnpm add @payloadcms/plugin-cloud-storage@beta

[haliacmon]

more errors this time.

webpack built e8766af5f2dc60d372e8 in 38675ms
ERROR in ./node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected]/node_modules/@payloadcms/plugin-cloud-storage/dist/adapters/s3/handleUpload.js 136:49-62
Module not found: Error: Can't resolve 'fs' in 'D:\_code2\Azak\payload-reproduction\node_modules\.pnpm\@[email protected]_@[email protected]_@[email protected][email protected]\node_modules\@payloadcms\plugin-cloud-storage\dist\adapters\s3'

ERROR in ./node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected]/node_modules/@payloadcms/plugin-cloud-storage/dist/adapters/s3/webpack.js 64:49-62
Module not found: Error: Can't resolve 'fs' in 'D:\_code2\Azak\payload-reproduction\node_modules\.pnpm\@[email protected]_@[email protected]_@[email protected][email protected]\node_modules\@payloadcms\plugin-cloud-storage\dist\adapters\s3'

ERROR in ./node_modules/.pnpm/[email protected]/node_modules/detect-file/index.js 7:9-22
Module not found: Error: Can't resolve 'fs' in 'D:\_code2\Azak\payload-reproduction\node_modules\.pnpm\[email protected]\node_modules\detect-file'

ERROR in ./node_modules/.pnpm/[email protected]/node_modules/fill-range/index.js 60:11-26
Module not found: Error: Can't resolve 'util' in 'D:\_code2\Azak\payload-reproduction\node_modules\.pnpm\[email protected]\node_modules\fill-range'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
        - add a fallback 'resolve.fallback: { "util": require.resolve("util/") }'
        - install 'util'
If you don't want to include a polyfill, you can use an empty module like this:
        resolve.fallback: { "util": false }

ERROR in ./node_modules/.pnpm/[email protected]/node_modules/findup-sync/index.js 4:13-26
Module not found: Error: Can't resolve 'fs' in 'D:\_code2\Azak\payload-reproduction\node_modules\.pnpm\[email protected]\node_modules\findup-sync'

ERROR in ./node_modules/.pnpm/[email protected]/node_modules/global-prefix/index.js 7:9-22
Module not found: Error: Can't resolve 'fs' in 'D:\_code2\Azak\payload-reproduction\node_modules\.pnpm\[email protected]\node_modules\global-prefix'

ERROR in ./node_modules/.pnpm/[email protected]/node_modules/homedir-polyfill/index.js 2:9-22
Module not found: Error: Can't resolve 'os' in 'D:\_code2\Azak\payload-reproduction\node_modules\.pnpm\[email protected]\node_modules\homedir-polyfill'    

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
        - add a fallback 'resolve.fallback: { "os": require.resolve("os-browserify/browser") }'
        - install 'os-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
        resolve.fallback: { "os": false }

ERROR in ./node_modules/.pnpm/[email protected]/node_modules/homedir-polyfill/polyfill.js 2:9-22
Module not found: Error: Can't resolve 'fs' in 'D:\_code2\Azak\payload-reproduction\node_modules\.pnpm\[email protected]\node_modules\homedir-polyfill'    

ERROR in ./node_modules/.pnpm/[email protected]/node_modules/isexe/index.js 1:9-22
Module not found: Error: Can't resolve 'fs' in 'D:\_code2\Azak\payload-reproduction\node_modules\.pnpm\[email protected]\node_modules\isexe'

ERROR in ./node_modules/.pnpm/[email protected]/node_modules/isexe/mode.js 3:9-22
Module not found: Error: Can't resolve 'fs' in 'D:\_code2\Azak\payload-reproduction\node_modules\.pnpm\[email protected]\node_modules\isexe'

ERROR in ./node_modules/.pnpm/[email protected]/node_modules/isexe/windows.js 3:9-22
Module not found: Error: Can't resolve 'fs' in 'D:\_code2\Azak\payload-reproduction\node_modules\.pnpm\[email protected]\node_modules\isexe'

ERROR in ./node_modules/.pnpm/[email protected]/node_modules/micromatch/index.js 84:11-26
Module not found: Error: Can't resolve 'util' in 'D:\_code2\Azak\payload-reproduction\node_modules\.pnpm\[email protected]\node_modules\micromatch'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
        - add a fallback 'resolve.fallback: { "util": require.resolve("util/") }'
        - install 'util'
If you don't want to include a polyfill, you can use an empty module like this:
        resolve.fallback: { "util": false }

webpack compiled with 12 errors

[ericuldall]

+1 Webpack 5 also not playing nice on my builds. I've tried numerous fixes, polyfill libraries, installing fallbacks... Nothing seems to work and I would love to continue using the cloud storage plugin.

Update, My errors:

Module not found: Error: Can't resolve 'request' in '/node_modules/retry-request'

ERROR in ./node_modules/@google-cloud/paginator/build/src/resource-stream.js 19:17-34
Module not found: Error: Can't resolve 'stream' in '/node_modules/@google-cloud/paginator/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/@google-cloud/projectify/build/src/index.js 4:17-34
Module not found: Error: Can't resolve 'stream' in '/node_modules/@google-cloud/projectify/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/@google-cloud/storage/build/src/bucket.js 21:11-24
Module not found: Error: Can't resolve 'fs' in '/node_modules/@google-cloud/storage/build/src'

ERROR in ./node_modules/@google-cloud/storage/build/src/bucket.js 34:17-34
Module not found: Error: Can't resolve 'stream' in '/node_modules/@google-cloud/storage/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/@google-cloud/storage/build/src/crc32c.js 29:13-26
Module not found: Error: Can't resolve 'fs' in '/node_modules/@google-cloud/storage/build/src'

ERROR in ./node_modules/@google-cloud/storage/build/src/file.js 28:11-24
Module not found: Error: Can't resolve 'fs' in '/node_modules/@google-cloud/storage/build/src'

ERROR in ./node_modules/@google-cloud/storage/build/src/file.js 31:17-34
Module not found: Error: Can't resolve 'stream' in '/node_modules/@google-cloud/storage/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/@google-cloud/storage/build/src/file.js 32:13-28
Module not found: Error: Can't resolve 'zlib' in '/node_modules/@google-cloud/storage/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "zlib": require.resolve("browserify-zlib") }'
	- install 'browserify-zlib'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "zlib": false }

ERROR in ./node_modules/@google-cloud/storage/build/src/hash-stream-validator.js 30:17-34
Module not found: Error: Can't resolve 'stream' in '/node_modules/@google-cloud/storage/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/@google-cloud/storage/build/src/nodejs-common/util.js 27:17-34
Module not found: Error: Can't resolve 'stream' in '/node_modules/@google-cloud/storage/build/src/nodejs-common'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/@google-cloud/storage/build/src/resumable-upload.js 28:17-34
Module not found: Error: Can't resolve 'stream' in '/node_modules/@google-cloud/storage/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/@google-cloud/storage/build/src/storage.js 20:17-34
Module not found: Error: Can't resolve 'stream' in '/node_modules/@google-cloud/storage/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/@google-cloud/storage/build/src/transfer-manager.js 28:13-26
Module not found: Error: Can't resolve 'fs' in '/node_modules/@google-cloud/storage/build/src'

ERROR in ./node_modules/@google-cloud/storage/build/src/util.js 17:20-42
Module not found: Error: Can't resolve 'querystring' in '/node_modules/@google-cloud/storage/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "querystring": require.resolve("querystring-es3") }'
	- install 'querystring-es3'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "querystring": false }

ERROR in ./node_modules/@google-cloud/storage/build/src/util.js 18:17-34
Module not found: Error: Can't resolve 'stream' in '/node_modules/@google-cloud/storage/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/debug/src/node.js 5:12-26
Module not found: Error: Can't resolve 'tty' in '/node_modules/debug/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "tty": require.resolve("tty-browserify") }'
	- install 'tty-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "tty": false }

ERROR in ./node_modules/gaxios/build/src/gaxios.js 22:38-60
Module not found: Error: Can't resolve 'querystring' in '/node_modules/gaxios/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "querystring": require.resolve("querystring-es3") }'
	- install 'querystring-es3'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "querystring": false }

ERROR in ./node_modules/gcp-metadata/build/src/gcp-residency.js 19:13-26
Module not found: Error: Can't resolve 'fs' in '/node_modules/gcp-metadata/build/src'

ERROR in ./node_modules/gcp-metadata/build/src/gcp-residency.js 20:13-26
Module not found: Error: Can't resolve 'os' in '/node_modules/gcp-metadata/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "os": require.resolve("os-browserify/browser") }'
	- install 'os-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "os": false }

ERROR in ./node_modules/google-auth-library/build/src/auth/baseexternalclient.js 17:15-32
Module not found: Error: Can't resolve 'stream' in '/node_modules/google-auth-library/build/src/auth'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/google-auth-library/build/src/auth/downscopedclient.js 17:15-32
Module not found: Error: Can't resolve 'stream' in '/node_modules/google-auth-library/build/src/auth'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/google-auth-library/build/src/auth/externalAccountAuthorizedUserClient.js 20:15-32
Module not found: Error: Can't resolve 'stream' in '/node_modules/google-auth-library/build/src/auth'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/google-auth-library/build/src/auth/googleauth.js 17:24-48
Module not found: Error: Can't resolve 'child_process' in '/node_modules/google-auth-library/build/src/auth'

ERROR in ./node_modules/google-auth-library/build/src/auth/googleauth.js 18:11-24
Module not found: Error: Can't resolve 'fs' in '/node_modules/google-auth-library/build/src/auth'

ERROR in ./node_modules/google-auth-library/build/src/auth/googleauth.js 20:11-24
Module not found: Error: Can't resolve 'os' in '/node_modules/google-auth-library/build/src/auth'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "os": require.resolve("os-browserify/browser") }'
	- install 'os-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "os": false }

ERROR in ./node_modules/google-auth-library/build/src/auth/identitypoolclient.js 18:11-24
Module not found: Error: Can't resolve 'fs' in '/node_modules/google-auth-library/build/src/auth'

ERROR in ./node_modules/google-auth-library/build/src/auth/oauth2client.js 18:20-42
Module not found: Error: Can't resolve 'querystring' in '/node_modules/google-auth-library/build/src/auth'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "querystring": require.resolve("querystring-es3") }'
	- install 'querystring-es3'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "querystring": false }

ERROR in ./node_modules/google-auth-library/build/src/auth/oauth2client.js 19:15-32
Module not found: Error: Can't resolve 'stream' in '/node_modules/google-auth-library/build/src/auth'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/google-auth-library/build/src/auth/oauth2common.js 17:20-42
Module not found: Error: Can't resolve 'querystring' in '/node_modules/google-auth-library/build/src/auth'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "querystring": require.resolve("querystring-es3") }'
	- install 'querystring-es3'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "querystring": false }

ERROR in ./node_modules/google-auth-library/build/src/auth/pluggable-auth-handler.js 19:21-45
Module not found: Error: Can't resolve 'child_process' in '/node_modules/google-auth-library/build/src/auth'

ERROR in ./node_modules/google-auth-library/build/src/auth/pluggable-auth-handler.js 20:11-24
Module not found: Error: Can't resolve 'fs' in '/node_modules/google-auth-library/build/src/auth'

ERROR in ./node_modules/google-auth-library/build/src/auth/stscredentials.js 18:20-42
Module not found: Error: Can't resolve 'querystring' in '/node_modules/google-auth-library/build/src/auth'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "querystring": require.resolve("querystring-es3") }'
	- install 'querystring-es3'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "querystring": false }

ERROR in ./node_modules/google-p12-pem/build/src/index.js 10:11-24
Module not found: Error: Can't resolve 'fs' in '/node_modules/google-p12-pem/build/src'

ERROR in ./node_modules/gtoken/build/src/index.js 10:11-24
Module not found: Error: Can't resolve 'fs' in '/node_modules/gtoken/build/src'

ERROR in ./node_modules/http-proxy-agent/dist/agent.js 15:30-44
Module not found: Error: Can't resolve 'net' in '/node_modules/http-proxy-agent/dist'

ERROR in ./node_modules/http-proxy-agent/dist/agent.js 16:30-44
Module not found: Error: Can't resolve 'tls' in '/node_modules/http-proxy-agent/dist'

ERROR in ./node_modules/https-proxy-agent/dist/agent.js 15:30-44
Module not found: Error: Can't resolve 'net' in '/node_modules/https-proxy-agent/dist'

ERROR in ./node_modules/https-proxy-agent/dist/agent.js 16:30-44
Module not found: Error: Can't resolve 'tls' in '/node_modules/https-proxy-agent/dist'

ERROR in ./node_modules/jws/lib/data-stream.js 3:13-30
Module not found: Error: Can't resolve 'stream' in '/node_modules/jws/lib'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/jws/lib/sign-stream.js 5:13-30
Module not found: Error: Can't resolve 'stream' in '/node_modules/jws/lib'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/jws/lib/verify-stream.js 5:13-30
Module not found: Error: Can't resolve 'stream' in '/node_modules/jws/lib'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/readable-stream/lib/internal/streams/stream.js 1:0-34
Module not found: Error: Can't resolve 'stream' in '/node_modules/readable-stream/lib/internal/streams'
Did you mean './stream'?
Requests that should resolve in the current directory need to start with './'.
Requests that start with a name are treated as module requests and resolve within module directories (node_modules, /node_modules, /node_modules/@payloadcms/bundler-webpack/node_modules).
If changing the source code is not an option there is also a resolve options called 'preferRelative' which tries to resolve these kind of requests in the current directory too.

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/readable-stream/readable.js 1:13-30
Module not found: Error: Can't resolve 'stream' in '/node_modules/readable-stream'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/retry-request/index.js 3:22-39
Module not found: Error: Can't resolve 'stream' in '/node_modules/retry-request'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/supports-color/index.js 2:11-24
Module not found: Error: Can't resolve 'os' in '/node_modules/supports-color'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "os": require.resolve("os-browserify/browser") }'
	- install 'os-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "os": false }

ERROR in ./node_modules/teeny-request/build/src/index.js 21:17-34
Module not found: Error: Can't resolve 'stream' in '/node_modules/teeny-request/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/teeny-request/build/src/index.js 68:19-41
Module not found: Error: Can't resolve 'querystring' in '/node_modules/teeny-request/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "querystring": require.resolve("querystring-es3") }'
	- install 'querystring-es3'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "querystring": false }

ERROR in ./node_modules/node-fetch/lib/index.mjs 1:0-28
Module not found: Error: Can't resolve 'stream' in '/node_modules/node-fetch/lib'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
	- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "stream": false }

ERROR in ./node_modules/node-fetch/lib/index.mjs 6:0-24
Module not found: Error: Can't resolve 'zlib' in '/node_modules/node-fetch/lib'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "zlib": require.resolve("browserify-zlib") }'
	- install 'browserify-zlib'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "zlib": false }

ERROR in ./node_modules/node-fetch/lib/index.mjs
Cannot read properties of undefined (reading 'module')

[denolfe]

Reopening and looking into it.

[ericuldall]

Wondering if we can just back port webpack to v4??

[ericuldall]

Another note, I tried moving to viteBundler, that was full of errors, though I'd love to move over to that once I have some more time to troubleshoot that.

[ericuldall]

@denolfe does your fix only address issues with pnpm? I just use npm.

[denolfe]

@haliacmon Just published a new beta to try.

@ericuldall We've identified some issues w/ the vite bundler with this package that are yet to be resolved 👍

[denolfe]

@denolfe does your fix only address issues with pnpm? I just use npm.

I would not expect any issues w/ npm.

[ericuldall]

@denolfe well that's good to know, but I can't even get the vite bundler to work on a blank install :-/

[denolfe]

@ericuldall Sounds like it warrants a separate issue.

[ericuldall]

@ericuldall Sounds like it warrants a separate issue.

Yes, I'll tackle that in the main repo when the time comes. For now i'm trying to move fast and just get my webpack build working with gcs again :D

[haliacmon]

@denolfe Now I get single "Can't resolve 'fs' " error.

I have tried same steps with npm but the result was same. If there is anything I can do to help diagnose, test etc. let me know.

webpack built 4ac35407a2a750e5e806 in 4227ms
ERROR in ./node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected][email protected]/node_modules/@payloadcms/plugin-cloud-storage/dist/adapters/s3/handleUpload.js 136:49-62
Module not found: Error: Can't resolve 'fs' in 'D:\_code2\Azak\payload-reproduction\node_modules\.pnpm\@[email protected]_@[email protected]_@[email protected][email protected]\node_modules\@payloadcms\plugin-cloud-storage\dist\adapters\s3'

webpack compiled with 1 error

[denolfe]

This is quite confusing as we tested your exact repo to make the fix. Do you have a collection with slug my-collection-slug? We had to add one.

[haliacmon]

I'm new to payload cms, just started to discover it. I might misunderstand some terminology. I dont have a slug yet... I think. I just did "npx create-payload-app@latest reproduce-error-s3 -t blank --use-pnpm" and set it up with postgress. then opened localhost:3000 and created admin account. So in the collections I only have "Users" and one user, me/admin. So far zero errors.

but when I tried to confiqure cloud-storage-plugin & S3, I got Can't resolve 'fs' error.

[haliacmon]

I found a a workaround.

webpack: (config) => {
  config.resolve.fallback.fs = false;

  return config;
},

If I add this to buildConfig() error goes away

export default buildConfig({
  admin: {
    user: Users.slug,
    bundler: webpackBundler(),
    webpack: (config) => {
      config.resolve.fallback.fs = false;
      
      // This is to get rid of typescript error
      // if (config.resolve?.fallback)
      // config.resolve.fallback = { ...config.resolve.fallback, fs: false };

      return config;
    },
  },
...

[ericuldall]

I found a a workaround.

webpack: (config) => {
  config.resolve.fallback.fs = false;

  return config;
},

If I add this to buildConfig() error goes away

export default buildConfig({
  admin: {
    user: Users.slug,
    bundler: webpackBundler(),
    webpack: (config) => {
      config.resolve.fallback.fs = false;
      
      // This is to get rid of typescript error
      // if (config.resolve?.fallback)
      // config.resolve.fallback = { ...config.resolve.fallback, fs: false };

      return config;
    },
  },
...

Your mileage may vary by disabling those features. I've been trying similar stuff. something else to note if you want to preserve + override the webpack config you can do it like this:

  webpack: (config) => ({
    ...config,
    resolve: {
      ...config.resolve,
      fallback: {
        ...config.resolve.fallback,
        fs: false
      }
    }    
  }),

[Lismont]

I found a a workaround.

webpack: (config) => {
  config.resolve.fallback.fs = false;

  return config;
},

If I add this to buildConfig() error goes away

export default buildConfig({
  admin: {
    user: Users.slug,
    bundler: webpackBundler(),
    webpack: (config) => {
      config.resolve.fallback.fs = false;
      
      // This is to get rid of typescript error
      // if (config.resolve?.fallback)
      // config.resolve.fallback = { ...config.resolve.fallback, fs: false };

      return config;
    },
  },
...

thanks this helped me (I bumped into the same problem)

[tyteen4a03]

Can confirm I'm also running into this issue.

[IRediTOTO]

I always get this error. This is what stopped me from using PayloadCMS, I didn't get this error when using nextJS. :| I'm trying to customize a plugin to upload images to cloud, I use another library called "publilio_js_sdk". Looking forward to your support I configured webpack like this, it fixed the "fs" not found error

webpack: (config) => {
            return {
                ...config,
                resolve: {
                    ...config.resolve,
                    alias: {
                        ...config.resolve.alias,
                        // publitio_js_sdk: path.resolve(__dirname, "../mock.js"),
                        // "fs-extra": path.resolve(__dirname, "../mock.js"),
                    },
                    fallback: {
                        ...config.resolve.fallback,
                        fs: false,
                        stream: false,
                        constants: false,
                        assert: false,
                        util: false,
                    },
                },
            };
        },

but then I got this error again :|

ERROR (payload): TypeError: dataOrStream.on is not a function

[d717an]

Apologies, you explain it right in your docs:

If the code is included in any way in your config but conditionally disabled in another fashion, you may run into issues such as Webpack Build Error: Can't Resolve 'fs' and 'stream' or similar because the plugin must be run at all times in order to properly extend the webpack config.

Replacing the placeholder 'my-collection-slug' with something that exists like media completely fixed the problem.

---

old comment, no longer applicable:

Using npx create-payload-app@latest with the website demo on MacOS, I was also running into these webpack errors for fs, os, and util (a total of 12 errors, most for fs).

Switching to @payloadcms/plugin-cloud-storage@beta brought this down to a single webpack error: Can't resolve 'fs' in '~/payload-s3/node_modules/@payloadcms/plugin-cloud-storage/dist/adapters/s3.

Adding the fallback object to payload.config.ts allowed me to finally get back into the /admin panel.

  fallback: {
    ...config.resolve.fallback,
    fs: false,
  },

[BarrySaikSoundry]

For anyone landing here - Don't forget to do this: https://payloadcms.com/docs/upload/overview

[huseyinonalcom]

I've come across this issue as well, I've used the node-polyfill-webpack-plugin package to solve it.

payload.config.ts:

...
import { cloudStorage } from "@payloadcms/plugin-cloud-storage";
import { s3Adapter } from "@payloadcms/plugin-cloud-storage/s3";
import NodePolyfillPlugin from "node-polyfill-webpack-plugin";

const s3ad = s3Adapter({
  config: {
    credentials: {
      accessKeyId: process.env.S3_ACCESS_KEY_ID,
      secretAccessKey: process.env.S3_SECRET_ACCESS_KEY,
    },
    region: process.env.S3_REGION,
  },
  bucket: process.env.S3_BUCKET,
});

export default buildConfig({
  admin: {
    user: Users.slug,
    bundler: webpackBundler(),
    webpack: (config) => ({
      ...config,
      plugins: [...config.plugins, new NodePolyfillPlugin()],
    }),
  },
...
  plugins: [
    cloudStorage({
      collections: {
        logos: {
          adapter: s3ad,
        },
        productImages: {
          adapter: s3ad,
        },
      },
    }),
  ],
 ...

here are my dependencies:

{
...
  "dependencies": {
    "@aws-sdk/client-s3": "^3.582.0",
    "@aws-sdk/lib-storage": "^3.582.0",
    "@payloadcms/bundler-webpack": "^1.0.0",
    "@payloadcms/db-postgres": "^0.x",
    "@payloadcms/plugin-cloud": "^3.0.0",
    "@payloadcms/plugin-cloud-storage": "^1.1.2",
    "@payloadcms/richtext-slate": "^1.0.0",
    "aws-crt": "^1.21.2",
    "cross-env": "^7.0.3",
    "dotenv": "^8.2.0",
    "express": "^4.19.2",
    "node-polyfill-webpack-plugin": "^3.0.0",
    "payload": "^2.0.0"
  },
  "devDependencies": {
    "@types/express": "^4.17.9",
    "copyfiles": "^2.4.1",
    "nodemon": "^2.0.6",
    "ts-node": "^9.1.1",
    "typescript": "^4.8.4"
  }
}

here's the upload config of a collection:

...
const Logos: CollectionConfig = {
  slug: "logos",
  upload: {
    disableLocalStorage: true,
    staticURL: "https://hocv1testingcdn.cloudfront.net",
    staticDir: "logos",
    mimeTypes: ["image/*"],
  },
...

I haven't done any extensive testing yet, but the admin panels seems to work perfectly.

[AlessioGr]

Replacing the placeholder 'my-collection-slug' with something that exists like media completely fixed the problem.

@d717an is correct. This issue, as reproduced here, only occurs if there is an invalid plugin-cloud-storage plugin configuration, e.g. an invalid collection slug.

This has now been fixed by https://github.com/payloadcms/payload/pull/6786 which will throw a proper error if the plugin config is invalid! This will now work as expected, as payload is not supposed to start up with this plugin enabled, while its config is invalid. More info in that PR description