Cannot use amplify-js in browser environment (breaking vite / snowpack / esbuild)

[sem4phor]

trafficstars

Describe the bug

The package aws-amplify is meant to be used in the browser, isnt it? Therefore it should not rely on node builtins, which sadly is the case and prevents the usage of this package in many js build tools like rollup, snowpack or vitejs.

The @aws-sdk package which is used by aws-amplify relies on various node build in modules like fs, process or http2, ....

I described the issue also here (https://github.com/vitejs/vite/issues/1374) and Evan You the creator of vuejs and vite mentioned that webpack will drop support for node build ins in version five. Because webpack is used by tools such as create-react-app and vue-cli this will block many developers from using the amplify package soon I think!

To Reproduce Steps to reproduce the behavior:

1. Download https://github.com/sem4phor/aws-cli-bug
2. Run npm run dev (node required)
3. See error in console

Expected behavior The package works in the browser and does not import any node build ins which are not available in browser environment.

Code Snippet https://github.com/sem4phor/aws-cli-bug

Screenshots

[plugin:vite:imports] Failed to resolve import "process". Does the file exist?
/Users/XXXXX/node_modules/@aws-sdk/util-user-agent-node/dist/es/index.js:1:22
1  |  import process from "process";
   |                       ^
2  |  export function defaultUserAgent(packageName, packageVersion) {
3  |      var engine = process.platform + "/" + process.version;
Error: Failed to resolve import "process". Does the file exist?
    at TransformContext.error (C:\Users\XXXXX\node_modules\vite\dist\node\chunks\dep-262f13eb.js:59552:50)
    at TransformContext.transform (C:\XXXXX\node_modules\vite\dist\node\chunks\dep-262f13eb.js:52957:30)
    at async Object.transform (C:\Users\XXXXX\node_modules\vite\dist\node\chunks\dep-262f13eb.js:59742:30)
    at async transformRequest (C:\Users\XXXXX\node_modules\vite\dist\node\chunks\dep-262f13eb.js:65977:29)
    at async C:\Users\XXXXX\node_modules\vite\dist\node\chunks\dep-262f13eb.js:66068:32

What is Configured? Not relevant for this error.

• OS: Windows 10
• Browser: Chrome 87
• aws-amplify 3.3.13

[Jexordexan]

I'm experiencing the same issue, I have had no luck finding a workaround. I am also using vite. Also @sem4phor, in the issue body, you mention @aws-cli but I believe the problem is with @aws-sdk.

[soerenmartius]

I second that. The problem seems to be in @aws-sdkSeems like the library has dependencies on node built-in types which is really bad. Seems like the library was built for webpack and is not usable with vite or rollup at the moment? Also, the coupling increases the size of the package significantly.

Are there any plans to get rid of the node dependencies anytime soon?

[ebisbe]

I have the same issue

[Mootook]

I was also getting this issue when trying to migrate an existing project onto Vite.

Apparently work is being done on the aws-sdk to update this: https://github.com/vitejs/vite/issues/1502#issuecomment-758822680

But I had to follow this to get my app running as Vite is using rollup: https://dev.to/swyx/making-aws-amplify-work-with-rollup-2d9m

[ebisbe]

But I had to follow this to get my app running as Vite is using rollup: https://dev.to/swyx/making-aws-amplify-work-with-rollup-2d9m

@Mootook Did that link work for you? I tried and couldn't make it work. I'm new to Rollup / Vite so I wasn't even sure it would work. I'll try again if that worked then.

[mbp]

But I had to follow this to get my app running as Vite is using rollup: https://dev.to/swyx/making-aws-amplify-work-with-rollup-2d9m

@Mootook Did that link work for you? I tried and couldn't make it work. I'm new to Rollup / Vite so I wasn't even sure it would work. I'll try again if that worked then.

I have the same issue (and also new to Rollup / Vite), so I am not sure how you would make the changes in e.g. vite.config.ts file where the syntax is different. Let me know if you make it work!

[sem4phor]

@Mootook It would be great if you could provide a repository with your suggested workaround :)

[ebisbe]

From what I have read the plugin interface of Vite is the same as Rollup so they should be compatible. What I don't understand is that in https://vite-rollup-plugins.patak.dev you can see that Json and node-resolve are already covered on Vite.

[Mootook]

@ebisbe, I didn't know that, good catch.

I have uploaded a small demo https://github.com/Mootook/aws-vite-demo/tree/main

But you're right, the plugin extension in vite.config.js is not necessary. Commenting it out, removing them as dependencies, and rebuilding seems to have no effect.

The only other issue I ran into was a global is not defined error, which has a solution in the index.html file.

[sammartinez]

@sem4phor Are you using Amplify JavaScript or using AWS SDK? I believe your issue is related to the AWS SDK instead of Amplify JS. Can you confirm?

[ebisbe]

@sammartinez Doesn't Aws-amplify use Aws-sdk? I use Aws-amplify but my errors come from node_modules/@aws-sdk and I have not installed in the package.json

[ebisbe]

@ebisbe, I didn't know that, good catch.

I have uploaded a small demo https://github.com/Mootook/aws-vite-demo/tree/main

But you're right, the plugin extension in vite.config.js is not necessary. Commenting it out, removing them as dependencies, and rebuilding seems to have no effect.

The only other issue I ran into was a global is not defined error, which has a solution in the index.html file.

Your code is working indeed. But when I add your vite.config.js it still fails in mine... I've been trying to sample the minimum to make it work on my project but it doesn't. Besides what you have I'm using Tailwind

[sammartinez]

@ebisbe correct, we use AWS SDK but it is a dependency and separate team and repo that handles errors related to it.

[Mootook]

@ebisbe, I updated my repo to use tailwind and it's working.

What errors are you getting? Is it still failing on the amplify dependency?

there is also a vite specific discord for general troubleshooting https://discord.gg/y44Ad4hF

[mbp]

@Mootook correct me if I am wrong, but I don't actually see you using aws-amplify in your repo (other than the package.json dependency)

If you insert the following code into main.js:

import Amplify, { Auth } from 'aws-amplify';

Amplify.configure({});

Then npm run build will start failing with

'request' is not exported by __vite-browser-external, imported by node_modules\@aws-sdk\credential-provider-imds\dist\es\remoteProvider\httpRequest.js

This is the error message originally reported in the chain of issues: https://github.com/vitejs/vite/issues/1502 => https://github.com/vitejs/vite/issues/1374#issuecomment-754820938 => this

[Mootook]

@mbp Ah! Yes, you're correct. Apologies, I just went off the original steps to reproduce, which just included npm run dev (which does work, even with Amplify.configure()...odd), but the error is present in npm run build.

So credential provider is expecting the node http global which is not bundled with rollup. There is the 'rollup-plugin-node-builtins' package, but it seems outdated and sort of undercuts what Vite is striving for.

[ebisbe]

@ebisbe, I updated my repo to use tailwind and it's working.

What errors are you getting? Is it still failing on the amplify dependency?

there is also a vite specific discord for general troubleshooting https://discord.gg/y44Ad4hF

It's funny because when I moved my code to your repo it's all working but in mine it still failing. I've forked your repo with my code added in https://github.com/ebisbe/aws-vite-demo and my failing code in here https://github.com/ebisbe/skedr-prototype/tree/live

The error I get is:

⌊☻ yarn dev
yarn run v1.22.10
$ vite --open
Pre-bundling dependencies:
  vue
  vue-router
  aws-amplify
  luxon
  lodash/groupBy
  (...and 1 more)
(this will be run only when your dependencies or config have changed)
 > node_modules/@aws-sdk/credential-provider-imds/dist/es/remoteProvider/httpRequest.js: error: No matching export for import "request"
    4 │ import { request } from "http";
      ╵          ~~~~~~~

 > node_modules/@aws-sdk/shared-ini-file-loader/dist/es/index.js: error: No matching export for import "readFile"
    2 │ import { readFile } from "fs";
      ╵          ~~~~~~~~

 > node_modules/@aws-sdk/shared-ini-file-loader/dist/es/index.js: error: No matching export for import "homedir"
    3 │ import { homedir } from "os";
      ╵          ~~~~~~~

 > node_modules/@aws-sdk/shared-ini-file-loader/dist/es/index.js: error: No matching export for import "join"
    4 │ import { join, sep } from "path";
      ╵          ~~~~

 > node_modules/@aws-sdk/shared-ini-file-loader/dist/es/index.js: error: No matching export for import "sep"
    4 │ import { join, sep } from "path";
      ╵                ~~~

 > node_modules/@aws-sdk/credential-provider-process/dist/es/index.js: error: No matching export for import "exec"
    4 │ import { exec } from "child_process";
      ╵          ~~~~

 > node_modules/@aws-sdk/node-http-handler/dist/es/write-request-body.js: error: No matching export for import "Readable"
    1 │ import { Readable } from "stream";
      ╵          ~~~~~~~~

 > node_modules/@aws-sdk/node-http-handler/dist/es/node-http-handler.js: error: No matching export for import "Agent"
    3 │ import { Agent as hAgent, request as hRequest } from "http";
      ╵          ~~~~~

 > node_modules/@aws-sdk/node-http-handler/dist/es/node-http-handler.js: error: No matching export for import "request"
    3 │ import { Agent as hAgent, request as hRequest } from "http";
      ╵                           ~~~~~~~

 > node_modules/@aws-sdk/node-http-handler/dist/es/node-http-handler.js: error: No matching export for import "Agent"
    4 │ import { Agent as hsAgent, request as hsRequest } from "https";
      ╵          ~~~~~

 > node_modules/@aws-sdk/node-http-handler/dist/es/node-http-handler.js: error: No matching export for import "request"
    4 │ import { Agent as hsAgent, request as hsRequest } from "https";
      ╵                            ~~~~~~~

 > node_modules/@aws-sdk/node-http-handler/dist/es/node-http2-handler.js: error: No matching export for import "connect"
    4 │ import { connect, constants } from "http2";
      ╵          ~~~~~~~

 > node_modules/@aws-sdk/node-http-handler/dist/es/node-http2-handler.js: error: No matching export for import "constants"
    4 │ import { connect, constants } from "http2";
      ╵                   ~~~~~~~~~

 > node_modules/@aws-sdk/node-http-handler/dist/es/stream-collector/collector.js: error: No matching export for import "Writable"
    2 │ import { Writable } from "stream";
      ╵          ~~~~~~~~

 > node_modules/@aws-sdk/util-body-length-node/dist/es/index.js: error: No matching export for import "lstatSync"
    1 │ import { lstatSync } from "fs";
      ╵          ~~~~~~~~~

 > node_modules/@aws-sdk/eventstream-serde-node/dist/es/EventStreamMarshaller.js: error: No matching export for import "Readable"
    4 │ import { Readable } from "stream";
      ╵          ~~~~~~~~

 > node_modules/@aws-sdk/hash-stream-node/dist/es/hash-calculator.js: error: No matching export for import "Writable"
    2 │ import { Writable } from "stream";
      ╵          ~~~~~~~~

 > node_modules/@aws-sdk/hash-stream-node/dist/es/index.js: error: No matching export for import "createReadStream"
    1 │ import { createReadStream } from "fs";
      ╵          ~~~~~~~~~~~~~~~~

error when starting dev server:
Error: Build failed with 18 errors:
node_modules/@aws-sdk/credential-provider-imds/dist/es/remoteProvider/httpRequest.js:4:9: error: No matching export for import "request"
node_modules/@aws-sdk/credential-provider-process/dist/es/index.js:4:9: error: No matching export for import "exec"
node_modules/@aws-sdk/eventstream-serde-node/dist/es/EventStreamMarshaller.js:4:9: error: No matching export for import "Readable"
node_modules/@aws-sdk/hash-stream-node/dist/es/hash-calculator.js:2:9: error: No matching export for import "Writable"
node_modules/@aws-sdk/hash-stream-node/dist/es/index.js:1:9: error: No matching export for import "createReadStream"
...
    at failureErrorWithLog (/Volumes/Data/Development/skedr/next_prototype/node_modules/esbuild/lib/main.js:1124:15)
    at buildResponseToResult (/Volumes/Data/Development/skedr/next_prototype/node_modules/esbuild/lib/main.js:879:32)
    at /Volumes/Data/Development/skedr/next_prototype/node_modules/esbuild/lib/main.js:974:20
    at /Volumes/Data/Development/skedr/next_prototype/node_modules/esbuild/lib/main.js:541:9
    at handleIncomingPacket (/Volumes/Data/Development/skedr/next_prototype/node_modules/esbuild/lib/main.js:624:9)
    at Socket.readFromStdout (/Volumes/Data/Development/skedr/next_prototype/node_modules/esbuild/lib/main.js:508:7)
    at Socket.emit (events.js:314:20)
    at addChunk (_stream_readable.js:298:12)
    at readableAddChunk (_stream_readable.js:273:9)
    at Socket.Readable.push (_stream_readable.js:214:10)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[sem4phor]

Are you using Amplify JavaScript or using AWS SDK? I believe your issue is related to the AWS SDK instead of Amplify JS. Can you confirm?

@sammartinez I am using amplify which has aws-sdk as it's dependency (see reproduction repository)

The vue label is kinda wrong. It is an issue with js bundlers which use native ecmascript modules. Vite is not vue-only.

[sammartinez]

@sem4phor I guess I am a little confused and will need more information from you. What is your use case you are looking to do ? In looking at the code sample provided, thank you by the way it was helpful, I am not seeing a direct dependency of aws-amplify or an aws-exports file for testing the configuration, just an empty object passed. To some of the errors called out in the stack traces above, we have done some clean up on the library recently to remove unused dependencies. Can you look at testing the latest version of aws-amplify with your sample?

[madmod]

I too am having this issue. I have imported @aws-sdk/client-cognito-identity and @aws-sdk/credential-provider-cognito-identity directly. I am using Vite and not using Amplify fyi. Hopefully that helps you track down the root cause.

[madmod]

I have determined that the following modules cause build errors related to expecting various node js modules in the browser such as http and fs.

        '@aws-sdk/credential-provider-imds',
        '@aws-sdk/credential-provider-process',
        '@aws-sdk/shared-ini-file-loader',
        '@aws-sdk/eventstream-serde-node',
        '@aws-sdk/hash-node',
        '@aws-sdk/node-http-handler',
        '@aws-sdk/util-body-length-node',
        '@aws-sdk/util-user-agent-node',

[bhvngt]

I am facing the same issue while using vite.

I am getting the previously reported error 'request' is not exported by __vite-browser-external, imported by node_modules\@aws-sdk\credential-provider-imds\dist\es\remoteProvider\httpRequest.js. Is this something that is fixed? I have tried this with the latest version of aws-amplify 3.3.22.

I tried with rollup config as suggested earlier in this thread. I also experimented with rollup-plugin-polyfill-node. Both of them fails to fix the issue.

Here's my vite.config.js

import svelte from "rollup-plugin-svelte-hot";
import autoPreprocess from "svelte-preprocess";
import nodeResolve from "@rollup/plugin-node-resolve";
import json from "@rollup/plugin-json";

export default {
    alias: {
        "@": resolve(__dirname, "src"),
    },
    root: "src",
    plugins: [
        svelte({
            preprocess: autoPreprocess(),
            dev: true,
            hot: true,
        }),
    ],
    build: {
        rollupOptions: {
            plugins: [
                nodeResolve({
                    browser: true,
                    preferBuiltins: false,
                }),
                json(),
            ],
        },
    },
};

Error dump

'request' is not exported by __vite-browser-external, imported by node_modules/.pnpm/@aws-sdk/[email protected]/node_modules/@aws-sdk/credential-provider-imds/dist/es/remoteProvider/httpRequest.js
file: /temp/tst/node_modules/.pnpm/@aws-sdk/[email protected]/node_modules/@aws-sdk/credential-provider-imds/dist/es/remoteProvider/httpRequest.js:4:9
2: import { ProviderError } from "@aws-sdk/property-provider";
3: import { Buffer } from "buffer";
4: import { request } from "http";
            ^
5: /**
6:  * @internal
error during build:
Error: 'request' is not exported by __vite-browser-external, imported by node_modules/.pnpm/@aws-sdk/[email protected]/node_modules/@aws-sdk/credential-provider-imds/dist/es/remoteProvider/httpRequest.js
    at error (/temp/tst/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/shared/rollup.js:5279:30)
    at Module.error (/temp/tst/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/shared/rollup.js:9996:16)
    at Module.traceVariable (/temp/tst/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/shared/rollup.js:10389:29)
    at ModuleScope.findVariable (/temp/tst/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/shared/rollup.js:8847:39)
    at FunctionScope.findVariable (/temp/tst/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/shared/rollup.js:2641:38)
    at ChildScope.findVariable (/temp/tst/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/shared/rollup.js:2641:38)
    at FunctionScope.findVariable (/temp/tst/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/shared/rollup.js:2641:38)
    at ChildScope.findVariable (/temp/tst/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/shared/rollup.js:2641:38)
    at Identifier.bind (/temp/tst/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/shared/rollup.js:4017:40)
    at CallExpression.bind (/temp/tst/node_modules/.pnpm/[email protected]/node_modules/rollup/dist/shared/rollup.js:2728:23)

[jarrodu]

I am so glad I found this issue since I was about to write my own.

I am also having this issue.

[vovapyc]

Hi @bhavinkamani-gt, have you manage to fix it?

[jarrodu]

I noticed that the "to-be-reproduced" tag was added to this issue. I am willing to spend a bit of time setting up a minimal example of the issue I am having. Is there a protocol for submitting an example or is a temporary public git repo with a README enough?

[bhvngt]

@vovawed No, I haven't yet found the solution.. As a workaround, I am using snowpack to do my production builds. Snowpack has a built-in support for node-polyfills with snowpack build --polyfill-node.

As commented here. This should have been fixed in this library by isolating node related code. However, it doesn't seem to have happen.

@sammartinez Would you have any update on this issue?

[sem4phor]

I think to be reprouced relates to a maintainer who needs to reproduce this. I have already submitted a reproduction repository...

Moreover the label Amplify UI Components is wrong, too IMO. The issue persists without importing ui components @sammartinez

[sammartinez]

Thanks for this feedback, I have removed the label. I will have my oncall look into this tomorrow.

[jarrodu]

When using Vite and the authentication documentation here: https://docs.amplify.aws/ui/auth/authenticator/q/framework/vue

I hit the same nodejs issues this issue describes.

So with this,

 "dependencies": {
    "@aws-amplify/ui-components": "^1.1.0",
    "aws-amplify": "^3.3.24",
    "vue": "^3.0.5"
  }

I get this,

'request' is not exported by __vite-browser-external,

[asmajlovicmars]

Having the same issue with SvelteKit and Amplify, and not including aws-sdk. Just Amplify js, but aws-sdk is called implicitly, as a dependency I guess...

request' is not exported by __vite-browser-external, imported by node_modules/@aws-sdk/credential-provider-imds/dist/es/remoteProvider/httpRequest.js file: /Users/asmajlovic/code/sveltekit/p1/node_modules/@aws-sdk/credential-provider-imds/dist/es/remoteProvider/httpRequest.js:4:9 2: import { ProviderError } from "@aws-sdk/property-provider"; 3: import { Buffer } from "buffer"; 4: import { request } from "http"; ^ 5: /** 6: * @internal

'request' is not exported by __vite-browser-external, imported by node_modules/@aws-sdk/credential-provider-imds/dist/es/remoteProvider/httpRequest.js