aws-sdk-js
aws-sdk-js copied to clipboard
aws
Publish native es modules
Build steps for dev servers don't have to be a thing anymore. With native modules and relative path module specifiers, we could just run the code in the browser as-is.
import CognitoIdentityServiceProvider from '../../aws-sdk/es/clients/cognitoidentityserviceprovider';
Having native es modules in this library would really be swell.
Thanks for your consideration.
Thank you for your advice. It's very nice and intuitive to have this feature. However, this may need a major version update. 😉
Pardon my ignorance of the internal workings of your library, but it seems to me that you could release a minor version that doesn't break backwards, only adds an es-module version in /dist/es/
The semantics of ES modules are different enough from script-based module formats like CommonJS, UMD, and AMD that I don't think we could automate a conversion of the SDK from its current format to an ESM-based format. Since we release several times a week, if the conversion cannot be automated, then I'm not sure it's a feasible solution.
The conversion from ESM to CommonJS, however, is fairly straightforward, so if the SDK were written following ES module semantics, we could release both versions. That, however, would require a major version bump.
Got it. Well I hope it comes soon 😉
On Mon, Oct 23, 2017, 8:40 PM Jonathan Eskew [email protected] wrote:
The semantics of ES modules are different enough from script-based module formats like CommonJS, UMD, and AMD that I don't think we could automate a conversion of the SDK from its current format to an ESM-based format. Since we release several times a week, if the conversion cannot be automated, then I'm not sure it's a feasible solution.
The conversion from ESM to CommonJS, however, is fairly straightforward, so if the SDK were written following ES module semantics, we could release both versions. That, however, would require a major version bump.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aws/aws-sdk-js/issues/1766#issuecomment-338739157, or mute the thread https://github.com/notifications/unsubscribe-auth/ABZgNKUmqdtvcP5FIC0aamq_pA5UUXXIks5svM-qgaJpZM4QAegK .
There is one tool I am aware of that takes CommonJS modules as input and outputs ES modules, which is exactly what you need to automate the conversion.
https://github.com/rollup/rollup-plugin-commonjs
That Rollup plugin is intended to be used as part of a build process to import CommonJS dependencies, but I suspect you could hack it to suit your needs relatively easily.
By the way, I disagree that offering a new type of build would demand a major version bump. It's a new feature and should not impact existing users whatsoever.
On that note, I'm experimenting with using that plugin to get the ball rolling, and actually successfully bundled using:
import sourcemaps from 'rollup-plugin-sourcemaps';
import resolve from 'rollup-plugin-node-resolve';
import builtins from 'rollup-plugin-node-builtins';
import globals from 'rollup-plugin-node-globals';
import commonjs from 'rollup-plugin-commonjs';
import json from 'rollup-plugin-json';
export default {
input: 'src/eve-redux/eve-redux-store.js',
output: {
file: 'src/bundle.js',
name: 'EVE',
format: 'iife',
},
watch: {
include: 'src/**',
},
plugins: [
resolve(),
commonjs({
include: 'node_modules/**',
ignoreGlobal: true,
namedExports: {
'node_modules/inferno-redux/index.js': ['Provider'],
'node_modules/aws-sdk/global.js': ['util'],
},
}),
sourcemaps(),
json(),
builtins(),
globals(),
],
};
Are you able to trick Rollup into converting the SDK to ES modules by doing...
format: 'es' // instead of 'iife'
?
That might actually work! It would still be bundled, though, presumably. So I guess we'd have to then figure out a way to de-bundle it, in order to get source code to start work on a PR.
Well, rollup doesn't give any errors, although browser complains Uncaught TypeError: Failed to resolve module specifier 'aws-sdk'
bundle.js:1
import { S3 } from 'aws-sdk';
The AWS JS SDK having a native ESM format is especially useful for those of us who are starting to migrate away from "mandatory build phase" web development while still using "modern" libraries like React and HTM. In particular, the forward-thinking Pika tooling says that aws-sdk isn't compatible.
@jeskew After re-reading what you wrote here:
The conversion from ESM to CommonJS, however, is fairly straightforward, so if the SDK were written following ES module semantics, we could release both versions. That, however, would require a major version bump.
It seems like this would be a reasonable plan moving towards this goal:
- Rewrite all require() calls as import statements throughout the codebase
- Create a build script that converts the newly ESM codebase to UMD for backwards compatibility
- Include the raw ESM codebase in new releases so that ESM-based tools can consume it.
This would involve some planning, to ensure that the code behaves the same as before, the execution of this plan happens during the course of a single day, and preparation to create or modify the relevant build tools ahead of time.
But this issue has been open for almost 2 years. I know it's probably not high priority for AWS but ES Modules are already here, the web is ready for this, and making this issue a reality instead of a dream will help projects that use ESM tooling to move forward more easily.
This would be pretty great especially for usage with deno. I believe the change could be scripted using a tool like jscodeshift would be happy to contribute if it is something that would be considered :+1:
If the data helps: We're seeing more people having trouble using aws-sdk with Snowpack, Rollup, and ESM in general. Shipping native es modules would go a long way in unblocking these users!
I'm also interested in using this in Deno, though that would require more than simply using ESmodules syntax; imports would need to be re-written to include file extensions as well (which also means re-writing some imports to import /index.js rather than the directory name).
Any ETA available yet on this issue?
Are there any help wanted tickets as first steps? I would like to contribute towards this goal 😃
Now that AWS Lambda supports NodeJS 14 and also loads native JS modules (as long as "type": "module" has been set within an additionally uploaded package.json), it would be great to do something like this
import * as AWS from 'aws-sdk';
That this is not possible is in fact the only reason why we have been (automatically) downgrading our entire backend to the require syntax for years 😅 (and also the reason why we have to include and maintain a build system like Rollup at all).
@sebiniemann if you're using "type": "module", you can actually do that already. You probably want to remove the * as part, though.
Unfortunately, since the AWS SDK is still CommonJS, there are no named exports, so doing a wildcard import means you have to use AWS.default everywhere. If you instead do import AWS from ..., then you can just use AWS.
Thanks for the feedback :)
We just tried this (including without * as), but the pre-installed SDK on AWS Lambda seems not to be compatible (or we are still missing something):
"Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /var/task/index.js", "require() of ES modules is not supported", "trace": [ "Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /var/task/index.js", "require() of ES modules is not supported.", "require() of /var/task/index.js from /var/runtime/UserFunction.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.", "Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /var/task/package.json.", "", " at Object.Module._extensions..js (internal/modules/cjs/loader.js:1080:13)", " at Module.load (internal/modules/cjs/loader.js:928:32)", " at Function.Module._load (internal/modules/cjs/loader.js:769:14)", " at Module.require (internal/modules/cjs/loader.js:952:19)", " at require (internal/modules/cjs/helpers.js:88:18)", " at _tryRequire (/var/runtime/UserFunction.js:75:12)", " at _loadUserApp (/var/runtime/UserFunction.js:95:12)", " at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)", " at Object.
(/var/runtime/index.js:43:30)", " at Module._compile (internal/modules/cjs/loader.js:1063:30)" ]
We created a basic NodeJS 14 Lambda (no layers or additional configuration) with the following minimal code (besides the package.json for module support):
import AWS from 'aws-sdk';
async function handler() {
return {
statusCode: 200,
body: JSON.stringify('Hello from Lambda!'),
};
};
export {
handler,
};
// EDIT
We just tried it without import AWS from 'aws-sdk'; and got the same error. It seems as if the AWS Lambda NodeJS 14 environment itself is not ready for ES6 imports.
That error implies that something is transpiling your code or at least converting your import statements to require() calls. Make sure you get rid of any bundlers or other tooling that might be doing that. You might be right that Lambda just isn't ready for this yet, but it's more likely something else.
The code was just pasted into the AWS Lambda editor. No build process involved (at least from our side).
After consulting with AWS support, it was confirmed that AWS Lambda is currently not compatible with "type": "module" and therefore not ready for ES6 imports.
It would be possible to be compatible with both CommonJS and ECMAScript Modules by adding another package.json that does not contain "type": "module", explicitly indicating that CommonJS is used within AWS' own Lambda runtime scripts. As far as I could see, this would need to be placed outside of /var/task (in /var, based on the error message). However, it seems that this is unfortunately not possible for the user.
@sebiniemann https://aws.amazon.com/about-aws/whats-new/2022/01/aws-lambda-es-modules-top-level-await-node-js-14/?nc1=h_ls
It would be possible to be compatible with both CommonJS and ECMAScript Modules by adding another package.json that does not contain "type": "module",
This is we did with package.json w/ "type": "module", and it seems to work for S3 if nothing else:
import * as awscjs from 'aws-sdk';
const AWS = awscjs.default;
const S3 = new AWS.S3(...
Unsure if there are any pitfalls here.
@ignoramous you can simplify your code to this:
import AWS from 'aws-sdk';
const S3 = new AWS.S3();
That works ^^
The thing that's unfortunately not currently supported is named imports:
import { S3 } from 'aws-sdk';
const S3 = new S3();
Please add ESM support. Lack of ESM support is a major stumbling block in using this library, particularly with vite. If you have to do a major version upgrade, please do so.
I'm getting the following error when attempting to use a dynamic import to support isomorphic javascript using vite & solid-js:
at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]/node_modules/aws-sdk/lib/aws.js:1:14
at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
at eval (/src/_esg/urth__ctx_.ts:30:15)
Trace: ReferenceError: require is not defined
at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]/node_modules/aws-sdk/lib/aws.js:1:14
at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
at eval (/src/_esg/mi_eco_20221130__ctx_.ts:34:15)
An unhandled error occured: ReferenceError: require is not defined
at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]/node_modules/aws-sdk/lib/aws.js:1:14
at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
An unhandled error occured: ReferenceError: require is not defined
at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]/node_modules/aws-sdk/lib/aws.js:1:14
at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
An unhandled error occured: ReferenceError: require is not defined
at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]/node_modules/aws-sdk/lib/aws.js:1:14
at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
An unhandled error occured: ReferenceError: require is not defined
at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]/node_modules/aws-sdk/lib/aws.js:1:14
at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15)
An unhandled error occured: Error: Timed out after 10000ms.
at promise_timeout (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/@[email protected]/node_modules/@ctx-core/function/src/promise_timeout/index.js:1:60)
at Proxy.subscribe_wait (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/@[email protected]/node_modules/@ctx-core/nanostores/src/subscribe_wait/index.js:15:30)
at esg_mi_eco__preload (/home/brian/work/portfoliome/censible-core/apps/mi-web/src/_http/preload.ts:19:2)
at async Object.handle (/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]/node_modules/itty-router/dist/itty-router.js:1:658)
at async handleRequest (/home/brian/work/portfoliome/censible-core/apps/mi-web/src/entry-server.tsx:49:8)
at async eval (/home/brian/work/portfoliome/censible-core/vendor/solid-start/packages/start/entry-server/StartServer.tsx:45:11)
at async devFetch (file:///home/brian/work/portfoliome/censible-core/vendor/solid-start/packages/start/dev/server.js:46:12)
at async startHandler (file:///home/brian/work/portfoliome/censible-core/vendor/solid-start/packages/start/dev/server.js:115:20)
Using createRequire renders it unable to inline the library to the lambda deployment using vite. Unless there is another solution, this will potentially require major reengineering. Perhaps there are example repos out there with solutions to these problems.
Please add ESM support. Lack of ESM support is a major stumbling block in using this library, particularly with vite. If you have to do a major version upgrade, please do so.
I'm getting the following error when attempting to use a dynamic import to support isomorphic javascript using vite & solid-js:
at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]/node_modules/aws-sdk/lib/aws.js:1:14 at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15) at eval (/src/_esg/urth__ctx_.ts:30:15) Trace: ReferenceError: require is not defined at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]/node_modules/aws-sdk/lib/aws.js:1:14 at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15) at eval (/src/_esg/mi_eco_20221130__ctx_.ts:34:15) An unhandled error occured: ReferenceError: require is not defined at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]/node_modules/aws-sdk/lib/aws.js:1:14 at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15) An unhandled error occured: ReferenceError: require is not defined at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]/node_modules/aws-sdk/lib/aws.js:1:14 at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15) An unhandled error occured: ReferenceError: require is not defined at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]/node_modules/aws-sdk/lib/aws.js:1:14 at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15) An unhandled error occured: ReferenceError: require is not defined at /@fs/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]/node_modules/aws-sdk/lib/aws.js:1:14 at instantiateModule (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]_qfz55zahqkp66vn23sxaaw3yfe/node_modules/vite/dist/node/chunks/dep-5e7f419b.js:52224:15) An unhandled error occured: Error: Timed out after 10000ms. at promise_timeout (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/@[email protected]/node_modules/@ctx-core/function/src/promise_timeout/index.js:1:60) at Proxy.subscribe_wait (file:///home/brian/work/portfoliome/censible-core/node_modules/.pnpm/@[email protected]/node_modules/@ctx-core/nanostores/src/subscribe_wait/index.js:15:30) at esg_mi_eco__preload (/home/brian/work/portfoliome/censible-core/apps/mi-web/src/_http/preload.ts:19:2) at async Object.handle (/home/brian/work/portfoliome/censible-core/node_modules/.pnpm/[email protected]/node_modules/itty-router/dist/itty-router.js:1:658) at async handleRequest (/home/brian/work/portfoliome/censible-core/apps/mi-web/src/entry-server.tsx:49:8) at async eval (/home/brian/work/portfoliome/censible-core/vendor/solid-start/packages/start/entry-server/StartServer.tsx:45:11) at async devFetch (file:///home/brian/work/portfoliome/censible-core/vendor/solid-start/packages/start/dev/server.js:46:12) at async startHandler (file:///home/brian/work/portfoliome/censible-core/vendor/solid-start/packages/start/dev/server.js:115:20)Using
createRequirerenders it unable to inline the library to the lambda deployment using vite. Unless there is another solution, this will potentially require major reengineering. Perhaps there are example repos out there with solutions to these problems.
+1 to this comment - does AWS not want this project to be widely adopted? this is a killer for Vite users....
Hello, the v3 of the SDK has native ES modules. We have no plans to implement this in v2.
