Enable consuming of ES modules in extensions

[igorskyflyer]

When developing extensions and using either JavaScript or TypeScript we are unable to consume ES modules, only somewhat legacy CommonJS modules, setting the type to module and rewriting the extension to use import instead of require breaks the extension, generating an exception that states that all modules should use import instead of require in internal VS Code JavaScript files, I conclude it's caused by the type: module that forces Node to treat all .js files as ES modules. Tried using TypeScript which transpiles its own syntax to CommonJS module - so that's a no, I have also tried using just .mjs extension, again the same issue.

What is the status of this issue and are there plans to enable using of ES modules in extension development? That (could) bring somewhat big performance gains when bundling extensions with, for example, esbuild because it would enable tree-shaking - dead code removal, thus loading only necessary code. But I think this is not an extension API only issue, right? This needs to be done for VS Code itself?

[vscodebot[bot]]

(Experimental duplicate detection) Thanks for submitting this issue. Please also check if it is already covered by an existing one, like:

• Intellisense vs JavaScript Modules - lack of ".js" in completion (#126171)

[igorskyflyer]

Closing after 15 days of no reply.

[alexdima]

cc @jrieken

[vscode-triage-bot]

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.

Happy Coding!

[Lemmingh]

Sounds like a duplicate of #116056. Is it possible to reopen that issue?

---

The VS Code extension host currently only accepts CJS module, as shown in the (trimmed) error message below:

Activating extension failed

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module
require() of ES modules is not supported.

Instead change the requiring code to use import()

at internal/modules/cjs/loader.js:823:14

at require (internal/modules/cjs/helpers.js:88:18)
at Function.t [as __$__nodeRequire] (c:\Program Files\Microsoft VS Code\resources\app\out\vs\loader.js:5:101)
at v._loadCommonJSModule
at v._doActivateExtension
at v._activateExtension (c:\Program Files\Microsoft VS Code\resources\app\out\vs\workbench\services\extensions\node\extensionHostProcess.js:99:11695)

However, I think ES module will be more convenient in future.

Besides, TypeScript can transpile dynamic import() in CJS module in a surprising way (https://github.com/microsoft/TypeScript/issues/43329), while it's never a problem in ES module where import() is emitted as is. VS Code extension authors can benefit from building extension as ES module.

[andyleejordan]

For what it's worth, I ran into this trying to update node-fetch from 2.x to 3.x, since the newer version is ESM only. Looks to be the direction things are going, so extension developers are only more likely in the future to run into issues due to lack of ESM support.

[andyleejordan]

FYI @TylerLeonhardt one of the Code issues I'd love to see fixed 😃

[Lemmingh]

TypeScript 4.5 will perhaps have a new module option called node12, which preserves import() in CJS module. Then, although your entry point still have to be a CJS module now, you can load ES modules internally in an asynchronous manner.

See

• https://devblogs.microsoft.com/typescript/announcing-typescript-4-5-beta/

• https://github.com/microsoft/TypeScript/issues/45418#issuecomment-953399609

• The links in my comment

[andyleejordan]

Excuse me, @TylerLeonhardt, do you know why my comment above was marked as spam? I have to assume that was a mistake.

[alexdima]

@andschwa I'm sorry, it might have been me that marked the comment as spam, but I don't remember doing it. I personally tend to hide comments that do not bring any value to the underlying discussion or distract from it.

[jasonwilliams]

TypeScript 4.5 will perhaps have a new module option called node12, which preserves import() in CJS module. Then, although your entry point still have to be a CJS module now, you can load ES modules internally in an asynchronous manner.

See

• https://devblogs.microsoft.com/typescript/announcing-typescript-4-5-beta/
• TypeScript 4.5 Iteration Plan TypeScript#45418 (comment)
• The links in my comment

This has now been done in https://github.com/microsoft/TypeScript/pull/45884 Looks like the remaining issue to watch is https://github.com/microsoft/TypeScript/issues/46452 (assuming VSCode is waiting for this to happen before carrying on). Seems like they're still aiming for a 4.6 release

[Silic0nS0ldier]

Conversation here seems to have drifted off topic. While node12 module resolution logic in TypeScript will help with extension authoring (and directly consuming ES module packages in VSCode), it is not required to support ES modules. Extensions are regular JavaScript, so all that needs to happen is for dynamic import support to be introduced.

I aim to author ESM wherever I can (easier to share code across the ecosystem) so I had a look at the VSCode source to see if I can push this forward.

Best I can tell, there are 2 environments we care about (within the scope of this issue, web extensions have additional limitations so I'm ignoring it for now). They can run in the main thread (which is hopefully rare) or in the extension host.

Extension Host

What i've managed to gleam from VSCode source is that the extension host is started with the worker_threads API, which potentially could be imposing a dynamic require restriction (I'll test this in NodeJS to clarify).

https://github.com/microsoft/vscode/blob/4a1a8b07ff093e7dcb8067c185eec49502b56aa5/src/vs/platform/extensions/electron-main/workerMainProcessExtensionHostStarter.ts#L12 https://github.com/microsoft/vscode/blob/4a1a8b07ff093e7dcb8067c185eec49502b56aa5/src/vs/platform/extensions/electron-main/workerMainProcessExtensionHostStarter.ts#L22-L24

Extensions hosted here are imported with require (or rather what I believe is a webpack escaped version). I've included more of the call chain in the snippets below to help with context.

https://github.com/microsoft/vscode/blob/9fd6ee7095e76a304556c06bbef16fc4f05cf697/src/vs/workbench/api/common/extHostExtensionService.ts#L171-L172

https://github.com/microsoft/vscode/blob/9fd6ee7095e76a304556c06bbef16fc4f05cf697/src/vs/workbench/api/common/extHostExtensionService.ts#L394

https://github.com/microsoft/vscode/blob/9fd6ee7095e76a304556c06bbef16fc4f05cf697/src/vs/workbench/api/node/extHostExtensionService.ts#L103

Main Thread

This follows a different path, but appears to enter the extension host before being redirected back to the main thread (likely necessary to orchestrate instantiation of extension dependencies).

https://github.com/microsoft/vscode/blob/9fd6ee7095e76a304556c06bbef16fc4f05cf697/src/vs/workbench/api/common/extHostExtensionService.ts#L167-L170

https://github.com/microsoft/vscode/blob/788e39aad864837eb3b74e3d935d52594f4c8c96/src/vs/workbench/api/browser/mainThreadExtensionService.ts#L48-L50

https://github.com/microsoft/vscode/blob/e24174932025cb302c706d62b6dcae76d010fe83/src/vs/workbench/services/extensions/common/abstractExtensionService.ts#L955

And that is about as far as I could get.

EDIT: Or this could be an upstream limitation I guess. https://github.com/electron/electron/issues/21457 EDIT 2: Definitely https://github.com/electron/electron/issues/21457#issuecomment-647728163

[Lemmingh]

AFAIK, generally speaking, VS Code extension hosts and VS Code extensions are complied by TypeScript and run on the same engines, thus, loading ES modules in VS Code extension hosts and VS Code extensions is essentially the same problem.

---

Let's take a look at the runtime first.

There're two extension hosts. One is on Node.js. The other is on web worker.

Node.js's support for ES module is usable, although some features are experimental.

Module loading methods in Node.js

Method	L: ES	L: CJS	A: ES	A: CJS
import	Y	Y	Y	-
import()	Y	Y	Y	Y
require()	-	Y	-	Y

• ES: ECMAScript module
• CJS: CommonJS module
• L: Can Load
• A: Available In
• Y: Yes
• -: No

Web browsers' support is a bit awkward, as ES module is still not available in web workers on Firefox.

https://caniuse.com/mdn-javascript_statements_import_worker_support

---

TypeScript, as part of the toolchain, is critical. But when setting "module": "CommonJS", the dynamic import() expressions are replaced with require() calls, which is fatal. That's why people pay great attention to changes in TypeScript.

Bundlers, such as webpack and Rollup, also have a few problems with ES module.

---

We can learn from #135450 that VS Code needs to intercept the loading of vscode to control extension API access. I guess that's not a big problem, because we can configure bundlers to always emit require("vscode"), so that the extension hosts won't have to worry about other module systems.

EDIT:

Please forget what I said about require("vscode") above.

I thought it's enough to register a globalThis.require() or modify the module.prototype.require(), and gave an example demonstrating how to call require() in ES modules. But it's just not feasible for a JavaScript function to trace the caller module.

Then, it's still a big problem to talk to JavaScript engines to intercept the import "vscode". Both Node.js and WHATWG have proposals.

[savetheclocktower]

Having stumbled across electron#21457 while investigating why an Atom extension couldn't use ESM imports, I'm a bit flummoxed. The crux of it seems to be that (a) Electron is complicated, (b) supporting ESM within Electron is complicated due to security issues and clashes in resolution algorithms between Node and browser environments; and (c) nobody seems to be sure what to do about it, including Electron's maintainers. The result is an issue that's been open for two years, is too important to be closed as a won't-fix, but seems no closer to resolution than on the day it was opened.

In a vacuum I'd take it as a symptom of a loss of mindshare and relevance — people seem much less sanguine about Electron than they used to be, and Atom (Electron's original reason for being) feels like it's basically in maintenance mode now. But then I remembered about Slack, and Figma, and Discord, and VSCode, and I'm now wondering how the platform underpinning all of these major apps is so paralyzed over ES modules. Is electron#21457 a blocker for this issue, as @Silic0nS0ldier suspects?

The Node ecosystem will only embrace ESM more and more. For too long, publishing packages on NPM has involved module bundlers and transpilation and some amount of voodoo, but many package authors did it anyway, because there was the promise of an eventual blue-sky future when all of that would be unnecessary. Without ESM support in Electron, I imagine the only recourse for VSCode extension authors (like Atom package authors) is to resort to those same transpilers, and/or tools like standard-things/esm which were always intended as transitional stopgaps and will only work less and less well over time.

So I’m curious about whether electron#21457 is on VSCode's radar, and whether its developers (and Slack's, and Discord's…) can perhaps be part of the brainstorming needed to get it moving.

The mystery I'm stuck on is why this impasse doesn't feel more urgent to major Electron apps like VSCode, but the answer could be quite simple. Perhaps there are aspects of VSCode's internals that make this problem less salient; or even remove Electron as a blocker; I'm not an expert there. But when VSCode is asked “Do you support ES modules?” in the year 2022, even a “yes, but…“ answer is a red flag when the answer is a flat “yes” most everywhere else.

[lewisl9029]

EDIT: Unfortunately this only works in local extension dev, not when packaged. See the next 2 messages for details.

I started building a new extension a while back, here's the workaround I'm using at the moment that doesn't involve any third party libraries/transpilers:

In package.json, we point to a .cjs file as the main entrypoint:

  "main": "./main.cjs",
  "type": "module",

In main.cjs, we use dynamic import to import the ESM module, and then await it in activate:

const importingMain = import('./main.js')

module.exports = {
  activate: async (context) => {
    ;(await importingMain).activate(context)
  },
}

In main.js, we use createRequire to import vscode which is still CommonJS, but otherwise everything else can be in native esm downstream:

import { createRequire } from 'module'
const require = createRequire(import.meta.url)

const vscode = require('vscode')

export const activate = (context) => {
  //...
}

The setup seems to be working OK for me so far, hope it's helpful for others as well!

[Silic0nS0ldier]

@lewisl9029 I tried using your dynamic approach (similar to what I tried prior to posting back in December https://github.com/microsoft/vscode/issues/130367#issuecomment-987414840) without success.

First issue, vsce interprets ./main.cjs as ./main.cjs.js. Easily enough worked around by shifting ESM code into its own folder which contains package.json with "type": "module", or using .mjs.

Second issue, I get an error once the dynamic import is hit. Same I hit previously. TypeError [ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING]: A dynamic import callback was not specified. Most search hits for this will turn up Jest which doesn't support ESM in a lot of cases (it replaces the module loader in CJS code). EDIT: This is mentioned at https://github.com/microsoft/vscode/issues/116056

Has anyone else had success? And if so, could you share your VSCode version and OS? Are you using VSCE to package your extension?

[lewisl9029]

@Silic0nS0ldier I haven't ran in to the first issue, but glad you found a workaround.

Re: the second issue, I seem to remember testing a packaged version with this pattern and seeing it work, but my memory must be failing me, because when I try to repro now I also see the same ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING error.

I traced it back to the options to vm.Script missing a importModuleDynamically function to implement dynamic import resolution.

I suppose if we have someone sufficiently motivated, and if we can get VSCode team's blessing, we could implement that function to get this pattern to work with packaged extensions. But I suspect there would be a high bar there in terms of security/compatibility implications and won't become available for quite a few releases.

In the mean time though, unfortunately it does mean this is pattern is only useful for local dev, which practically means not very useful at all. Apologies for getting people's hopes up only to disappoint. 😞

[Silic0nS0ldier]

@lewisl9029 Could be that support was removed in a past release, likely involuntarily. Regardless knowing that dynamic import works in the extension development host is useful information, it means Electron isn't the blocker.

[lewisl9029]

Found another semi-blocker to using native unbundled ESM: https://code.visualstudio.com/api/working-with-extensions/bundling-extension

The first reason to bundle your Visual Studio Code extension is to make sure it works for everyone using VS Code on any platform. Only bundled extensions can be used in VS Code for Web environments like github.dev and vscode.dev. When VS Code is running in the browser, it can only load one file for your extension so the extension code needs to be bundled into one single web-friendly JavaScript file. This also applies to Notebook Output Renderers, where VS Code will also only load one file for your renderer extension.

This is a rather unfortunate constraint, because there are plenty of ways to flatten module waterfalls without bundling these days.

Unfortunately since I'm targeting Web as well, looks like I'm going to have to bundle regardless of ESM support. 😞

[Silic0nS0ldier]

I believe I've worked out why dynamic import is failing. Within the extension host a custom loader is used, VSCode Loader, an AMD implementation.

The NodeScriptLoader (loader is built to work in multiple runtimes) loads modules using require('vm').Script which if not given a callback for option importModuleDynamically will cause loaded modules to error with ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING. NodeJS docs

To support dynamic import 2 things need to happen.

1. Provide a dynamic import callback (importModuleDynamically) in NodeScriptLoader (VSCode Loader).
   • https://github.com/microsoft/vscode-loader/blob/0f368c94f43a1616cc2867f9d2d5f724652b6f3b/src/core/scriptLoader.ts#L395
   • https://github.com/microsoft/vscode-loader/blob/0f368c94f43a1616cc2867f9d2d5f724652b6f3b/src/core/scriptLoader.ts#L470
2. Update the compiled loader script at src/vs/loader.js (VSCode).

Problems I can see here:

• Does NodeJS 14 support importModuleDynamically? ~It does not appear in @types/[email protected]~. EDIT: This is undocumented EDIT 2: This is available in VSCode 1.63.2/NodeJS 4.16.0
• ES modules have different module resolution logic. I don't think import() can be used on its own (it resolve relative to the module scope). import.meta.resolve would help but is marked experimental. EDIT: import.meta.resolve does not work for code inside the context the VSCode loader creates. If wanting to create your own ESM support you'll need to supply your own module resolution logic.

[frank-dspeed]

Just my 5cent once you understood the both module systems your clear already able to use ESM only excempt as already saying is bootstraping of some electron code

as a Engine Coder it is clear why there is no ESM Support on bootstrap it is because NodeJS did not finish the Nativ Module bindings for the ESM Integration ok there is dlopen but thats it

Short version

dynamic import() with full file path to a real ESM file will always work! and did work the last 7 years since it exists and did work 8+ years since it is in dev thats how long i use ESM inside Electron and vscode as also extensions just my 5cent.

there is a ESM Module Loader coded in CJS its called esm and is aviable via npm i esm

It implements FULL! i Repeat FULL! More then Complet implementation of the ESM loader in Userland written in CJS. that even Works to bootstrap.

[Manish3323]

@frank-dspeed , you are correct. The issue seems to be more related to how typescript works with dynamic imports. Stable version of Typescript converts dynamic import() to require statements while doing transpilation. This is problematic. Typescript's beta seems to be trying to solve this issue. (https://github.com/microsoft/TypeScript/issues/46452) target: 'NodeNext' is something i am looking forward to. This beta version still has some bugs. I am waiting for it to get stable.

[frank-dspeed]

@Manish3323 i am evaluating that a lot since some monthes and can tell you it solves nothing my final solution is simple but effecient i coded my own universal loader that can load ESM and CJS and invinted my own Module Standards that are universal compatible.

The Main Problem between the both ecosystems is that ESM is Async by default while some environments do depend on something that is called a sideEffect and a sideEffect in the Nativ Environment means that you can not use ESM to catch the first events as it will get executed after the first process.nextTick also you can not require anything syncron from ESM as ESM is already Async by default even if the engine starts it it is Async.

also all NodeNativ code can also only get required so all nativ code dependent node modules. can only get loaded via the CJS context that is bound in sync to the engine as it is Sync by default.

if you want my conclusion about getting good support then the conlusion is the support is fully there while the coders are not aware of the edgecases like avoid default exports and module.exports so go namedExports only if your forced to supply a default then simply always export a namespace with the same modules assigned to it that you export and also export the names this way the engine does no magic and you module works also transpiled with a single ts file.

i publish the code of the loader soon it is in general rollup hooking into require to require also mjs files as cjs or mjs or vice versa on runtime and register the modules in the require cache.

https://github.com/stealify/ecmascript-loader

so it allows to require any node_module without any problems when it is ESM it will resolve everything and transpil everything as also register all modules and return the result.

[DavidAnson]

I’ve hit this issue with the latest (0.48.0) release of the markdownlint extension: https://marketplace.visualstudio.com/items?itemName=DavidAnson.vscode-markdownlint. As one of the comments above notes, import works perfectly well during development (both for normal and web worker modes), but then fails after publishing a package to the Marketplace and using it. Unlike the people above that may be able to recompile or alter their code, my scenario involves loading a user script from their workspace at runtime, so I need import to function correctly. The error I get from VS Code is “Invalid host defined options”. I’m happy to provide more information or steps to reproduce if that would be helpful.

[fwouts]

@DavidAnson I was in a similar situation (my extension relies on https://github.com/vitejs/vite which recently migrated to ESM).

Here is the workaround I implemented:

• VS Code extension spins up a local Node server which contains all core logic (using https://github.com/sindresorhus/execa)
• VS Code extension communicates with the Node server via HTTP (ideally it would be via IPC, but I couldn't quite figure that out especially since node-ipc was recently compromised)

It's slower and uses more memory, but at least it doesn't crash 😕

See the code here: https://github.com/fwouts/previewjs/blob/a1222788f95b58cbbade800a4adbf584990fc291/integrations/vscode/src/index.ts#L40

[DavidAnson]

@fwouts, thank you so much! In my case, there has been very little adoption of .mjs configuration files (it's a new feature of the CLI), so I removed the documentation for this in the extension and will leave it to fail at runtime for anyone who tries. My hope is that VS Code will address the issue soon and it will start working without any further action.

[lucono]

I ran into this issue with the latest release of Karma Test Explorer while trying to update some dependencies that had now migrated to ESM. I was able to make it work as an ESM package by using esbuild to bundle to cjs, and bundling the output files with a .cjs extension as seen in the esbuild config here.

Also had to alias the entry main.cjs file to main.cjs.js, which got it past a vsce packaging hiccup where it would look for main.cjs.js and not recognize the main.cjs in the package.json as a cjs file.

[ShenHongFei]

@lucono I have submitted a pr to vsce for .cjs extension https://github.com/microsoft/vscode-vsce/pull/740

[ShenHongFei]

Create a file named patches/@[email protected]

# support cjs entry `main: "./xxx.cjs"` in package.json
# https://github.com/microsoft/vscode-vsce/pull/740

diff --git a/out/package.js b/out/package.js
index 3711bf76db20fa2b351089a8c32ae8c437a9b7d9..f9da497b9925f24e4596a85cb41aa2cd2d43946e 100644
--- a/out/package.js
+++ b/out/package.js
@@ -693,7 +693,7 @@ class LaunchEntryPointProcessor extends BaseProcessor {
         }
     }
     appendJSExt(filePath) {
-        if (filePath.endsWith('.js')) {
+        if (filePath.endsWith('.js') || filePath.endsWith('.cjs')) {
             return filePath;
         }
         return filePath + '.js';

add following section in package.json

    "pnpm": {
        "patchedDependencies": {
            "@vscode/[email protected]": "patches/@[email protected]"
        }
    }

pnpm install patched @vscode/vsce package

pnpm install

[ilg-ul]

There were some new discussion in the electron project, and I tried to make them aware that this problem is also important for the VS Code extensions: https://github.com/electron/electron/issues/21457#issuecomment-1458926542

There was also a more technical comment (https://github.com/electron/electron/issues/21457#issuecomment-1460028943), which I'm not able to reply, so, it would be great if someone with a better understanding of the problem could also join the discussion, perhaps we can move things forward.