TS-ESNode
TS-ESNode copied to clipboard
Yarn2 support
Hey there, this is a neat project and could hopefully someday do away with needing ts-node for everything.
I want to open this as more of a TO-DO for when I or someone else finds time. Right now I'm using yarn2, which uses a virtual filesystem and doesn't have a node_modules folder. All packages are instead stored in .zip
files and extracted when needed.
A few takeaways from my hour trying to get this to work with yarn2.0.0-rc.29 and node 13.9:
-
It looks like this package depends on there being a node_modules folder, so that'd need to be dealt with
-
Yarn2's virtual filesystem is accessible via a few different routes but the simplest is by starting node as
node --require ./.pnp.js
from the project root. Delving into exactly what happens when.pnp.js
is hooked like this is a big TODO, but I believe it monkeypatches thefs
module to automatically unzip packages whenever they're imported / required.The catch22 with using yarn2 + this neat package is:
- When
package.json
has"type": "module"
,require
is not defined at startup. Since the CLI flag--require ...
seems to literally userequire(...)
, this fails with an error. - Without
--require ./.pnp.js
, none of the workspace's packages are available, meaning--loader=@k-foss/ts-esnode
results inCannot find package '@k-foss/ts-esnode ...
- When
So maybe sometime later this month I'll have some bandwidth to tackle this. But compliments again for finding a way to leverage these new node features to potentially make typescript less painful.
Sorry I missed this earlier. I think I know why this is failing and I'm not sure how it could be resolved without finding how to remove my hack for supporting commonJS modules, because without my dynamic module type all imported node modules not ESNext format give a XYZ does not export ABC, if you had something like this:
import { ABC } from 'XYZ'
So to solve that and allow for compat with older modules or TypeScript projects with ESModule compat enabled I have to load all node_modules as dynamic modules with a dynamicinstanciate hook that manually imports with a createRequire and then returns the values on the object and spreads out anything withing imported.default. It's super hacky but it was the only way to achieve my goal of drop in support in most projects without any refactoring.
Here is how I load all "node_modules"
/**
* This dynamically imports the `node_modules` module and creates a dynamic module with all the same exports.
* @param url fileURL given by Node.JS
*/
export async function dynamicInstantiate(url: string) {
// Create a Node.JS Require using the `node_modules` folder as the base URL.
const require = createRequire(
`${url.split('/node_modules/')[0].replace('file://', '')}/node_modules/`,
);
// Import the module file path
let dynModule = require(url.replace(/.*\/node_modules\//, ''));
/**
* This is needed to allow for default exports in CommonJS modules.
*/
if (dynModule.default)
dynModule = {
...dynModule.default,
...dynModule,
};
const linkKeys = Object.keys(dynModule);
return {
exports: [...linkKeys, 'default'],
execute: (module: any) => {
module.default.set(dynModule);
// For all elements in the import set the module's key.
for (const linkKey of linkKeys) module[linkKey].set(dynModule[linkKey]);
},
};
}
That hack is the only part of this codebase I hate/want to get rid of. If anyone has any solutions or alternative ways of supporting destructing of legacy node_modules please submit a PR or let me know and I can attempt it.
I will attempt to refactor the file and module loader functions this weekend.
@armarti Can you test to see if the issue is still present with the changes made in #28 I changed how I require in Common.JS modules, which could possibly allow the Yarn2 hack to redirect require to work, although I really don't want to setup Yarn myself.
Ha just seeing this again, impressive progress you've made. With luck I'll get back to the project I was working on in the next week or two and I could try this out.
Yeah, when I started this project it was meant as a proof of concept to be integrated into TS-Node
now I've morphed it into its own ecosystem.
Wait, I wonder if my new getSource based CJS module compat/(TOTALLY NOT A HACK) makes Yarn2 Work now...
Doesn't work magically but I do think that this will be easier to be done
The core of this issue is that Yarn2 doesn't support NodeJS loader hooks, because their hack to allow for the "compressed" modules assumes that ESModules don't exist in Node.JS. This can for sure be done. I'm not sure as to how complex this will be. I myself don't use Yarn2. And as much as I'd love to solve this myself, unless I decide to understand the internals of Yarn2 I don't think I can do this myself.
One way I could see this working is if TS-ESNode is loaded from disk instead of attempting to load the installed version from Yarn
Just thought of a potential workaround, using a bin script that loads TS-ESNode. Turns out that even that won't work because the virtual filesystem appears to not include the package.json
file that includes the "type": "module
JSON