locutus
locutus copied to clipboard
ES Modules?
Is there any plan on converting from CommonJS format to ES Modules format?
There are benefits to using ES Modules for distribution code (e.g. tree-shaking of unused exports, scope hoisting, industry standard), and there are tools which can help in that.
Hi there, the downside for the time being is that stable Node.js versions (haven't kept track so well but I don't think 8 or 10 have support for it?) don't support it. So anybody wanting to grab a function would then first have to use e.g. Rollup in order for it to function, which is not so common on servers yet as it is for webbrowsers. I do think that Rollup supported tree-shaking for commonjs requires or no?
The process would definitely use Rollup to prepare CommonJS exports as default and ESM exports as optional.
For example, PHP number_format
would look like this for CommonJS:
const numberFormat = require('locutus/php/strings/number_format');
And for ESM:
import numberFormat from 'locutus/esm/php/strings/number_format'
This way Locutus could bump minor version with ESM support as opposed to switching to ESM as default which would require major version bump.
You are correct in that Node 8 doesn’t support ESM, but plug-and-play support can be achieved with ES Module loader.
Does this make sense to you? I have some experience with this and I could probably prepare initial PR for this.
Thanks! I realize that when folks install from npm we could ship a built/bundled version. In fact we already do so we can use es6 and not worry about older Node platforms. So I guess we could rewrite imports to requires at that stage also.
It's just that, currently, you'd be able to even copy paste a function in node, and that goes out the window until module supports hits a good majority of node.js installs (unless we ask everybody to install ES module loader with their apps, and hack the global require
). I think on that trade-off, it's not worth it just yet to move to es6 modules. If we revisit in a year or so, and node supports import
out of the box, I'd be very happy to change all require
s in a heartbeat. When I'm balancing the pros and cons i don't think it's the right time for a project that also caters to the serverside (i'm very much in favor of imports for browser/bundle-only projects today already). Or what do you think?
I'd love to have an ESM version. Of course our code fits the example that it's bundled with Webpack and runs in the browsers. So I understand the want/need to keep it compatible for native Node applications that can't take advantage of the ESM syntax. We're using all ESM syntax where possible, which is for the majority of our code.
I think having an ESM path in Locutus sounds good. With Webpack for example it can easily alias locutus/php/strings
to locutus/esm/php/strings
for imports.
Another option would be to create an ESM branch in Git and then publish the branch with Node's beta tag https://docs.npmjs.com/getting-started/using-tags Then the ESM branch wouldn't be installed unless the user specified @beta
during the install.
The only downside I can think of this is that two files have to be managed instead of one, because you can't even merge the master branch into the ESM branch. If that's not a concern, though, then I don't see why this isn't a viable option. Then when Node natively supports it, you can merge the ESM branch into the master branch.
Interestingly, I don't know if it has to do with the types
declaration from DefinitelyTyped or Webpack, but we're able to import with the ESM syntax.
Maintaining two branches is a concern I’m afraid :) as is offering multiple ways to require/import, that’s just a source of confusion, tickets, and extra explaining we have to do in the docs and these instances.
(If you feel differently about maintaining two branches, it’s possible to fork ofc!)
As a maintainer these are considerable downsides to me, where the upsides aren’t so apparent, again, won’t roll up also treeshake requires? And won’t you be able to include single functions, meaning treeshaking is less of a requirement?
We’ll change to imports, that I can predict/assure you. I don’t think now is the right time though (sorry!) I’d like to do it when less hacks/workarounds are required to make it work for all platforms. When a good majority of node installs supports esmodules out of the box, let’s revisit?
Sent from mobile, pardon the brevity.
On 19 Sep 2018, at 16:39, KnightYoshi [email protected] wrote:
I'd love to have an ESM version. Of course our code fits the example that it's bundled with Webpack and runs in the browsers. So I understand the want/need to keep it compatible for native Node applications that can't take advantage of the ESM syntax. We're using all ESM syntax where possible, which is for the majority of our code.
I think having an ESM path in Locutus sounds good. With Webpack for example it can easily alias locutus/php/strings to locutus/esm/php/strings for imports.
Another option would be to create an ESM branch in Git and then publish the branch with Node's beta tag https://docs.npmjs.com/getting-started/using-tags Then the ESM branch wouldn't be installed unless the user specified @beta during the install.
The only downside I can think of this is that two files have to be managed instead of one, because you can't even merge the master branch into the ESM branch. If that's not a concern, though, then I don't see why this isn't a viable option. Then when Node natively supports it, you can merge the ESM branch into the master branch.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
Yes, I understand the part with the complexity that would come up with the project, but I think it could be flexible with the proper distribution method.
- The source code is in the
src/
directory and is never published to npm - The Rollup prepares two versions: CommonJS that goes into the root project (as it is right now) and the ES Modules version that goes into
esm/
directory - Documentation and module consumation remains the same for all those who have used CommonJS approach, but we document that ES Modules are available under the
esm/
"namespace".
Another option would be to create an ESM branch in Git and then publish the branch with Node's beta tag
I think this could create larger maintenance issues.
won’t roll up also treeshake requires?
Rollup has tree shaking and scope hoisting for ES Modules only, not for CommonJS.
And won’t you be able to include single functions, meaning treeshaking is less of a requirement?
True, but still ES Modules have their benefits over CommonJS, especially in browser landscape.
In the end, it’s your decisions and I’m glad you’re open to the option for using ES Modules by default!
Thanks, I didn't know that about RollUp! I thought only Webpack had the constraint on ESmodules. And RollUp would treeshake whatever you threw at it. That doesn't change though that folks are requiring single functions and that treeshaking isn't so beneficial then.
True, but still ES Modules have their benefits over CommonJS, especially in browser landscape.
Can you eloborate on that, glancing over the article it mostly seems to address bundlesizes and performance when bundled via different bundlers? I don't see how requiring vs importing a single function would impact this.
Documentation and module consumation remains the same for all those who have used CommonJS approach, but we document that ES Modules are available under the
esm/
"namespace".
Once we go all in with Locutus, we'll have to deprecate and move everybody's code back from using esm/
, or keep pushing out clones under that namespace. Since I don't see overly compelling upsides for this particular project, this bit is rather unattractive to me. I'm also considering to have a module per function, like @locutus/php-strings-sprintf
via Lerna.js. While perfectly possible, pushing out the extra namespace per module seems like it would increase chances-of-headaches and build-times.
I don't see how requiring vs importing a single function would impact this.
Importing 10 CommonJS single functions results in 10 function calls for that modules exports. Importing 10 ES Modules single functions, with proper scope hoisting support (Rollup, Webpack and Parcel support that) results in flattened 1 function call. It’s probably negligible in the case of 10 modules, but it can show performance degradations in large codebases (the article goes in depth for this situation).
Once we go all in with Locutus, we'll have to deprecate and move everybody's code back from using esm/, or keep pushing out clones under that namespace.
True, this could be major problem for usage, but on the other hand it also means major version bump which at least expects of developers to revisit their codebase and make proper changes.
I'm also considering to have a module per function, like @locutus/php-strings-sprintf via Lerna.js.
Oh, this could be great! I suppose majority of Locutus users are actually using some, and not all functions in their codebase.
While perfectly possible, pushing out the extra namespace per module seems like it would increase chances-of-headaches and build-times.
Do you mean @locutus
or esm/
namespace? For former, this makes sense since it’s ecosystem standard and I don’t see problems with that. For latter, esm/
in case of "one function per package" can (and should) be avoided with main
and module
property of package.json
so you can have both distribution files in the package root directory.
For latter,
esm/
in case of "one function per package" can (and should) be avoided withmain
andmodule
property ofpackage.json
so you can have both distribution files in the package root directory.
That sounds very reasonable and would solve a big part of my concerns. Would you like to take a swing at that? So every function is a module with a main
and module
entry point?
Do you have some baseline we can work with?
Is the plan to have project structure similar to the one Babel uses, so PHP’s number_format
is actually inside packages/php-strings-number_format
(or packages/locutus-php-strings-number_format
) and scoped to @locutus/php-strings-number_format
?
I think that sounds great yes. I wonder if in the case of php we should make an exception and rip out the middle part so that it becomes php-sprintf for example. Not sure, exceptions also mean more if statements and surprises and workarounds, for it does feel more natural to consume the module that way, maybe.
As for internal directory structure, do you think we can keep the nesting category? So ./packages/php/strings/sprintf.js (module: @locutus/php-sprintf) and ./packages/ruby/Math/acos.js (module: @locutus/ruby-math-acos).
All up for debate, what do you think?
Sent from mobile, pardon the brevity.
On 20 Sep 2018, at 13:40, Ivan Nikolić [email protected] wrote:
I don't see how requiring vs importing a single function would impact this.
Importing 10 CommonJS single functions results in 10 function calls for that modules exports. Importing 10 ES Modules single functions, with proper scope hoisting support (Rollup, Webpack and Parcel support that) results in flattened 1 function call. It’s probably negligible in the case of 10 modules, but it can show performance degradations in large codebases (the article goes in depth for this situation).
Once we go all in with Locutus, we'll have to deprecate and move everybody's code back from using esm/, or keep pushing out clones under that namespace.
True, this could be major problem for usage, but on the other hand it also means major version bump which at least expects of developers to revisit their codebase and make proper changes.
I'm also considering to have a module per function, like @locutus/php-strings-sprintf via Lerna.js.
Oh, this could be great! I suppose majority of Locutus users are actually using some, and not all functions in their codebase.
While perfectly possible, pushing out the extra namespace per module seems like it would increase chances-of-headaches and build-times.
Do you mean @locutus or esm/ namespace? For former, this makes sense since it’s ecosystem standard and I don’t see problems with that. For latter, esm/ in case of "one function per package" can (and should) be avoided with main and module property of package.json so you can have both distribution files in the package root directory.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
I think @locutus/<language>-<module>
sounds good. I don't see any benefit from having it be more verbose with the path in the name.
Don’t you think for some languages that are more class/object oriented you might get collisions between Math and String, or function names are going to sound less meaningful without the parent they belong to? Since php is so function oriented, nearly all array functions start with array_ so it’s pretty obvious, also; that’s how you call them from within the language, not array.array_search() or what have you. In ruby though it’s Math.acos and not plain acos. So I think this nesting makes sense(?)
Sent from mobile, pardon the brevity.
On 22 Sep 2018, at 17:03, KnightYoshi [email protected] wrote:
I think @locutus/
- sounds good. I don't see any benefit from having it be more verbose with the path in the name. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
When you put it that way, then I guess it makes sense.
As for internal directory structure, do you think we can keep the nesting category?
I suppose we’re going the way of Lerna, so first step would be to see if Lerna has some option to automatically create distribution directories for packages. Keeping files in the current directory state clashes with the functionality of package.json
, so if we don’t have proper relocation process in place, directory structure should be packages/php/strings/number_format/index.js
and inside that directory package.json
for that module at least, test infrastructure at most (see Babel structure).
I wonder if in the case of php we should make an exception and rip out the middle part so that it becomes php-sprintf for example
I wouldn’t go that way. This could introduce a lot of edge cases and confusing moments. @locutus/php-array-array_reverse
should be okay :)
Ok, I'm cool with packages/php/strings/number_format/index.js
and consistently using the class in module names @locutus/php-array-array_reverse
. Tests are generated and extracted from header comments, I think we should probably not output the generated mocha tests inside the packages/php/strings/number_format/
to avoid people from hacking on those generated files. Likely this also makes/keeps the tests faster.
Another approach could maybe also be to regard Lerna packages as build residue. e.g. we'd keep everything as is, but as part of the build step, which already does AST parsing and crazy things, it would populate a .lerna/packages
directory with index.js
files and package.json
s were needed , just so we can publish individual modules to npm, but we don't let that decision change how we currently work on/automate our functions.
Wondering if @kukawski has any strong opinions or insights on this matter?
@kvz I find it very nice to have each function as separate module published under a namespace and maintained from single repository. Regarding our test setup that relies on comments in the source files - sometimes I found it quite limiting, because I would like to test a function thoroughly, which would end up with infinitely long comment. Maybe we should consider writing test files manually? Of course there are things to consider or double-check for feasibility - like the folder structure in the repository. Currently, IIRC, user can require whole category of functions, because we generate index.js. Do we want to keep it? How do we deal with shared code? Where do we keep it? How do we deal with module dependecies? Some functions rely on other functions in the locutus package. Then, how do we deal with existing users, if suddenly the way how to use the module changes? Can we change the locutus module to be a namespace in npm? I've never checked that.
Regarding our test setup that relies on comments in the source files - sometimes I found it quite limiting, because I would like to test a function thoroughly, which would end up with infinitely long comment. Maybe we should consider writing test files manually?
For starters, maybe use what currently exists and then switch to manual writing, push MVP and then iterate?
Currently, IIRC, user can require whole category of functions, because we generate index.js. Do we want to keep it?
I think this could be possible, maybe event with Lerna as main tool.
How do we deal with shared code? Where do we keep it? How do we deal with module dependecies? Some functions rely on other functions in the locutus package.
If we go the way of Lerna, we should probably check what other projects do in regards to that (e.g. Babel).
Then, how do we deal with existing users, if suddenly the way how to use the module changes? Can we change the locutus module to be a namespace in npm?
npm has deprecation process for existing packages. If we publish current implementation (everyting in index.js) under @locutus/locutus
, I suppose majority of users will have easier transition period (I think npm offers solution where locutus
could be aliased to @locutus/locutus
). For everything else (single function) users should use namespaced modules. This should probably be major version bump. What do you think?
With regards to Babel 7+ they use a namespace structure
{
"devDependencies": {
"@babel/core": "7.1.0",
"@babel/plugin-proposal-class-properties": "7.1.0",
"@babel/plugin-proposal-export-namespace-from": "7.0.0",
"@babel/plugin-proposal-object-rest-spread": "7.0.0",
"@babel/plugin-syntax-dynamic-import": "^7.0.0",
"@babel/plugin-transform-modules-commonjs": "7.1.0",
"@babel/preset-env": "^7.1.0",
}
Instead of every function being it's own package, would it be better to have them grouped by the language that they emulate?
One more case of popular utility library switching to ESM and sharing their process: https://blog.date-fns.org/post/ecmascript-modules-in-date-fns-v2-i1v007ou81rm
Have there been any thoughts about this since October?
One thing that could be done is to write it in ES6 first, and then use Babel to transpile it to CommonJS for distribution on npm. That way up front there's only one code base to work in, and then production distribution would have ESM syntax and CommonJS.
We can already write in ES6 as we do indeed also already transpile to ES5 so that could be a viable option yes. I don't have the time for it, but if anybody feels strongly enough to implement this PRs are welcome
So does this mean I can't use import?
If you set up Babel in your project you can.