moduloze

moduloze copied to clipboard

[Editor: this question was moved here from another thread]

@MartinKolarik:

I briefly checked how moduloze currently works with other formats and I noticed there's a required config option dep-map. I guess it's needed to map require() to UMD globals but I see it's somehow used in ESM processing as well. With an empty map, both ESM and UMD outputs look like I'd expect though. Can you explain briefly why the map is needed to make sure I understand the current approach for UMD/ESM?

In UMD builds, the dependency-map is "required" for several reasons, to map resource URLs to their names:

1. The name of a resource is used by the browser-global aspect of UMD behavior as the registered global identifier (aka, "the export").

2. The name of a resource is used by the AMD aspect of UMD behavior to register a module definition by its name (aka "the export"), so that this name can later be used to indicate it as a dependency in another module's definition.

3. The name of that resource is then used by the AMD aspect of UMD behavior in the list of dependencies passed to an AMD module's definition, so that AMD knows which dependency instance(s) to retrieve from its internal definitions and provide to (aka "import for") this module. That same name is also used as the actual parameter identifier in the outer definition function for a module that needs that dependency.

So, if you require(..) some URL that Moduloze doesn't have a name for (no dependency map, or not listed), Moduloze has to generate a unique name (like Mz8382423923) for it, for it to be used in all those various ways.

If the dependency is never referenced elsewhere in the app by some other name, this auto-generated name is probably not a problem. But it absolutely might "break" your app if you're just assuming that "MyCoolModule" will be able to reference it from somewhere else, but you didn't tell Moduloze that name.

That's the main purpose of the dependency-map.

Secondarily, ESM uses the dependency-map if you're also building the "index" / "rollup" ESM module (the one that imports and re-exports all your modules as a single module). It needs names to use in all its re-exports. Of course, if it used auto-generated names for those, none of your code would know (or be able to predict) those bindings, so it'd be unuseful.

---

BTW, I definitely think something like this discussion should make its way into the "Conversion Guide" docs. That's an open TODO. :)

I asked about this because if we wanted to integrate this as a feature at jsDelivr, we'd probably want it to be zero-config from the user's perspective. Let's take these two files as an example:

// a.js
const b = require('./b'); // could be also "b", as-in require from node_modules
console.log(b.f());

// b.js
module.exports = {
    f () {
        return 1;
    }
};

If both of these files were run through moduloze (without maps), and loaded as two independent UMD bundles, it wouldn't work right now because of the random names. But I'm wondering if it could, if:

1. Import names were generated deterministically, based on the import path (probably a resolved relative path with extension).
2. Export names would be generated in the same way (using relative paths), with the option to override them (to support files loaded from node_modules).

it wouldn't work right now because of the random names.

It internally works, meaning the modules can all reference each other just fine and all auto-execute. But it doesn't externally work, meaning if another non-moduloze-processed part of your app wants to access these resources, they won't have a name to do so.

But that's not Moduloze's fault, it's the fault that you're talking about a packaging format (UMD/AMD/etc) that requires names but you haven't given it names to use. That's a weakness of the module format, not of the tool.

But I'm wondering if it could, if:

I considered implementing some sort of algorithm to name the resources based on the URL, but ultimately decided that the complexity wouldn't be practical, because even those names would still be problematic in that they wouldn't necessarily always be intuitive/predictable. For example, should those identifiers be camelCased, or CapitalFirst or should they be snake_case, etc? So many nuances and variations, whatever I picked and baked in, would be too difficult for people to rely on in their apps.

It would also mean that the moduloze's naming convention choices implicitly become part of the business logic of any app using it, which means that moduloze is now much more restricted in future changes in that respect (without causing breakage, not to mention the complexity of using modules that were compiled by different versions of moduloze, etc).

if we wanted to integrate this as a feature at jsDelivr, we'd probably want it to be zero-config from the user's perspective.

That makes sense. Unfortunately, there's nothing we can do about the fact that some module systems (like browser globals, AMD, and... ultimately, UMD) require names to load and use modules, not just their URLs. That basically requires some sort of meta-data, either that you (the CDN) are made aware of, or that the author is aware of and uses in their own build.

An author couldn't manually distribute their modules out on jsDelivr (or any CDN) in the UMD/AMD/globals format without having chosen a name for them, or nobody could consume and use them. That's just part of the authoring/distribution process if you plan to support a name-required packaging format.

So I agree, it seems like you can't build them automatically without some source for those names.

Of course, that limitation only applies to UMD. You could still host a service running a tool like moduloze to up-convert from globals/CJS to ESM (minus the index/rollup build), and just not offer the UMD conversion/hosting at all. The benefit of upconverting to ESM should standalone, even if you couldn't take advantage of also upconverting to UMD.

---

Or, for the UMD support specifically, you could pick some sort of convention, as suggested, like inferring from the filename, or maybe requiring each to be in its own directory and using that name, etc. Then you'd just need to programmatically specify the dependency-map as part of the call to moduloze.

But ultimately, whatever convention you pick there, it'd have to be something that the authors documented for their users, so users knew what identifier names to use in their apps. This seems really brittle for your CDN, with the exact same drawbacks as me baking that stuff into the tool.

The other option is to pick like a convention similar to source maps, where each file to be converted to UMD has to have a "{FILENAME}.umd-map.json" alongside it, providing a name to use. Or whatever other ideas we may bikeshed. That's not exactly "zero configuration", but it doesn't seem all that unreasonable (especially since people and tooling already do sourcemap files similarly).