webpack
Import/export of module without explicit references to symbols disables export mangling for module
Bug report
***EDIT: I originally wrote that this issue was due to "barrel files" but as I was creating the repro case, I realized that isn't true. Pure barrel files seem to work. I've refined the issue description below.
What is the current behavior?
If there is a module that imports/exports another module without explicitly referencing any of the exports of that module, webpack disables export mangling for the module.
In a large code base, this creates a significant code bloat because all of the references everywhere to the exports of the modules will use the full name instead of a mangled one or two characters. This really adds up.
If the current behavior is a bug, please provide the steps to reproduce.
A simple repro case for this can be found here, with a description in the README: https://github.com/scameron/webpack-issue-mangling
What is the expected behavior?
Ideally, the export mangling would continue to be enabled even if I have modules that do not explicitly reference the exports but just "pass-through" the imports as subsequent exports. I can't think of any reason why this case would need to disable export mangling, but maybe I'm missing something.
I took a quick look at the code, but I'm not sure this is correct. Here's what I found though:
In the FlagDependencyUsagePlugin, where it analysis dependencies between modules and determines whether or not export mangling should be used or not used, it has a check to see if the usedExports array is empty, meaning that the module being processed does explicitly use any of the exports of the dependency module being considered. If this is true, it will call SetUsedInUnknownWay on the export info, which internally sets canMangleUse to false.
For reference:
- The
FlagDependencyUsagePlugincode that checks for empty "used exports" during dependency processing is here: FlagDependencyUsagePlugin.processReferencedModule - The
ExportInfocode that setscanMangleUseto false is here: ExportsInfo.setUsedInUnknownWay
Sorry for the delay, I saw an answer somewhere about why the current implementation is exactly like this, but unfortunately I can't find it, to be honest, I also can't find a reason why not to do this, the only option is to try to fix it and send a PR, maybe one of the tests will fail and this will allow us to determine why the logic is exactly like this. Do you want to try it?
Perhaps this was done in conjunction with this - https://github.com/scameron/webpack-issue-mangling/blob/main/webpack.config.js#L22, i.e. we expect that terser will remove them as unused values, and we will not lose performance in this place, because in big files/big projects we can have a lot of such exports.
Hi @alexander-akait,
I originally found this problem in our Production build where both Terser and module concatenation are enabled. The values still show up as full names. They cannot be removed because they are legimately used, they just aren't used by the intermediate module doing the import/export (the Consolidator.ts module in my repo case).
I don't mind trying to push a fix but my problem is that I don't really have a good understanding of the meaning of the different states that can be set on the export info. Today it calls setUsedInUnknownWay but what is the correct state of Consolidator's usage of Enums2 when they are just being returned via Provider and not explicitly referenced?
Yeah, I'm a bit out of my depth here, I don't know what any of these states and conditions really mean.
I debugged my repro case and tried to change this line here from exportsInfo.setUsedInUnknownWay(runtime) to exportsInfo.setAllKnownExportsUsed(runtime) with my naive logic being that if a module depends on another module but doesn't use any exports, it is possibly a pass-through that is just exporting everything itself.
Very strange result though, it works for the case that was breaking but it breaks the case that was working.
Before my change, the references to the enums looked like this in the build output for Consumer.ts:
54: (e, n, t) => {
"use strict";
t.r(n);
var u = t(254),
r = t(142);
const o = {
value1: u.o.ENUM1,
value2: u.o.ENUM2,
value3: u.q.ENUM3,
value4: u.q.ENUM4,
value5: r.MyEnum3.ENUM5,
value6: r.MyEnum3.ENUM6,
value7: r.MyEnum4.ENUM7,
value8: r.MyEnum4.ENUM8
};
console.log(o)
}
But after my change it looks like this:
54: (e, n, t) => {
"use strict";
t(254);
var o = t(142);
const r = {
value1: (void 0).ENUM1,
value2: (void 0).ENUM2,
value3: (void 0).ENUM3,
value4: (void 0).ENUM4,
value5: o.j.ENUM5,
value6: o.j.ENUM6,
value7: o.q.ENUM7,
value8: o.q.ENUM8
};
console.log(r)
}
So my Enum2 references are now mangled, but the Enum references that used to be correct are now invalid void references.
Clearly I have no idea what I'm doing here. I don't think I can get much further without more background knowledge about how FlagDependencyUsagePlugin is meant to work.
Hi @alexander-akait, I got a little further with some more debugging. I have something that works, at least in my particular cases, and I've created a PR here: https://github.com/webpack/webpack/pull/19219.
This issue had no activity for at least three months.
It's subject to automatic issue closing if there is no activity in the next 15 days.
Issue was closed because of inactivity.
If you think this is still a valid issue, please file a new issue with additional information.
@saurabhraghuvanshii, I took a stab at a fix in that PR and while it did fix the problem in my build, it broke a lot of automated tests. I meant to come back and try to understand the breaks better, but unfortunately I never found the time to go deeper.
So the PR is almost certainly wrong because I was making changes without really understanding the concepts in the code I was changing. But maybe there are some hints there in the fact that the change does seem to resolve the problem in my build (which is enormous and very complex).
Hopefully you can make more sense of this than I was able to.
@scameron thanks, I'll Start working on as soon as I free from my all works.
@saurabhraghuvanshii, yes, I will try it out. I'm so excited to see a fix for this one!
@saurabhraghuvanshii, sorry that the PR didn't work out. Thanks so much for taking a good crack at it, though.
One thing I came across recently that you might be interested is from this comment thread: https://github.com/orgs/webpack/discussions/16863#discussioncomment-7292793
There's a link there to this article that describes the approach in some detail: https://vercel.com/blog/how-we-optimized-package-imports-in-next-js
It basically performs an optimization where it takes pass-through rexports and it removes them altogether by rewiring the consumer and the target together with a direct dependency. That goes even further than simply fixing the problem in this issue, it also eliminates barrel files altogether, which shrinks the build and makes it a lot faster.
I guess this is implemented in Next.js but I'm not sure if it's available as open source or not. One reply to my question describes their approach as been quite heavy and complicated.
If it was possible to bring this kind of thing into standard Webpack as an optional optimization, I think that would be a really great improvement. Probably not a small effort, though...