Webpack module federation breaks with multiple entrypoints

[codepunkt]

This time it's @sokra who told me this is probably the place to open an issue.

• Operating System: Windows 10
• Node Version: 12.16.2
• NPM Version: 6.14.4
• webpack Version: 5.0.0-beta.22
• webpack-dev-server Version: 3.11.0
• Browser: Chrome 84.0.4147.105, Firefox 79.0, Edge 84.0.522.48

• [x] This is a bug

Code

https://github.com/codepunkt/module-federation-examples/tree/dynamic-host-remote

Expected Behavior

Using webpack-dev-server instead of webpack should still support module federation with additional content tacked onto the remoteEntry by defining it as an additional entry.

Actual Behavior

Running "yarn start" to start webpack-dev-server breaks module federation and thus breaks the app in development mode.

For Bugs; How can we reproduce the behavior?

1. clone repository
2. ensure you're on branch "dynamic-host-remote"
3. run yarn on repo root
4. go into "dynamic-host-remote" directory
5. run yarn start in "dynamic-host-remote" directory
6. open localhost:3001 in the browser
7. encounter an error in the browser console that happens when executing app2/remoteEntry.js with the additional contents that were added to this entrypoint by webpack-dev-server
8. OPTIONAL: run yarn build && yarn serve and revisit localhost:3001 to see production build working just fine.

[alexander-akait]

~~@webpack-bot move to webpack/webpack-dev-server~~ My mistake

[alexander-akait]

Feel free to send a fix

[codepunkt]

If I had a clue what the problem is, i'd send a PR right away. Spent a few hours with this, don't have the slightest idea 🤔

[alexander-akait]

/cc @sokra Any ideas on this?

[ScriptedAlchemy]

Okay, so the issue here is that WDS is likely appending its own entrypoint to the array / injecting a "require" to the hmr client. That typically serves as the entry module of an application. When ModuleFederation emits a remoteEnty with the same name as the entrypoint, they are merged.

This lets us attach initialization code, to federated remotes. something like changing publicPath.

webpack_public_path = new URL(document.currentScript.src).origin + "/";

https://github.com/module-federation/module-federation-examples/pull/277

The remote becomes 400kb big (usually like 5kb) and the module returned to the global is not __webpack_require__("webpack/container/entry/app1"); - therefore window.app1 will not be __webpack_require__("webpack/container/entry/app1"); and the federated API is never set to the global

@sokra i believe that either the WDS entry needs to return the container - which may lead to HMR capabilities? or the WDS entry module should not be applied to federated entry points when combined.

module.exports = {
  entry: {
    app1: "./src/setPublicPath",
    main: "./src/index",
  },
plugins: [
    new ModuleFederationPlugin({
      name: "app1",
      remotes: {
        app2: "app2@http://localhost:3002/remoteEntry.js",
      },
      shared: { react: { singleton: true }, "react-dom": { singleton: true } },
    }),
    new HtmlWebpackPlugin({
      template: "./public/index.html",
    }),
  ],

[sokra]

The wds entrypoint should be prepended instead of appended. The last value is exported.

[ScriptedAlchemy]

Will take a look, thanks @sokra

[ScriptedAlchemy]

Looks like we want to modify this

 const prependEntry = (originalEntry, additionalEntries) => {
      if (typeof originalEntry === 'function') {
        return () =>
          Promise.resolve(originalEntry()).then((entry) =>
            prependEntry(entry, additionalEntries)
          );
      }

      if (typeof originalEntry === 'object' && !Array.isArray(originalEntry)) {
        /** @type {Object<string,string>} */
        const clone = {};

        Object.keys(originalEntry).forEach((key) => {
          // entry[key] should be a string here
          const entryDescription = originalEntry[key];
          if (typeof entryDescription === 'object' && entryDescription.import) {
            clone[key] = Object.assign({}, entryDescription, {
              import: prependEntry(entryDescription.import, additionalEntries),
            });
          } else {
            clone[key] = prependEntry(entryDescription, additionalEntries);
          }
        });

[sokra]

It already prepends... So there is somethings else wrong

[sokra]

The bug is here:

https://github.com/webpack/webpack-dev-server/blob/4ab1f21bc85cc1695255c739160ad00dc14375f1/lib/utils/addEntries.js#L86-L92

Desprite the function name prepend it actually appends...

Change it to:

      /** @type {Entry} */
      const entriesClone = [].concat(originalEntry);
      additionalEntries.forEach((newEntry) => {
        if (!entriesClone.includes(newEntry)) {
          entriesClone.push(newEntry);
        }
      });

Could somebody send a PR for that? As test case you can use output.library = "TEST" and check if this created the correct global.

[alexander-akait]

Great, feel free to send a PR

[snitin315]

I will send a PR.

[codepunkt]

I'm not sure this will actually fix HMR with module federation - but i'm looking forward to try!

[ScriptedAlchemy]

Opening PR, will need to work on / help on test case

[knagaitsev]

I looked into this more closely, and here is the full problem:

In most cases, the dev server is trying to add some variation of these 2 entries (unless you disable their injection): webpack-dev-server/client/default/index.js and webpack/hot/dev-server.js.

Current behavior (webpack 5):

Our config has a single entry: main.js.

The 2 dev server entries are appended to the entry list (as explained above), resulting in something like:

main: {
  import: {
    'main.js',
    'webpack-dev-server/client/default/index.js',
    'webpack/hot/dev-server.js'
  }
}

Then we call the entryOption hook with all of these entries here: https://github.com/webpack/webpack-dev-server/blob/4ab1f21bc85cc1695255c739160ad00dc14375f1/lib/utils/updateCompiler.js#L51

This results in an entry list on the webpack side that looks like this: main.js, main.js, webpack-dev-server/client/default/index.js, webpack/hot/dev-server.js

So it would seem that prepending the 2 entries would fix the issue: main.js, webpack-dev-server/client/default/index.js, webpack/hot/dev-server.js, main.js

However, it is not that simple because webpack essentially filters out the duplicate entries, taking only the first of the duplicates (https://github.com/webpack/webpack/blob/bdf4a2b942bed9d78815af828f7935ddfcd3d567/lib/Compilation.js#L1764), so we get: main.js, webpack-dev-server/client/default/index.js, webpack/hot/dev-server.js (regardless of if we append or prepend in the webpack step). The very last in this list is currently the one that is exported, so in no case is it ever main.js while there are other entries being injected.

Thus, the solution is to either change how this filtering works, or change the selection of entries by webpack such that the very first entry from the entryOption hook is used, rather than the very last.

I think the latter is preferred, because I realize now that it is not ideal to be pushing duplicate entries into the entryOption hook (as we do currently). Instead, we should only be pushing webpack-dev-server/client/default/index.js and webpack/hot/dev-server.js into the entryOption hook, as they have not yet been registered as entries. On the webpack side, these entries should not be considered as the module exports, as they were added later after the initial configuration entries.

[sokra]

Then we call the entryOption hook with all of these entries here:

https://github.com/webpack/webpack-dev-server/blob/4ab1f21bc85cc1695255c739160ad00dc14375f1/lib/utils/updateCompiler.js#L51

Why does webpack-dev-server do that? The hooks is not owned by webpack-dev-server. It should not call it. It's already called by webpack itself. That's probably also the reason why the entries are added twice...

Seems like this seem to be some kind of hack to reapply the entry option, because it has been modified after the options has been applied, which is too late. It doesn't make sense to modify the options after they have been applied to the compiler (converted to plugin). You need to use a plugin instead.

A plugin to add these entries for webpack-dev-server could be like that:

compiler.hooks.make.tapAsync({
    name: "webpack-dev-server",
    stage: -100
}, (compilation, callback) => {
    const options = {
        name: undefined // global entry, added before all entries
    };
    const dep = EntryPlugin.createDependency("webpack-dev-server/client/...", options);
    compilation.addEntry(context, dep, options, err => {
        callback(err);
    });
});

Note the hack works in webpack 4 as each entrypoint can only have a single module (arrays are wrapped in a artificial module), so reapplying the entryOption also adds it twice, but the second one overrides the first one.

There is no global entry in webpack 4, so the above plugin doesn't work for webpack 4. I would keep the hack as legacy code, as a clean solution is more complex.

[codepunkt]

Thanks everyone for chiming in.

What confuses me a little about this is that webpack-dev-server, which i always thought of as a "major part of webpack", actually seems to be an afterthought when it comes to new developments.

I don't understand a lot of the details everyone wrote in this issue, but i'm glad this problem is being ironed out and fixed.

[pmmmwh]

Note the hack works in webpack 4 as each entrypoint can only have a single module (arrays are wrapped in a artificial module), so reapplying the entryOption also adds it twice, but the second one overrides the first one.

There is no global entry in webpack 4, so the above plugin doesn't work for webpack 4. I would keep the hack as legacy code, as a clean solution is more complex.

@sokra is there a "correct"/"clean" way to inject global entries as a plugin (for Webpack 4)? I'm asking about this for fast-refresh in general because we need to ensure global runtime code is being setup before user code. Similar to WDS, what we're doing today is this (without the entryOptions hack) within apply:

https://github.com/pmmmwh/react-refresh-webpack-plugin/blob/main/lib/utils/injectRefreshEntry.js

[MaximSagan]

Sorry to be that guy, but... any updates?

[alexander-akait]

Fully work on the next releases (included stable release) will be started on the next week

[wangmeijian]

下一个版本（包括稳定版本）的全部工作将在下周开始

When will it be updated?

[csvan]

下一个版本（包括稳定版本）的全部工作将在下周开始

When will it be updated?

When it's done, feel free to help :)

[raix]

@alexander-akait are there anything we can help out with eg. code/test/debug etc. not sure how much is missing/done?

(Despite visiting issues/pr's opened/closed and trying to follow things merged to master etc. I still don't have an overview on how to help out here)

[alexander-akait]

@raix Here fix https://github.com/webpack/webpack-dev-server/pull/2920, can you try locally? I need rewrite some tests, if it will work I will focus on it and will do release, I already work on webpack-dev-server (focused on webpack-cli and webpack-dev-server, want to finish all problem to move ahead)

[robinlarsson]

@alexander-akait are there any updates around this issue? We would love to start using Module Federation and this fix of yours would be very useful. Are you still in need of testing for #2920 locally?

[csvan]

Now that #2920 is merged, I tried running with Module Federation locally, but it seems to fail here (in the entry point of the federated module):

function checkDeferredModulesImpl() {
    var result;
    for (var i = 0; i < deferredModules.length; i++) {
        var deferredModule = deferredModules[i];
        var fulfilled = true;
        for (var j = 1; j < deferredModule.length; j++) {
            var depId = deferredModule[j];
            if (installedChunks[depId] !== 0) fulfilled = false;
        }
        if (fulfilled) {
            deferredModules.splice(i--, 1);
            result = __webpack_require__(__webpack_require__.s = deferredModule[0]);
        }
    }
    if (deferredModules.length === 0) {
        __webpack_require__.x();
        __webpack_require__.x = x => {
        };
    }
    return result;
}

The reason is that the following line will always set fulfilled to false, because dev-server injects its own entries into deferredModule:

if (installedChunks[depId] !== 0) fulfilled = false;

Thus result will always be undefined.

[alexander-akait]

@csvan Can you provide example? I think we need fix it better

[csvan]

@alexander-akait not yet unfortunately, noticed it in one of our own projects when running dev-server with a Module Federation project. I could try setting a small repro up.

I wonder where this comes from though, I suppose it is added by Webpack:

var deferredModules = [

      ["./node_modules/webpack-dev-server/client/default/index.js?http://0.0.0.0", "vendors-node_modules_webpack-dev-server_client_default_index_js_http_0_0_0_0-node_modules_web-394369"],

      ["webpack/container/entry/supportwidget", "vendors-node_modules_webpack-dev-server_client_default_index_js_http_0_0_0_0-node_modules_web-394369"]
    ];

[csvan]

@alexander-akait repro here: https://github.com/csvan/wds-mf-repro

It needs the latest version of webpack-dev-server though, the repro references the last beta

[alexander-akait]

Thanks, I will look at this in near future, the next beta will be on this week, so I try to resolve it