Support having multiple entries

[RSO]

I'm submitting a feature request

Current behavior: Whenever a file is changed, the rebuild that currently happens is quite expensive.

Expected/desired behavior: We've noticed something similar in our main webpack-config and greatly improved performance by splitting of vendor packages through the CommonsChunkPlugin. I would expect that allowing multiple entries (and therefor allow splitting off a vendor chunk) would improve the recompile speed of the webpack instance, but honestly I can't tell for certain.

• What is the motivation / use case for changing the behavior? Although we are quite happy with our current setup, recompiles take 9 seconds after a single line has changed in a spec. This seems unnecessary to me.

I tried figuring this out on my own, but it seems that the current setup relies pretty heavily on the single entry. Therefor I figured to open this issue to get some discussion going and hopefully get some insights on how to potentially implement this (if it really is a win).

Ref #54

[joshwiens]

@RSO - We will be working with @wied03 & @aaronjensen to roll the beneficial features of their projects back home ( so to speak ).

CommonsChunkPlugin & Performance in general is at the top of the list.

[RSO]

Awesome! I'll be on vacation for the next three weeks, but when I come back I'd love to help out!

[brianmhunt]

Not sure, but perhaps food for thought since Wallaby supports multiple entries: https://github.com/jeffling/wallaby-webpack

Cheers!

[oviava]

Willing to provide a hand with this as it's something quite critical in our application. We've gotten to a point that the test files are way to large when loaded in the browser.

[adstep]

I'm also hitting this issue. Really would like the ability to split off all dependencies into separate file(s) using CommonChunkPlugin our currently generated test file is quite large and building sourcemaps for it is quite expensive.

[filipesilva]

At Angular CLI we're also running into this. If there's anything I can do to help let me know.

[TheLarkInn]

So I was just assisting @vsavkin with something related to slow build times. One thing that I suggested is to (for dev environment only), use DllPlugin to create a bundle of just your vendors (in a separate build step), then in your test config leverage DllReferencePlugin to "consume" that Dll bundle that was created. This should help mitigate a majority of speed related problems.

Next question would be. If we programmatically leverage CommonChunksPlugin against node_modules inside of karma-webpack, would this potentially help mitigate the issue.

[filipesilva]

@TheLarkInn is there a good way to generically create a DLL bundle from all the vendor libs used in a app? I ask because that would be needed for seemless usage in karma-webpack.

[TheLarkInn]

Yes, one could apply the plugins underneath in the plugin source but would need to be vetted

[filipesilva]

Heya all, I've been looking at a different approach to improve rebuild times for Angular CLI, which I believe works in any project since there's no webpack config change.

So what happens in webpack is that rebuilding chunks is fast in and of itself. But if anything changes on a given chunk, sourcemaps are rebuilt and that takes a long time.

This is why often builds stay on 92% optimizing assets and why turning off sourcemaps makes it faster. CommonsChunkPlugin addresses this by making a new chunk, so that rebuilds are localized.

karma-webpack doesn't currently support multiple entries or CommonsChunkPlugin, but there's another scenario where a separate chunk is created: dynamic imports.

We can force webpack to make a separate, lazy chunk for all the spec files and dependencies. But we also have to list dependencies manually, so they don't get duplicated. This way, when sourcemaps are recalculated, it's super fast because it's not doing the vendor libs.

In the CLI, these where the changes needed (code in TypeScript):

• deleted old spec loading code in src/test.ts:

// Then we find all the tests.
const context = require.context('./', true, /\.spec\.ts$/);
// And load the modules.
context.keys().map(context);
// Finally, start Karma to run the tests.
__karma__.start();

• added dynamic import:

declare var System: any;
// Then we find all the tests.
System.import('./test-specs.ts')
  // Finally, start Karma to run the tests.
  .then(() => __karma__.start());

• added new file (src/test-specs.ts) that rounds up all specs:

// Find all the tests.
const context = (<any>require).context('./', true, /\.spec\.ts$/);
// And load the modules.
context.keys().map(context);

On a toy project, with some 30 specs, there was a 30x speed increase on rebuilds (~3s to 0.01s). On a real-world medium sized project, improvement was 15x instead.

There is a single drawback that we found... there seems to be some incompatibility when using webpack.SourceMapDevToolPlugin, I can't get the sourcemaps trace back to the original files with it. I get two //# sourceMappingURL=data:application/json; at the end of the bundle, which I think screws the sourcemap.

If I remove webpack.SourceMapDevToolPlugin I get good sourcemaps but warnings like this:

25 04 2017 14:48:07.367:WARN [web-server]: 404: /base/src/polyfills.js.map
25 04 2017 14:48:07.370:WARN [web-server]: 404: /base/src/test.js.map

Anyway, I hope this helps.

[the-dr-lazy]

Hello Is there any plan for supporting DllReferencePlugin ? I think @TheLarkInn suggestion is very great, dll is a good solution for expensive chunks. I should waste lot of time for angular vendor's compilation and bundling but with dll I can compile & bundle one time and use it for another times

[joshwiens]

@filipesilva - Is the toy project you are referencing public somewhere? I'm digging into the standards updates and rolling in the bail fix along with some of the long standing performance issues and requested features. A common project to use as a performance litmus test between the cli & webpack teams certainly couldn't hurt.

[filipesilva]

@d3viant0ne the toy project I used was just ng new proj-name and then I did a bunch of ng generate component my-comp1 (up to 30). It wasn't a specific 'real' project.

Btw I've also been looking at an alternative karma setup to support multiple entries and CommonChunks, based on adding customContextFile and wrapping the karma scripts with outputted webpack files. Had some success with it but not sure if it's easily generalizable, since it really needs knowledge about script order and polyfills.

[joshwiens]

I'd be curious to take a look at it regardless and i'll setup a little perf repo inside contrib as a common benchmark though realistically, I would really have to try to make karma run slower at this point :)

[filipesilva]

Heya, I just wanted to mention that for now we're I think we're changing the the alternative I mentioned (https://github.com/angular/angular-cli/pull/6160). Out of alternatives I tested it was the one that required the less user changes (none).

I might be generalizable by not using a static context/debug.hml, instead finding a way to hook into Karma's processing of that file and adding the webpack files in the right order.

[codymikol]

As karma is now deprecated and coming up on EOL, we are no longer planning on any significant enhancements to this project and are instead going to focus on security updates, stability, and a migration path forward as karma's lifecycle comes to an end.

Thank you for supporting and using this project!