webpack-dev-server
webpack-dev-server copied to clipboard
webpack
Poor performance with big project
- [x] This is a bug
- [ ] This is a modification request
Hi guys! First of all, Thank you for that amazing job! You are really making the difference.
I have a really big project with the build splitted with DLL (but it doesn't seem to be related to this). I've splitted it by functionality and type of files, so when starting in watch mode, webpack will watch less files as possible to improve build performance. There is also a proxy set. The application when starting in browser, has around 130 files to be downloaded (chunks, assets, etc) and it took too much time to load the application. Checking the 'Timming' tab on Chrome for some requests, the step 'Waiting TTFB' is the slower part of any request(takes around 10 seconds to complete). I've started a simple express server with the same files generated and proxy to check these values, and it's faster than using dev-server(around 1s) and I'm not able to find where is the slowest part inside the dev-server configuration.
In summary:
- big application, big chunk files
- some DLL builds before starting watch-mode + devserver
- proxy for rest-service
- assets to be a static folder configuration
- start devserver in watchmode with some features only and DLL reference configured
- devserver HTTP is several times slower than simple express server with http-proxy-middleware
My sixth sense think it can be related to thread-pool, but i didn't found anything that i can start looking for. It happens in version 3 too.
Please paste the results of webpack-cli info here, and mention other relevant information
System: OS: Linux 4.19 Debian GNU/Linux 10 (buster) 10 (buster) CPU: (8) x64 Intel(R) Core(TM) i5-10310U CPU @ 1.70GHz Memory: 5.58 GB / 11.70 GB Binaries: Node: 16.4.0 - /usr/local/bin/node Yarn: 1.22.5 - /usr/local/bin/yarn npm: 7.19.1 - /usr/local/bin/npm Packages: add-asset-html-webpack-plugin: ~3.2.0 => 3.2.0 copy-webpack-plugin: ~9.0.0 => 9.0.1 html-webpack-inject-style-plugin: ~1.0.1 => 1.0.1 html-webpack-plugin: ~5.3.1 => 5.3.2 remove-files-webpack-plugin: ^1.4.5 => 1.4.5 webpack: ~5.44.0 => 5.44.0 webpack-cli: ^4.7.2 => 4.7.2 webpack-dev-server: ^4.0.0-rc.0 => 4.0.0-rc.0 webpack-remove-empty-scripts: ~0.7.1 => 0.7.1 webpack-rtl-plugin: ~2.0.0 => 2.0.0
Expected Behavior
The 'Waiting TTFB' time in Timing tab in DevTools on Chrome be quicker than 10s. Around 1s would be nice with watching overhead.
Actual Behavior
The 'Waiting TTFB' time in Timing tab in DevTools on Chrome takes at least 10s.
Sorry I can't help here, listing of what you have is not give me possible to say why it is slow
Create a test repo would be very difficult to provide =/ Is there any 'strategy' to diagnose or some tools do you recommend that i can use ?
No magic, here simple example https://github.com/webpack/webpack/issues/12102#issuecomment-869811826
@alexander-akait thank you for your information. With this tool i could get that the slow part is the watching mode. I put to ignore all files and it took the same time as a simple express server. Do you know how the webpack select the files to watch? If it is by entrypoint or get everything?
Based on your configuration, please do profiling otherwise I can't help
Hi @alexander-akait - one perf issue we are facing in our larger project is the stats.toJson() call. I am using the below patch, and everything seems to be working just as expected (in terms of when page refreshes and when it doesn't). Instead of checking toJson-ing all the assets, we just compare the hash to decide if we need to refresh. I tried looking back in the history for why we need to check assets, but there have been a few too many refactors for me to trace back to the reasoning. Do you know of any issues that would prevent the below change from limitting the toJson work?
diff --git a/node_modules/webpack-dev-server/lib/Server.js b/node_modules/webpack-dev-server/lib/Server.js
index edfc688..0263fe4 100644
--- a/node_modules/webpack-dev-server/lib/Server.js
+++ b/node_modules/webpack-dev-server/lib/Server.js
@@ -36,13 +36,14 @@ class Server {
this.webSocketProxies = [];
this.sockets = [];
this.compiler = compiler;
+ this.currentHash = undefined;
}
static get DEFAULT_STATS() {
return {
all: false,
hash: true,
- assets: true,
+ assets: false,
warnings: true,
errors: true,
errorDetails: false,
@@ -1976,13 +1977,14 @@ class Server {
// Send stats to a socket or multiple sockets
sendStats(clients, stats, force) {
+ this.currentHash = this.currentHash ?? stats.hash;
+
const shouldEmit =
!force &&
stats &&
(!stats.errors || stats.errors.length === 0) &&
(!stats.warnings || stats.warnings.length === 0) &&
- stats.assets &&
- stats.assets.every((asset) => !asset.emitted);
+ this.currentHash === stats.hash
if (shouldEmit) {
this.sendMessage(clients, "still-ok");
@@ -1990,6 +1992,7 @@ class Server {
return;
}
+ this.currentHash = stats.hash;
this.sendMessage(clients, "hash", stats.hash);
if (stats.errors.length > 0 || stats.warnings.length > 0) {
Not sure the poor performance in OP is the same "type" as this, so lmk if you want me to move this thread.
@markjm hm, we get only small piece of stats https://github.com/webpack/webpack-dev-server/blob/master/lib/Server.js#L1170, but I think you are right, we don't need to look at emitted assets, they are always emmited in done, storing currentHash and new hash should not decrease perf, can you try locally set assets to false and check perf?
is this still an open issue in terms of performance? seems this patch is out of sync with the latest Server.js file
Closing due to inactivity. Please test with latest version and feel free to reopen if still regressions. It doesn't mean we fixed all possible perf cases - in our roadmap ETags and immutable cache (when file has content hash it is immitable), also we are open to any other improvements, but based on this feedback we can't do more, anyway feel free to open a new issue if you have ideas/questions/etc