Error: Cannot set headers after they are sent to the client when using `devMiddleware.etag` and `static`

[mjames-c]

Bug report

Error using devServer. devMiddleware.etag: "weak" with devServer.static causes webpack-dev-server to crash with the following runtime error ([email protected])

  Error: Cannot set headers after they are sent to the client
    at ServerResponse.setHeader (node:_http_outgoing:655:11)
    at SendStream.redirect (<omitted>/node_modules/.pnpm/[email protected]/node_modules/serve-static/index.js:202:9)
    at SendStream.emit (node:events:518:28)
    at SendStream.emit (node:domain:489:12)
    at SendStream.redirect (<omitted>/node_modules/.pnpm/[email protected]/node_modules/send/index.js:475:10)
    at onstat (<omitted>/node_modules/.pnpm/[email protected]/node_modules/send/index.js:722:41)
    at FSReqCallback.oncomplete (node:fs:198:5) {
  code: 'ERR_HTTP_HEADERS_SENT'
}

How Do We Reproduce?

1. update your webpack config with the following:

// webpack.config.js
{
  entry: './some/dir',
  devMiddleware: {
    etag: 'weak',
  },
  static: {
    directory: "./some/dir",
  },
}

2. spin up the dev server and then load the page in your browser (you may need to refresh once)

Please paste the results of npx webpack-cli info here, and mention other relevant information

  System:
    OS: Linux 6.8 Ubuntu 22.04.5 LTS 22.04.5 LTS (Jammy Jellyfish)
    CPU: (16) x64 Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz
    Memory: 102.13 GB / 123.81 GB
  Packages:
    babel-loader: ^8.0.6 => 8.2.4 
    css-loader: ^5.2.4 => 5.2.6 
    ejs-loader: ^0.5.0 => 0.5.0 
    file-loader: ^5.0.2 => 5.0.2 
    html-webpack-plugin: ~5.5.0 => 5.5.3 
    loader-runner: ^4.1.0 => 4.2.0 
    loader-utils: ^1.2.3 => 1.2.3 
    postcss-loader: ^5.2.0 => 5.2.0 
    raw-loader: ^4.0.0 => 4.0.2 
    svg-url-loader: ^3.0.3 => 3.0.3 
    svgo-loader: ^3.0.0 => 3.0.0 
    swc-loader: ^0.2.6 => 0.2.6 
    terser-webpack-plugin: ^5.2.4 => 5.3.9 
    thread-loader: ^4.0.2 => 4.0.2 
    ts-loader: ^9.4.4 => 9.4.4 
    url-loader: ^4.1.1 => 4.1.1 
    webfontloader: ^1.6.28 => 1.6.28 
    webpack: 5.99.9 => 5.99.9 
    webpack-cli: ^5.1.4 => 5.1.4 
    webpack-dev-server: ^5.2.1 => 5.2.1 
    webpackbar: ^5.0.0-3 => 5.0.0-3 

[mjames-c]

I discovered that the underlying issue is with overlapping paths between generated assets and static assets. If ya'll are ok with supporting that this could easily be fixed by wrapping the express static middleware in a res.headersSent check

[alexander-akait]

PR welcome

[benjaminr-ps]

Getting the same issue, if lastModified is set to true, in devMiddleware. I would really like to see PR for this from @mjames-c , as well :)

Error: Can't set headers after they are sent.
    at SendStream.headersAlreadySent ( C:\widget\node_modules\send\index.js:386:13)
    at SendStream.send ( C:\widget\node_modules\send\index.js:612:10)
    at onstat ( C:\widget\node_modules\send\index.js:724:10)
    at FSReqCallback.oncomplete (node:fs:198:5)
node:_http_outgoing:659
    throw new ERR_HTTP_HEADERS_SENT('set');
    ^

Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
    at ServerResponse.setHeader (node:_http_outgoing:659:11)
    at SendStream.redirect ( C:\widget\node_modules\serve-static\index.js:202:9)
    at SendStream.emit (node:events:519:28)
    at SendStream.redirect ( C:\widget\node_modules\send\index.js:475:10)
    at onstat ( C:\widget\node_modules\send\index.js:722:41)
    at FSReqCallback.oncomplete (node:fs:198:5) {
  code: 'ERR_HTTP_HEADERS_SENT'
}

System-Info

  System:
    OS: Windows 11 10.0.22631
    CPU: (20) x64 13th Gen Intel(R) Core(TM) i7-13800H
    Memory: 3.82 GB / 31.62 GB
  Binaries:
    Node: 20.16.0 - C:\cygwin64\tmp\xfs-36f575b0\node.CMD
    Yarn: 3.3.1 - C:\cygwin64\tmp\xfs-36f575b0\yarn.CMD
    npm: 10.3.0 - ~\AppData\Roaming\npm\npm.CMD
  Browsers:
    Edge: Chromium (134.0.3124.51)
    Internet Explorer: 11.0.22621.3527
  Packages:
    babel-loader: ^9.2.1 => 9.2.1
    case-sensitive-paths-webpack-plugin: ^2.4.0 => 2.4.0
    copy-webpack-plugin: ^12.0.2 => 12.0.2
    css-loader: ^7.1.2 => 7.1.2
    css-minimizer-webpack-plugin: ^7.0.0 => 7.0.0
    eslint-webpack-plugin: ^4.2.0 => 4.2.0
    file-loader: ^6.2.0 => 6.2.0
    html-webpack-plugin: ^5.6.0 => 5.6.0
    postcss-loader: ^8.1.1 => 8.1.1
    resolve-url-loader: ^5.0.0 => 5.0.0
    sass-loader: ^16.0.2 => 16.0.2
    source-map-loader: ^5.0.0 => 5.0.0
    style-loader: ^4.0.0 => 4.0.0
    terser-webpack-plugin: ^5.3.10 => 5.3.10
    webpack: ^5.95.0 => 5.95.0
    webpack-cli: ^6.0.1 => 6.0.1
    webpack-dev-server: ^5.2.2 => 5.2.2

[alexander-akait]

@mjames-c @benjaminr-ps I really can't reproduce, both and webpack-dev-middleware and send (using under the hood of serve-static) have headersSent check, so it is impossible to happen except if you are using non default app (i.e. not express)

Please create reproducible test repo or provide configuration where I can catch this problem

[broehl-ptc]

@alexander-akait Thanks a lot for investigation. Unfortunately, I have not much time to build minimal example, currently.

The configuration & start script files are mainly from create-react-app package, but slightly adapted for webpack's latest version, since FB no more maintains.

It not immediately crashes. On the second request, only.

1. Load App -> webpack serves bundle.js, with Last-Modified header
2. Load App -> webpack responds with 200 OK and still same Last-Modified header, so browsers uses bundle.js from cache
3. webpack-dev-server crashes, very shortly after

By the response header, it seems to use Express, x-powered-by Express, and the webpack-dev-server seems to have express as dependency:

I am bit confused, since you stated express as non-default app.

files.zip

[alexander-akait]

Thanks for reproducible example, I will look deeply

[alexander-akait]

@broehl-ptc Do you have something special in your code? I see configuration, no code

[alexander-akait]

Fixed by https://github.com/webpack/webpack-dev-middleware/pull/2155

[broehl-ptc]

Thanks alexander, I applied your changes manually, and indeed it works very well. With that, using lastModified header, you save a lot of developer's time, as browser reloads do not fetch bundle.js fully ;)