spdy break content when you use difference protocols for dev server and proxy

[jrweinb]

• Operating System: MacOS High Sierra v 10.13.6
• Node Version: 8.9.2
• NPM Version: 6.4.1
• webpack Version: 4.26.0
• webpack-dev-server Version: 3.1.10

• [x] This is a bug
• [ ] This is a modification request

Code

  // webpack.config.js

config.devServer = {
    hot: true,
    contentBase: resolve(__dirname, '/'),
    disableHostCheck: true,
    publicPath: '/',
    headers: {
        'Accept-Encoding': '',
    },
    historyApiFallback: true,
    host: '0.0.0.0',
    compress: true,
    quiet: true,
    proxy: {
        '/api': {
            target: 'https://XXXXX',
            secure: false,
            changeOrigin: true,
        }
}

I added the Accept-Encoding header per this advice: https://github.com/webpack/webpack-dev-server/issues/354, but it does not solve the problem. I also tried changing the timeout settings without success

Expected Behavior

That when proxying a GET request to /api, the full JSON response will be received

Actual Behavior

Sometimes the request is properly fulfilled and sometimes the response is truncated. This only happens with > 3mb JSON response.

When disabling Chrome web security and making the request directly to the server (without the wds proxy), the request is always fulfilled properly.

For Bugs; How can we reproduce the behavior?

Make a GET request with a JSON response > 3mb through the wds proxy.

To test without the proxy, disable Chrome security and make the request directly to the server.

To disable web security, from terminal:

open /Applications/Google\ Chrome.app --args --user-data-dir="/var/tmp/Chrome dev session" --disable-web-security

[alexander-akait]

@jrweinb can you create minimum reproducible test repo?

[jrweinb]

@evilebottnawi - I was unable to reproduce the error when serving a large file from a local node server. I apologize, but I failed to mention in my bug report that I am on a corporate VPN...I believe that this is the source of the issue. I found a relevant issue in the http-proxy-middleware repo: https://github.com/chimurai/http-proxy-middleware/issues/171 However, adding a "Connection": "keep-alive" header does not solve the problem for me. The only solution is to run Chrome with web security disabled

[mehl321]

I also got this issue after upgrading from 3.1.10. Big JSON responses get either truncated or even corrupted.

[alexander-akait]

@mehl321 maybe you can create minimum reproducible test repo?

[mehl321]

I created a minimal repo on my machine with a similar config (proxy, HMR, fetching a similar JSON from another local server. No HTTPS though) but I couldn't reproduce the bug. There is a bug for sure because downgrading to 3.1.10 fixed the issue.

[alexander-akait]

@mehl321 so we can close issue?

[mehl321]

Close the issue if you want but I think the bug is still here. I just didn't manage to reproduce it.

[alexander-akait]

/cc @jrweinb friendly ping

[hkostia]

I'm encountering this too but couldn't make minimum reproducible test repository.

3.1.9 OK 3.1.10+ truncates/corrupts proxied https JSON responses

Operating System: Windows 10 Node Version: 8.12.0 NPM Version: 6.4.1 webpack Version: 4.29.0 webpack-dev-server Version: 3.1.10 - 3.1.14

[alexander-akait]

I think problem on proxy server side, we don't touch content from proxy

[emillynge]

+1 on this issue. downgrade to 3.1.9 worked for me. I have no reproducible test/repo for you.

[alexander-akait]

@eemeli we don't touch anything with 3.1.9 around proxy

[emillynge]

@evilebottnawi Well, it was literally the only change in my setup. Maybe a dependency changed? in which case it would be helpful to track down which dep could cause this.

[alexander-akait]

@emillynge yep, so we need reproducible test repo, it is help to solve this issue

[victors1681]

My Payload is 512KB and I have the same issue. JSON it gets corrupts in webpack-dev-server Version: 3.1.11 - 3.1.14. The only one is working fine is 3.1.10

[alexander-akait]

@victors1681 please create minimum reproducible test repo, otherwise issue was closed and marked as spam

[edwint229]

I meet some issue in version 3.1.14, my payload is 27.5KB, enabled https proxy. After I downgrade to 3.1.10, it works fine also. I think the issue should happened in proxy side, please help to check and fixed. Thanks

[alexander-akait]

@edwint229 Can you provide minimum reproducible test repo?

[houfeng0923]

i found our api response data (large json ) is truncated . the api server support http2 and it only reproduced in http2 . we can force proxy server use http1.1 always:

https: {
key: ...
cert: ...
spdy: {
   protocols: ['http1.1']
 }
}

// waiting for  'http2'  module stable, and express support

[alexander-akait]

@houfeng0923 Can you create minimum reproducible test repo?

[dnicoara-psft]

I'm having the same issue with webpack-dev-server 3.2.1 The JSON response is truncated. Downgrading to 3.1.10 solves the issue. Could someone take a look at this, please?

[houfeng0923]

@evilebottnawi , i guess a http2 bug in node-spdy or http-proxy-middleware . webpack-dev-server depend on express , express use node-spdy to support http2 feature .

i ceate a spdy-http2/node-spdy#357 here.

[alexander-akait]

@houfeng0923 thanks for investigateion

[lzane]

same issue here with Node v8 update to Node v10 solve my problem

[yadue]

node v10 forces webpack-dev-server to use https server instead of spdy, I think just an option to select manually would be a solution in that particular case

[alexander-akait]

@yadue why?

[yadue]

for users who are not using nodejs 10 for some reasons it uses only spdy which in webpack-dev-server above 3.1.10 fails for some reasons for the guys here, it truncates response in some cases. there is no way for us to use latest versions of webpack-dev-server since we can't manually select which protocol we want to use

webpack-dev-server detects node v10 and runs https server which works just fine, node < 10, runs spdy library which above version 3.4.7 fails for some users

[alexander-akait]

@yadue it is not solution, we should find problem and fix it, no need implement hacks

[alexander-akait]

What is problem create minimum reproducible test repo?

[yadue]

here you go: https://github.com/yadue/webpack-dev-server-proxy-issue npm install npm run start open https://app.localhost:4201/ you'll see 2 requests proxied to http://yadue.synology.me/api3/2jcegul7ephjg2emsfg4v5097g.json (10kb) http://yadue.synology.me/api3/2jcegul7ephjg2emsfg4v5097g-small.json (2kb) https://imgur.com/a/sXfyiuY

it really took me so long to find reproducible case.