HTTP2 download of parallel requests is slower than HTTP1.1

[stanislav-halyn]

Version

v20.17.0

Platform

MacOS Sonoma 14.5, 2,3 GHz Quad-Core Intel Core i7

Subsystem

No response

What steps will reproduce the bug?

Define an http2 and http1.1 clients and fetch some data in parallel.

You can use the script below. If you run it using the following command, it will send 25 parallel requests using HTTP1.1

node client.js

or you can run this command and it will send 25 parallel requests using HTTP2:

node client.js http2

client.js

import http2 from 'node:http2';
import https from 'node:https';

const isHttp2 = process.argv[2] === 'http2';

const MB = 1024 * 1024;
const WINDOW_SIZE = 32 * MB;

const url = new URL('https://nodejs.org/dist/v17.1.0/node-v17.1.0-x64.msi');

const client = http2.connect(url, {
  settings: {
    initialWindowSize: WINDOW_SIZE,
  },
});

client.on('connect', () => {
  client.setLocalWindowSize(WINDOW_SIZE);
});

const fetchHttp1 = (id) => {
  return new Promise((resolve) => {
    const req = https.request({
        host: url.host,
        path: url.pathname,
        headers: {
          connection: 'keep-alive',
        },
      }, (res) => {
        res.on('data', () => {});
        res.on('end', () => {
          console.log(`Complete request with id: ${id}`);
          resolve();
        });
      },
    );

    req.end();
  });
};

const fetchHttp2 = (id) => {
  return new Promise((resolve) => {
    const req = client.request({
      ':path': url.pathname,
    });
    req.on('data', () => {});
    req.on('end', () => {
      console.log(`Complete request with id: ${id}`);
      resolve();
    });
    req.end();
  });
};

const main = async () => {
  console.log(`Starting requests using HTTP${isHttp2 ? '2' : '1.1'} protocol.`);

  const startTime = Date.now();

  await Promise.all(
    Array.from({ length: 25 })
      .fill(null)
      .map((_, index) => isHttp2 ? fetchHttp2(index) : fetchHttp1(index)),
  );

  console.log(`Requests complete. Completion time: ${(Date.now() - startTime) / 1000}s`);
};

main();

How often does it reproduce? Is there a required condition?

It's not quite stable, because sometimes http2 has the same performance or even better performance.

What is the expected behavior? Why is that the expected behavior?

I expect http2 to be faster or at least have the same performance as http1.1.

What do you see instead?

Instead, I often see that http1.1 requests are faster. Most of the times http2 took 25s to complete while http1.1 took 20s.

There's a problem, though, because those results are not consistent when using the URL I provided in the example.

But, the results are consistent with some internal APIs I'm working with. First, I thought there might be a problem with our internal APIs, but making 25 parallel requests to our internal APIs using curl has consistently the same performance on both http 1.1 and http2.

I ran curl with the following command:

curl --config ./config.txt --parallel --parallel-max 25 --http2

where config.txt looked like this:

url = "our-internal-api"
output = "/dev/null"
// repeated 25 times

Additional information

I was thinking the reason for such slowness would be TCP stall because http2 uses only 1 tcp connection, so I checked in wireshark how many tcp connections creates node vs curl when using http2, and both created only 1 tcp connection.

Also, I noticed node receives first 10 http2 requests randomly, but then sequentially, while http1 requests are all received randomly.

http2:

---

http1: