gaxios icon indicating copy to clipboard operation
gaxios copied to clipboard

feat!: `fetch` (POC)

Open danielbankhead opened this issue 10 months ago • 2 comments

A proof-of-concept for migrating to native fetch. However, there are a few limitations today.

No native proxy support

It is possible to enable proxy support, however customers would have to install unidici downstream. Including it in this library would make it far too large. Here are some numbers:

package version size (minified)
gaxios 6.5.0 38.3 kB
undici 6.5.0 402.6 kB
@google-cloud/storage 7.10.0 484.2 kB

Adding undici would double the size of a downstream dependency like @google-cloud/storage.

However, there is discussion on adding the ProxyAgent within Node:

  • https://github.com/nodejs/node/issues/43187

Workaround

Potential workaround/fallback for customers:

import {instance, request} from 'gaxios';
import {fetch, ProxyAgent} from 'undici';

instance.defaults.fetchImplementation = fetch;

await request({
  url: '', 
  dispatcher: new ProxyAgent();
})

Environment support

Additionally, it wasn't until recently that undici added env support for HTTP_PROXY, HTTPS_PROXY, & NO_PROXY and it will take time for it to propagate to Node.js releases:

  • https://github.com/nodejs/undici/pull/2994
    • Available in: https://github.com/nodejs/undici/pull/3147
      • Landed in Node v22.3.0: https://github.com/nodejs/node/blob/main/doc/changelogs/CHANGELOG_V22.md#22.3.0

Performance

The native fetch implementation is 26% - 28% slower than node-fetch according to undici's benchmarks:

  • https://github.com/nodejs/undici?tab=readme-ov-file#benchmarks
    • UPDATE: Fixed in Node 22+ https://github.com/nodejs/undici/issues/1203#issuecomment-2071685448, where it is roughly 42% faster than node-fetch

Returning Streams

Native fetch returns a ReadableStream ('Web Streams') instead of stream.Readable. This may require a rewriting downstream for many applications that currently expects a Node stream.

However, here's a simple way to resolve:

import {Readable} from 'node:stream';

Readable.fromWeb(res.body);

Additional Notes

https://nodejs.org/en/about/previous-releases

Fixes: https://github.com/googleapis/gaxios/issues/625 🦕

danielbankhead avatar Apr 19 '24 19:04 danielbankhead

Warning: This pull request is touching the following templated files:

  • README.md - README.md is managed by synthtool. However, a partials file can be used to update the README, e.g.: https://github.com/googleapis/nodejs-storage/blob/main/.readme-partials.yaml

Really cool to see and thank you for digging into the numbers, both size wise and speed wise. Seems like there is a lot of work that still needs to happen on the node side for this to be competitive with node-fetch.

ddelgrosso1 avatar Apr 22 '24 13:04 ddelgrosso1