Fetch streaming upload

[yutakahirano]

Request for position on fetch streaming upload

• WebKittens who can provide input: @youennf

Information about the spec

• Spec Title: Fetch
• Spec URL: https://fetch.spec.whatwg.org/
• GitHub repository: https://github.com/whatwg/fetch/
• Issue Tracker (if not the repository's issue tracker): https://github.com/whatwg/fetch/issues/1438
• Explainer (if not README.md in the repository): https://github.com/yutakahirano/fetch-with-streams/blob/master/streaming-upload.md

Design reviews and vendor positions

• TAG Design Review: https://github.com/w3ctag/design-reviews/issues/754
• Mozilla standards-positions issue: https://github.com/mozilla/standards-positions/issues/663

Bugs tracking this feature

• WebKit Bugzilla: N/A
• Radar: N/A

Anything else we need to know

Streaming upload allows web developers to attach a ReadableStream to a Request. The web developer can pass the Request to fetch() to upload the ReadableStream.

[jakearchibald]

I'd like developers have a way to synchronously feature detect this feature. My current recommendation is:

const supportsRequestStreams = (() => {
  let duplexAccessed = false;

  const hasContentType = new Request('', {
    body: new ReadableStream(),
    method: 'POST',
    get duplex() {
      duplexAccessed = true;
      return 'half';
    },
  }).headers.has('Content-Type');

  return duplexAccessed && !hasContentType;
})();

This tests that:

• duplex is implemented as part of RequestInit
• body can be a ReadableStream. Otherwise, it's converted to a string and gets a text/plain content type.

Safari has already implemented streams as bodies of Request, but it rejects if this is passed to fetch().

Could Safari avoid implementing duplex until streams as request bodies are fully implemented (including the fetch parts)? Otherwise the feature detect will get messy and asynchronous.

duplex is only currently useful with this feature.

[annevk]

@jakearchibald now that Yutaka left Chrome, is there someone else picking this up? I suppose it has shipped already, but it seems that duplex should probably also be added to the Request object as per https://github.com/whatwg/fetch/issues/1486.

I tend to agree with you with regards to feature detection, does WPT verify that by chance?

[jakearchibald]

I'll pick this up, and https://github.com/whatwg/fetch/issues/1486

What's the right way to write a WPT for this? Something like:

try {
  await fetchWithStreamBody();
  return;
} catch (err) {
  assert(!supportsRequestStreams);
}

[annevk]

Yeah, that looks correct. I suppose we'll only need this test while engines are still implementing this so maybe add a comment in case someone wonders what's up in the future.

[jakearchibald]

PR up: https://github.com/web-platform-tests/wpt/pull/36048

[annevk]

Colleagues and I have looked at this and are supportive. If there is no further feedback from the WebKit community by October 10 I suggest we label this as "position: support".

[HazhoHuman]

Hey awesome folks; any updates about this, what is the destiny of it? can we safely implement it without worrying about backward compatibility if major changes happen later??

[annevk]

@HazhoHuman I'm afraid I don't understand the question. Could you provide more context?

[hawkinsw]

I'd like developers have a way to synchronously feature detect this feature. My current recommendation is:

const supportsRequestStreams = (() => {
  let duplexAccessed = false;

  const hasContentType = new Request('', {
    body: new ReadableStream(),
    method: 'POST',
    get duplex() {
      duplexAccessed = true;
      return 'half';
    },
  }).headers.has('Content-Type');

  return duplexAccessed && !hasContentType;
})();

This tests that:

* `duplex` is implemented as part of `RequestInit`

* `body` can be a `ReadableStream`. Otherwise, it's converted to a string and gets a `text/plain` content type.

Safari has already implemented streams as bodies of Request, but it rejects if this is passed to fetch().

Could Safari avoid implementing duplex until streams as request bodies are fully implemented (including the fetch parts)? Otherwise the feature detect will get messy and asynchronous.

duplex is only currently useful with this feature.

@jakearchibald Thank you so much for this! We are using this detection code in the go wasm backend!

We see that there is a problem with non-browser environments. In particular, the behavior for the Request constructor with an empty input string depends on the fact that the constructor has a this with a "relevant settings object" that has a "good" API base URL. For browsers, we are fine (see here). For node (and deno), this is not the case. The quick fix seems to be simply to put a simple URL as the input for the Request constructor (e.g., https://www.example.org/).

There is another issue with this detection code that is specific to a bug in deno's fetch implementation, but we are trying to fix that (see here).

Again, thank you so much for drawing this up!!

Sincerely, Will

edited on 12/27/2022 at 1907 EST.

[hawkinsw]

Hello -- I am terribly sorry that my comment was unhelpful. I was attempting to respond to @jakearchibald 's awesome note above. Again, I sincerely apologize for the noise.

[hober]

Closing as we've identified our position.

[jtbandes]

Hi, just wondering if there is another issue or bugzilla ticket tracking the implementation of this feature? I was not able to find anything in a quick search.