fetch

fetch copied to clipboard

With request body streams and response body streams one would think it is now possible to use fetch for full duplex communication over HTTP, like in the example below:

const intervalStream = new ReadableStream({
  start(c) {
    let count = 0;
    const timer = setInterval(() => {
      console.log("sent!");
      c.enqueue("Hello\n");
      if (count === 5) {
        clearInterval(timer);
        c.close();
      }
      count++;
    }, 1000);
  },
}).pipeThrough(new TextEncoderStream());

const resp = await fetch("https://full-duplex-server.deno.dev/", {
  method: "POST",
  body: intervalStream,
});

console.log("foo");

const reader = resp.body.pipeThrough(new TextDecoderStream()).getReader();

while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  console.log(value);
}

console.log("done!");

In this example the server just pipes the incoming request body stream into the response body steam with no modification.

If full duplex communication was working, you would see "sent!" and "Hello\n" being logged in close proximity to each other, live. This is however not the case, because the promise returned from fetch does not resolve until the entire request body stream has been sent and closed.

The solution here would be to resolve the Promise<Response> returned from fetch before the request body stream has completed. To do this, body upload streaming has to happen in parallel. The main issue with the spec in this area is that the promise returned from fetch is expected to reject if a chunk of the request body stream is not a Uint8Array (behaviour is tested in https://staging.wpt.fyi/results/fetch/api/basic/request-upload.any.html).

Are there any plans to resolve this shortcoming before request body streaming is sthbilized in browsers? HTTP/2 and QUIC are very capable duplex stream protocols (HTTP/1 is theoretically also capable) and it would be a shame if that power was not exposed to the web platform.

To be clear, I am not explicitly vouching for duplex streams to make it into the initial request body uploading implementation in browsers, I just want to make sure that it would be possible to add down the line. Deno implements fetch w/ request body streaming, and our implementation is capable of full duplex too, but it has to deviate from the spec to allow that. The backing HTTP implementations for both Node.js and Servo are also capable of full duplex body streams.

See #229 for some history. Since then, I've learned that this would require a lot of work in browsers and it's not planned to be supported initially.

However, I do think we should allow for it in theory, so if you could elaborate on what prohibits it that would help.

so if you could elaborate on what prohibits it that would help

The promise returned from fetch does not resolve until the entire request.body ReadableStream has been read and uploaded. This is fundamentally incompatible with duplex streaming, as you can not get a handle to the response.body until you close request.body.

These tests demonstrate that issue: https://github.com/web-platform-tests/wpt/blob/master/fetch/api/basic/request-upload.any.js. It is expected that the promise returned from fetch rejects if a chunk that is pulled off the request.body is not a Uint8Array. Instead this error should be communicated via cancelling the request.body ReadableStream the given error.

Does the specification also have such issues?

Now that you mention that example though I realize it is rather tricky if not impossible to do well here and be forward compatible. I suspect that applications will have the same kind of dependencies and browsers changing this will break them.

Yeah, there seems some overlap with #95 / #538 in terms of upload/request processing.

cc @yoichio @yutakahirano

Does the specification also have such issues?

As far as I can tell, yes. The request body is sent synchronously before receiving response headers (not in parallel). See the "To transmit request’s body body, run these steps:" section in HTTP-network fetch step 8.5. Essentially what needs to happen is, that this needs to be changed to happen in parallel with receiving the response headers.

Now that you mention that example though I realize it is rather tricky if not impossible to do well here and be forward compatible.

To elaborate on this: many (likely most) web servers are programmed to only send back response headers after the request body has been received and processed (from now on referred to as category one servers). Because the promise returned from fetch would only resolve once the response headers are received. For the user the behaviour is that "fetch completes after the request body is fully sent". This behaviour would not change once full duplex streaming is implemented in browsers because the response headers are not received by the browser until the request body has been sent (because only then are they sent by the server).

For servers that send back response headers before the request body has been received (from now on referred to as category two servers), the promise returned from fetch would still only resolve once the request body is fully sent (because as you say, browsers do not support full duplex HTTP right now). Once browsers implement full duplex HTTP the promise returned from fetch would resolve after the response headers are received, regardless of the state of the request body. This is technically a minor change in behaviour, but one that could be marked as a spec compliance bug that Chrome, FF, and Safari have. The Deno or Servo implementations could implement full duplex right away.

Because the Python WPT test server probably falls into category one, the WPT I mentioned above is technically correct, but only under the assumption that the server sends headers after the response body is received. For cases where you need to handle both category one and category two servers you should handle the "cancellation" of the ReadableStream. This would be the only way to get notified of request body errors for category two servers.

---

TLDR, to solve this following would need to be resolved in the spec:

• [ ] Request body sending in HTTP-network fetch should happen in parallel to waiting for response body headers.

One question this brings up: if request body sending on HTTP-network fetch happens in parallel, how and where are errors during request body sending propagated to the user?

In the case where the caller is not using upload streams? I suppose they would need FetchObserver (#607).

Yeah. For users that are using upload streams it can be surfaced as a cancellation of the stream.

Maybe the error could be surfaced as a rejection of the promise returned by fetch if the response headers have not yet been received, or alternatively as an error on the response body stream. Sort of like how aborting fetch works, but with a custom error message (the error that request body upload errored with), instead of an AbortError.

This would introduce the constraint that request body uploading must not outlive response body download. Good chance that is already a HTTP spec constraint anyway, but not sure.

My view:

• HTTP is a half-duplex request/response protocol and should remain so
• WebSocket or WebTransport should be used where full-duplex is required. In particular, they have mitigations against request smuggling.
• My team would be responsible for implementing this in Chromium, and even if we decided we wanted to do it, we don't have the resources to do the necessary rearchitecting of the network stack. Even just upload streaming has proven more challenging than expected.
• The Fetch Standard should continue to reflect what browsers actually implement.

@ricea the assertion that HTTP is half-duplex seems in contradiction with it allowing the client to consume the response before it has transmitted the request body. I tend to agree with regards to the cost/benefit tradeoff and what Fetch should specify (although I would like for things to be cooler).

@ricea Some comments:

HTTP is not half duplex. It allows for simultaneous request body and response body streaming at a per request level. This is supported in both HTTP/1.1. and HTTP/2. Here are some discussions around the topic from 8 years ago: https://datatracker.ietf.org/doc/html/draft-zhu-http-fullduplex#section-2.1. Note that this is full duplex on a per request level, not full duplex on a connection level (the latter is only supported in HTTP/2, and browsers make ample use of it). Go, Node.js, and many other runtimes support full-duplex HTTP body streams.

Regarding request smuggling: full-duplex is not any more vulnerable to this than any other request body streaming (#966). Whatever the ultimate restrictions are that result from the #966 discussion, they would apply to full-duplex request body streaming too.

Regarding the effort required for implementation: it is very understandable that there are no resources to implement this now. My concern is that with the current spec wording there is also no way to implement it in the future, even if opinion on this matter changes. We are effectively locked into half-duplex for fetch forever. I think the specification should leave room for specifying this behaviour in the future - who knows what happens 5 years from now :-)

Also some non-browser engines implement the Fetch specification now, or are planning to in the future (Deno, Node.js, Cloudflare Workers). I think it is worth discussing if the specification should cater to these audiences too - I think APIs being the same across browsers and servers is a great thing that nearly no other language can provide. Also the Servo network stack is already capable of full-duplex HTTP.

Regarding the effort required for implementation: it is very understandable that there are no resources to implement this now. My concern is that with the current spec wording there is also no way to implement it in the future, even if opinion on this matter changes. We are effectively locked into half-duplex for fetch forever. I think the specification should leave room for specifying this behaviour in the future - who knows what happens 5 years from now :-)

That's not really how specifications work (at least in the WHATWG). Implementations aren't prohibited from implementing things because the spec doesn't support them. Instead, implementations go ahead and implement ahead of or in parallel with spec work, and then bring their findings back to the spec for discussion. If there's multiple implementers interested in shipping something, then we change the spec, around the same time as it ships.

So I don't think you should be worried about the fact that the spec reflects implementations. That's working as intended. The spec can change, if resources to implement this free up.

Having implemented full-duplex HTTP clients and servers for Ruby, I fully support @lucacasonato's position and think we should guide the fetch spec towards being full duplex.

If browsers can't support it, that's understandable, but if someone wants to implement a browser (or client) that does, it should not be prevented from being compatible with the spec here IMHO.

[Problem:] the promise returned from fetch does not resolve until the entire request body stream has been sent and closed. The solution here would be to resolve the Promise<Response> returned from fetch before the request body stream has completed

Agreed. That would also be the expected behavior for upload-only streams. I want to know whether the connection is properly established, the promise to resolve, and then start streaming. I.e.:

I would expect the promise to resolve in the exact moment when the upload stream can start.

For download streams, the connection and streaming body are 2 separate steps, which can be awaited separately. The same would make sense for upload.

So what API changes would we make here and how are they observable in half-duplex implementations? Would we end up with hanging promises? (I don't think the upload stream racing with the response headers would be a good forward-looking design. You don't want to depend on that with regards to forwarding the error.)

Is that all we'd need to change? Is that a change we can make? I suspect you can already run into this scenario with blobs and I/O errors, for instance.

I think it's worth exploring this a bit, but if it turns out the existing API has half-duplex baked in already than @yutakahirano's suggestion of a fullDuplex opt-in seems like the way to go. And we could make that throw today so it'll be forward-compatible a couple years from now.

To be clear, I don't think we need to resolve the promise returned from fetch() any sooner. Resolving it once we have the response headers in is fine. That's the signal. If you care about the request you monitor its stream.

[not] upload stream racing with the response headers

Yes, indeed, I would expect the response headers to have returned before the fetch() promise returns. Once the fetch() promise returns, I would immediately start sending stream data. That's a simple and logical API.

To be clear, I don't think we need to resolve the promise returned from fetch() any sooner. Resolving it once we have the response headers in is fine. That's the signal. If you care about the request you monitor its stream.

This is also how I've implemented it in Ruby and it works well. You can in theory implement streaming headers, but I'm not sure there is an actual good use case for it.

Now that #1457 has merged this becomes a feature request.

FYI, Chrome only shipping half duplex has caused definitely user confusion. Users are expecting full duplex (without knowing that is what it is called), and as such even when specifying duplex: "half" (without knowing why, Chrome just says you have to add it) they do not understand that they are only getting half duplex.

For example, this Deno user expected full duplex streaming and was disappointed / confused as to why only half duplex is working: https://www.reddit.com/r/Deno/comments/150wet7/why_doesnt_servetls_stream_data_from_a/

@yutakahirano Is Chrome open to reconsider its stance on implementing full duplex if we do the spec work & WPT tests?

I'd also be interested to hear from WebKit (@annevk) and Gecko (@valenting) on this issue

Chrome only shipping half duplex has caused definitely user confusion

I'm honestly not surprised by this, and I would have expected full duplex by default, and I support making this full duplex by default if possible.

@yutakahirano Is Chrome open to reconsider its stance on implementing full duplex if we do the spec work & WPT tests?

Yutaka no longer works on Chrome. My response is unchanged from https://github.com/whatwg/fetch/issues/1254#issuecomment-862810798.

@ricea HTTP/2 specifically advertises bi-directional streaming as a feature, and HTTP/1 makes no such assertions about being half-duplex AFAIK. Can you clarify your assertion "HTTP is a half-duplex request/response protocol and should remain so" - ideally with references to relevant RFCs if possible.

The response is not sent until the request (including body) has been completely received. That's what I mean by "half-duplex".

Smart people have been known to disagree about what the HTTP RFCs say on this point, so I am going to stay out of that rabbit hole.

Do you mind pointing us at the statements regarding "half-duplex" in the RFCs that "Smart people have been known to disagree about"?

@ioquatix I don't think that inquiry is a productive use of everyone's time. Fact is that in all browser engines full duplex would be a substantial undertaking and alternatives such as WebSockets and soon WebTransport are available. The benefits and demand for full duplex HTTP would have to be quantified quite a bit to convince anyone I suspect.