fetch
fetch copied to clipboard
whatwg
Missing interface for uncompressed Content-Length when compression is used
In order to track progress of a fetch GET request(in a similar way to onprogress events of XMLHttpRequest) it is possible to read the body of a request with a ReadableStream and retrieve the total size of the request body from Content-Length header.
However, if the request is compressed using the Content-Enconding header the Content-Length does not give the total uncompressed size of the request body but the size of the compressed request body.
It would be nice to have an interface to get the total size of the uncompressed body in order to properly track the progress of a request. There is the FetchObserver(https://github.com/whatwg/fetch/issues/607) proposal but it doesn't seem to have gained any traction.
We do keep track of this in "decoded body size" (which is only exposed through Resource Timing at the moment) but that's incremented whenever we decode a chunk. I don't think we actually know the full size upfront as you would need for a progress indicator.
Would it be reasonable to expose the total number of pre-decompression bytes received so far? Something like response.body.receivedBytes, which would give the number of bytes received over the wire, which would be directly comparable to the Content-Length value.
As far as I know, it is not always possible to compute the expected total uncompressed payload size. With Content-Encoding: gzip it might be possible after receiving the gzip header, but it is not possible with Content-Encoding: deflate.
It seems reasonable in the abstract, but we'd need to figure out where to put that information. We cannot just stuff it on body I think, that's a ReadableStream and I'm not sure we'd want to create a subclass for this.
cc @domenic
With Content-Encoding: gzip it might be possible after receiving the gzip header, but it is not possible with Content-Encoding: deflate.
Sadly the gzip header doesn't contain size information. The gzip footer does, but that's not useful for calculating progress.
Putting the information on ReadableStream means it's not possible to use the convenience methods (e.g. json(), arrayBuffer(), blob()), as the body is locked to one reader. I'm not sure where the progress information should go, but ideally it would go somewhere that would work together with those convenience methods.
it could be handy to get ahold of the resource timing when making a request.
performance.getEntries() only returns completed requests.
if we could get an incomplete one then we could figure this out by ourself and it would be easier to know which one belongs to the actual request that you have made.
fetch(url, {
onTiming (resourceTiming) {
// do things with it.
}
})
Putting the information on ReadableStream means it's not possible to use the convenience methods (e.g. json(), arrayBuffer(), blob()), as the body is locked to one reader
you could always clone the response.
var clone = response.clone()
response.json().then(console.log)
clone.body.pipeTo(progressStream)
