readthedocs.org
readthedocs.org copied to clipboard
readthedocs
HTTP headers don't have Content-Length or Transfer-Encoding:chunked?
For example run curl -I https://requests.readthedocs.io/en/latest/
I see there is no Content-Length or Transfer-Encoding:chunked in the HTTP headers. I don't know if it's by design?
Although the spec allows this, I think it's not normal.
Hi @imba-tjd! I wasn't aware of this. I'm not sure why we are not sending Content-Length, but I'm not sure how useful it is either. Do you know some particular use case where this header would be helpful?
On the other hand, Transfer-Encoding: chunked seems to not be supported by HTTP/2, which is the protocol we are using:
Note: HTTP/2 doesn't support HTTP 1.1's chunked transfer encoding mechanism, as it provides its own, more efficient, mechanisms for data streaming.
Let me know if this clarifies the situation and if there is any actionable from our side that would be useful.
Adding the documentation reference:
- Content-Length: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Length
- Transfer-Encoding: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding
I found this because I have a L7 proxy program that firstly checks target's Content-Length. If it's too large, it refuse to response.
This appears to be something that Cloudflare is doing because it's streaming the responses: https://support.cloudflare.com/hc/en-us/articles/200168386-Why-is-my-dynamic-content-being-sent-with-chunked-encoding-
Note -- it isn't possible to have chunked encoding and content-length at the same time so this would explain why content-length would not be sent with these dynamic resource passing through Cloudflare.
We can work to enable this, but we'll have to look at what this would do to performance of larger pages.
Is this something we are still considering trying out or we can just close this issue? I'm not seeing too much value on it, honestly.