How to set the content length header?

[phmLabs]

Hi,

I want to add the content-length header if it is not already set. Is this possible?

Background: We are using Puppeteer to gather information from websites. Pupeeteer/Chrome is not able to return the actual transferred size of a document because it can only see the unzipped version. We need the size of the compressed version.

Greets Nils

[bjowes]

Hi @phmLabs ,

If your only prupose is to check the number of bytes in the compressed document, you can see that in Chrome Devtools on the network tab. Size is the number of bytes transferred on the network (includes headers/cookies), while Content is the uncompressed size.

https://stackoverflow.com/questions/8072921/chrome-dev-tools-size-vs-content

[phmLabs]

Thx @bjowes but we have to do it automatically because it's for thousands of websites.

[bjowes]

Ok, @phmLabs, thanks for clarifying. The http-mitm-proxy will delete the content-length header if it is present in the response from the server (code here). I don't know the full background of this, but I presume this was the easiest way to handle that the proxy may modify the response body before passing it to the client - hence the acutal lenght may have changed.

Are you required to use "content-length" specifically? Otherwise you can attach a onReponse handler which simply copies the content-length header into another header, for instance "x-content-length". This header will then not be deleted by the proxy.

However, if your server doesn't set the content-length header at all, I believe you are out of luck. The proxy processes the headers first and forwards them to the client. Then the data chunks are processed and forwarded to the client. So even if you would setup onResponseData handlers to count the number of bytes in the body, by the time you have the answer, the headers were already sent to the client.