Limit the response size for HTTP calls

[artpolikarpov]

Yesterday we faced an issue where we would send a request to a client’s webhook endpoint and the endpoint would respond with an error and a huge body (couple of megs). The amount of requests to that endpoint was pretty large, plus we retry unsuccessful webhooks up to 10 times, so the overall amount of requests was huge, and our memory started to fill with those response bodies (which we don’t even use in any way), eventually crashing the container.

It would be great if Meteor’s HTTP package had an option to limit the response size (aborting the connection after the response length exceeds the limit). I know that underlying npm request package doesn’t have this option either, but it at least allows to handle data chunks while the response is arriving and abort the connection at any moment. This could be used to implement such an option in HTTP package. Here’s an example of response length limiting using npm request package: https://stackoverflow.com/a/21577874.

[StorytellerCZ]

I think with the deprecation of the HTTP package this is a mute point, unless we need to implement it in fetch as well.