Bypass copying the request body when it is too large

[vinaypunera]

trafficstars

As of now the request body is by default read and copied to an in memory buffer. This behaviour increases the memory footprint in case we are uploading large data. Targeting use cases specific to passing large data in request body. Ideally, in case of large data we want to rely on http1.1 chunking or http2 streaming, for that we need to bypass reading the data in memory for efficiency.

This copying happens at two places in request middlewares(in sequence) -

1. https://github.com/go-resty/resty/blob/master/middleware.go#L445
2. https://github.com/go-resty/resty/blob/master/middleware.go#L208

Possible solution -

1. We can assume that for large data the request body will be an io.Reader and change code paths accordingly. We can introduce a flag notCopyRequestBody similar to notParseResponse

This issue is similar to - https://github.com/go-resty/resty/issues/495

[n0v3xx]

Think this is the same problem: https://github.com/go-resty/resty/issues/309

[vinaypunera]

It is similar, more towards the request body. But the idea is to not buffer data in memory.

[kevin19930919]

Recently try to read chunk-based data from a 3-party API, but need to fall back to using the standard net/http Looking forward to seeing resty support this feature.