renterd
renterd copied to clipboard
Stream uploads from http.ResponseWriter directly to host without buffering
The design of the RHP4 client allows us to directly pipe the body of the http.Request
all the way to the Stream
we use for uploading to a host.
This will only work if the client specifies the Content-Length
header but that should be the case for most uploads.
Uploads that don't will need to buffer the whole sector in memory.
As a result, we will be able to get rid of the latency before an upload actually starts since renterd
no longer needs to read 40MB per chunk into memory before starting the upload. We also don't lose precious time performing erasure coding and encryption before we start uploading since that will happen incrementally as we upload to the hosts.