what if i have a 10GB file to upload?

[fredwangwang]

Seems like the logic is going to cache the file in-memory, which is super expensive to a simple HTTP POST.

[sethgrid]

I've been moving across the county; a bit swamped. Any suggestions? PRs welcome :)

[fredwangwang]

sorry, i dont monitor my email that much...

Some thinkings: Provide additional layer to wrap some common file upload protocols. For example: plain post upload, multipart upload.

Then the wrapping layer can ask the user for raw file input instead of io.reader. The advantage of this is every time the upload fails and retries, the file can be rewind to the start of the stream. No data is lost, no need to store in-mem as well.

But talking this out loud here, it is probably not wise to keep retrying upload large files anyways. Some resumable upload protocol should be used instead of "brute force uploading". So adding a warning to stderr when detecting the user has a really large payload probably good enough?

[derekbassett]

Would it be possible in this case to detect an io.ReadSeeker? That way you could seek back to the beginning and read again?