Swift's Range: header support is being used for the wrong purpose

swiftclient's chunk_size support is just a sized socket read, thus, it only matters the number of underlying function calls/context switches. And that's why there is a comment saying "if 'chunk_size' is defined, you must fully read the object's contents before making another request.", because otherwise you have data left in your socket.

But in the latest pyrax, the chunk read is implemented by utilizing the Range: header, thus, for each file, there will be N of chunks being sent, synchronously. So the comment I mentioned above does not apply, but it's not the actual problem. The problem is that, each request comes with its header overhead and server logic overhead, so that users who are using chunk read may experience significant performance regression with the latest version of pyrax. In addition, users who were not using this feature are being forced to send 2 requests -- one for stats, to get the object size, one to download the object, while any requests is not free.

The Range: header is not designed for the purpose of chunk read; it's designed for parallel download. Cloudfiles and its deployment has lots of work to do to handle each requests, but the service itself can handle numerous requests at the same time. By utilizing Range" header, one user can send several multiple requests asynchronously to download 1 file (you may need gevent/eventlet), but downloading the file synchronously is definitely not the purpose.