Tom Russell

Perhaps we can gain some ideas from https://github.com/GoogleCloudPlatform/gsutil/tree/master/gslib on how that tool handles transferring files?

No idea but looks like it can be closed now.

@eliasdorneles are you ok with this being closed now with the stream code now in place?

@raphapassini After reviewing this a bit there are some changes still needed in order for this to work. The code looks fine. I am a little confused on some of...

@manugarri we are planning to update exporters to python 3 soon, if possible could you make sure your code reflects that? Thanks

No worries, we will fix this.

yw @manugarri, we should have the Python 3 version ready by early next week, I agree this PR should be able to be added in then. Thanks for your contribution.

@tribeclickgh There is a python 3 branch at https://github.com/scrapinghub/exporters/tree/python3 that is closer to being complete but we have not had time to finish it, your more than welcome to do...

I agree, this would be a very useful thing for some exports.