HERO+: Files > 2GB are not downloaded completely

[waider]

This seems to be a limitation of the camera's built-in web server rather than anything to do with the python code. If you try to download a file >2GB from the camera it will stop - without an error - at the 2GB boundary. You can retrieve the rest of the file by specifying a byte range in the HTTP header. urllib.request.urlretrieve doesn't support easily support this. It might be easier to use the Requests library (it generally is easier, as far as HTTP access is concerned) but I've not looked into this and this code currently has no other hard dependencies outside core Python libraries.

(note that urllib.request.urlretrieve is also flagged as legacy in the Python docs with a warning that it might become deprecated in future versions)

I'm experimenting with a fix involving an Opener to inject the Range header, but I suspect it's going to be messy.

[KonradIT]

Hi,

Couple of things:

• I'm refactoring the code so it's cleaner, more readable and up to new standards. I chose urllib because it does not require an installation of anything to keep goprocam as lean as possible. But I'm open to switch to requests. See the things needed to be done here, so some things might break in future releases. (version 3.0.4 is now considered legacy). See my progress
• I don't have a HERO+ so can't test but have downloaded >2GB files from older cameras in the past (namely HERO5 and HERO6).

[waider]

Good to hear you're planning on refactoring - I was looking at that myself but if you've got plans in the works I'll hold off. I'd definitely recommend moving to Requests for all the HTTP interaction as it's both easier to read and easier to write.

The specific bug I'm reporting here may only be present on the HERO+, since that's an old and no longer updated camera. I've got what I think is a working fix but it's pretty ugly; there's a few more tweaks I'd like to make to it before I submit it, and if you're moving to Requests it might turn out to be completely redundant (haven't checked the Requests API for "resume file download")

[waider]

I've dug at this some more and it looks unfixable. I used curl to fetch the files; it stops at the 2GB mark without an error, and attempting to continue the download (curl -C-) results in a timeout from the camera. The same files can't be downloaded via the iPhone app either which definitely points to a limitation on the camera. The only way I can think of to get the whole file over wifi is if the clipping interface allows you to chop the file into smaller pieces to retrieve, but since my SD card is full I have no space to apply clipping.

[waider]

Whoops, didn't realise I had permission to close this - I was trying to close the pop-up window on the sprint / kanban view :-/