cellpose
cellpose copied to clipboard
fix: handle 403 Forbidden in download_url_to_file
What changed?
- Extended HTTP headers to fully simulate a browser request (added Accept, Accept-Language, etc.), solving 403 Forbidden caused by incomplete headers.
- Added retry logic for HTTPError: If the server rejects Accept-Encoding, automatically retry without that header.
- Ensured the target directory exists before saving files, avoiding path-related errors.
Why?
I encountered 403 Forbidden errors when downloading files—some servers block requests with incomplete headers. These changes make the download function more robust and compatible, fixing the access issue while improving error handling.
Tested on
- URLs that previously returned 403 (now download successfully).
- Paths with non-existent directories (auto-created and saved correctly).
The 403s were happening because the old request lacked browser-grade headers (Accept, Accept-Language, etc.). Servers often block “script” traffic. I added Chrome-like headers and a fallback that strips gzip when the server chokes on it.