cget icon indicating copy to clipboard operation
cget copied to clipboard

Replace CGetURLOpener with urlretrieve

Open pcf000 opened this issue 2 years ago • 2 comments

For reasons I have not entirely traced, URLopener (and thus FancyURLopener) does not work with some proxies. I can see that instead of setting up a tunnel with a CONNECT request, it makes a GET request to the proxy, which in my case induces a 502-bad-gateway response. Even though it recognises the proxy settings, it doesn't quite do the right thing.

However, urlopen uses ProxyHandler to accommodate the situation, and does work through this troublesome proxy. urllib.request.urlretrieve has the same interface as URLopener.retrieve, and seems like a drop-in replacement. I added a try/except around the urlretrieve to get the BuildError for responses of 400 or greater, since there is no longer a custom http_error_default or a good place to hang it.

This change addresses my internal build problem and probably also issues #52 and #77. Plus, no more deprecation warnings.

pcf000 avatar Jun 23 '22 03:06 pcf000

Hi @pfultz2 , may I ask why this is not merged yet? It certainly sorted the issues I was facing with an internal proxy :)!

giuseros avatar Dec 06 '23 10:12 giuseros

There is failures in ubuntu CI, if you can merge the latest from master which updated it to 20.04.

pfultz2 avatar Dec 06 '23 15:12 pfultz2