ssl.SSLError when downloading coastlines...

[SpacelySpaceSprockets]

I have a brand new windows 10 computer with a brand new Anaconda installation on it. I'm trying to run some example cartopy code and I'm getting an error (full traceback below...:

ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)

I don't usually work on Windows, and the code works fine on my linux and osx machines. What do I need to do to get this working on windows 10?

Here is the example code I'm trying to run from the cartopy gallery

    import matplotlib.pyplot as plt
    
    import cartopy.crs as ccrs
    
    
    def main():
        fig = plt.figure(figsize=(10, 5))
        ax = fig.add_subplot(1, 1, 1, projection=ccrs.Robinson())
    
        # make the map global rather than have it zoom in to
        # the extents of any plotted data
        ax.set_global()
    
        ax.stock_img()
        ax.coastlines()
    
        ax.plot(-0.08, 51.53, 'o', transform=ccrs.PlateCarree())
        ax.plot([-0.08, 132], [51.53, 43.17], transform=ccrs.PlateCarree())
        ax.plot([-0.08, 132], [51.53, 43.17], transform=ccrs.Geodetic())
    
        plt.show()
    
    
    if __name__ == '__main__':
        main()

here is the complete traceback:

C:\Users\webster\programming\python\cartopy>python test1.py C:\Users\webster\Anaconda3\envs\infrapy_env\lib\site-packages\cartopy\io_init_.py:260: DownloadWarning: Downloading: https://naciscdn.org/naturalearth/110m/physical/ne_110m_coastline.zip warnings.warn('Downloading: {}'.format(url), DownloadWarning) Traceback (most recent call last): File "C:\Users\webster\Anaconda3\envs\infrapy_env\lib\urllib\request.py", line 1349, in do_open encode_chunked=req.has_header('Transfer-encoding')) File "C:\Users\webster\Anaconda3\envs\infrapy_env\lib\http\client.py", line 1272, in request self._send_request(method, url, body, headers, encode_chunked) File "C:\Users\webster\Anaconda3\envs\infrapy_env\lib\http\client.py", line 1318, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "C:\Users\webster\Anaconda3\envs\infrapy_env\lib\http\client.py", line 1267, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "C:\Users\webster\Anaconda3\envs\infrapy_env\lib\http\client.py", line 1038, in _send_output self.send(msg) File "C:\Users\webster\Anaconda3\envs\infrapy_env\lib\http\client.py", line 976, in send self.connect() File "C:\Users\webster\Anaconda3\envs\infrapy_env\lib\http\client.py", line 1433, in connect server_hostname=server_hostname) File "C:\Users\webster\Anaconda3\envs\infrapy_env\lib\ssl.py", line 407, in wrap_socket _context=self, _session=session) File "C:\Users\webster\Anaconda3\envs\infrapy_env\lib\ssl.py", line 817, in init self.do_handshake() File "C:\Users\webster\Anaconda3\envs\infrapy_env\lib\ssl.py", line 1077, in do_handshake self._sslobj.do_handshake() File "C:\Users\webster\Anaconda3\envs\infrapy_env\lib\ssl.py", line 689, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)

[dopplershift]

Seems like your SSL setup isn't happy on Windows. You could try running this script. Otherwise, the solution is going to be to download the shape files manually and put them in the right directory.

[Sinhika]

I ran into the same problem on Linux. Cartopy does not open https urls correctly on Python 3.8. Y'all need to fix the cartopy.io.Downloader method _urlopen() to pass an ssl context to urllib.request.urlopen() when it calls it with an https url in Python 3.8 or later--see the Python 3.8 docs for details. Spent most of the day chasing this one down at work.

ETA: running tools/feature_download.py won't help at all, it calls the same bugged method.

[SpacelySpaceSprockets]

I'm using 3.6.11, but maybe the same issue? I tried the feature_download.py and it did give the exact same error.

[dopplershift]

@Sinhika Works completely fine for me on Python 3.8 on my mac and on our Linux and Windows CI, so my best guess is that any SSL problems are related to your system configuration. According to the Python 3.8 docs, context can be None and it will call ssl.create_default_context() for you--so it's by no means required for SSL. So...I'm going to bristle at "bugged" without a test case that fails outside your environment.

I'd be happy to review and merge a PR plumbing in the capability for users to pass a custom value for context if that solves the problem for you.

[dopplershift]

@SpacelySpaceSprockets What does this code do on your system?

from urllib.request import urlopen
fobj = urlopen('https://naciscdn.org/naturalearth/110m/physical/ne_110m_coastline.zip')

or how about:

from urllib.request import urlopen
import ssl

fobj = urlopen('https://naciscdn.org/naturalearth/110m/physical/ne_110m_coastline.zip',
               context=ssl.create_default_context())

[SpacelySpaceSprockets]

@dopplershift same error for both, same as above

[Sinhika]

Me, too. This, however, works fine--probably because it bypasses cert checking:

fobj = urlopen('https://naciscdn.org/naturalearth/110m/physical/ne_110m_coastline.zip',
                         context=ssl.SSLContext())

Alas, at work anything related to networking is locked down, full of security weirdness, and probably not something I can get changed.

[SpacelySpaceSprockets]

@Sinhika that works for me. What are the security implications of doing it this way?

[dopplershift]

I'm not quite sure what the differences are, but there are some more workaround on this cpython issue.

[xylar]

We're seeing this on conda-forge. It's happening consistently but only for Windows and python 3.7. Very, very odd. https://github.com/conda-forge/cartopy-feedstock/pull/125

[xylar]

Not sure our problem is the same. We're not seeing CERTIFICATE_VERIFY_FAILED, just an unknown error. I'm leaning toward our issue being related to the new openssl 3.0.0.

[jjakenichol]

@Sinhika Works completely fine for me on Python 3.8 on my mac and on our Linux and Windows CI, so my best guess is that any SSL problems are related to your system configuration. According to the Python 3.8 docs, context can be None and it will call ssl.create_default_context() for you--so it's by no means required for SSL. So...I'm going to bristle at "bugged" without a test case that fails outside your environment.

I'd be happy to review and merge a PR plumbing in the capability for users to pass a custom value for context if that solves the problem for you.

I am having this same issue on a mac with python 3.9.

Seems like your SSL setup isn't happy on Windows. You could try running this script. Otherwise, the solution is going to be to download the shape files manually and put them in the right directory.

The link for that script is broken, what was it? Where can I download those shape files manually?

[jjakenichol]

Found the shapefile URLs with the cartopy_feature_download.py script. I haven't figured out the right directory where they need to reside.

[jjakenichol]

I never figured out where to place the manually downloaded files. I did end up solving my problem though with these two SE links:

https://stackoverflow.com/questions/27835619/urllib-and-ssl-certificate-verify-failed-error

https://stackoverflow.com/questions/44649449/brew-installation-of-python-3-6-1-ssl-certificate-verify-failed-certificate/44649450#44649450

urllib must be looking for SSL certs in a different place than requests.