SSLCertVerificationError when using urllib from Julia

[tcarion]

Hi!

When I'm executing this script in Julia:

using PyCall
py"""
import urllib.request as urlrq
import sys
print(sys.version)
print(sys.executable)
api_url = "https://dog-api.kinduff.com/api/facts"
resp = urlrq.urlopen(api_url)
print(resp.readlines())
"""

it fails on a URLError(SSLCertVerificationError)

3.8.3 | packaged by conda-forge | (default, Jun  1 2020, 17:55:00) 
[GCC 7.5.0]
/opt/anaconda3/bin/python3
ERROR: PyError ($(Expr(:escape, :(ccall(#= /home/tcarion/.julia/packages/PyCall/7a7w0/src/pyeval.jl:38 =# @pysym(:PyEval_EvalCode), PyPtr, (PyPtr, PyPtr, PyPtr), o, globals, locals))))) <class 'urllib.error.URLError'>
URLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)'))
  File "/home/tcarion/.julia/packages/PyCall/7a7w0/src/pyeval.jl", line 7, in <module>
    pynamespace(m::Module) =
  File "/opt/anaconda3/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/opt/anaconda3/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/opt/anaconda3/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/opt/anaconda3/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/opt/anaconda3/lib/python3.8/urllib/request.py", line 1393, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/opt/anaconda3/lib/python3.8/urllib/request.py", line 1353, in do_open
    raise URLError(err)

When I'm executing exactly the same directly with python (with the same python installation), everything works fine:

3.8.3 | packaged by conda-forge | (default, Jun  1 2020, 17:43:00) 
[GCC 7.5.0]
/opt/anaconda3/bin/python
[b'{"facts":["An African wolf dog known as the Basenji is the only dog in the world that cannot bark."],"success":true}']

I found a workaround by adding/replacing with these lines:

import certifi
import ssl
resp = urlrq.urlopen(api_url, context=ssl.create_default_context(cafile=certifi.where()))

The problem is that in my case urllib is used from another module, and I would like to find an easier workaround than changing the source code of the module. Would you have any other idea? Thanks in advance!

[tcarion]

Another workaround is to run

py"""
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
"""

but I would prefer to keep ssl certification, so I would be happy to hear if anyone has a better idea :relaxed:

[Roh-codeur]

if someone else faces this error, you can set this by:

ENV["SSL_CERT_FILE"]="/path/to/ssl/cert.pem"
ENV["SSL_CERT_DIR"]="/path/to//ssl/certs"

followed by: using Pkg; Pkg.build("PyCall")

hope this helps!

[stevengj]

Thanks! Setting up your environment variables is not something we can really do from the PyCall package…