pan-os-python icon indicating copy to clipboard operation
pan-os-python copied to clipboard
verified publisher icon PaloAltoNetworks

How can I ignore the certificate validation?

Open Seethaar opened this issue 7 years ago • 12 comments

I have a test firewall to try pandevice without any valid SSL certificate. In order to work with XML APIs of Palo Alto, I use curl -k -X .

I am trying to use pandevice.

pandevice.errors.PanURLError: URLError: reason: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)

I guess its warning on the SSL certificate. Is there a way, I can ignore the certificate checks?

Seethaar avatar Jul 02 '18 03:07 Seethaar

pandevice does not check certificates by default, so you should not get this error by default. Are you on Mac OSX or linux with an older OpenSSL, and if so, is it possible you're running into this TLS 1.0 issue?

http://pandevice.readthedocs.io/en/latest/readme.html#connect-to-pan-os-8-0-and-higher

btorresgil avatar Jul 02 '18 04:07 btorresgil

Hi Brian,

Thanks for getting back to me. I don't think its a TLS issue. when I issue a python command requests.get(url). I get the same error. I work around that by requests.get(url,verify=false).

Any thoughts?

Thanks,

AR

From: Brian Torres-Gil [email protected] Sent: Monday, 2 July 2018 2:48:27 PM To: PaloAltoNetworks/pandevice Cc: Ayyappan Ramanan; Author Subject: Re: [PaloAltoNetworks/pandevice] How can I ignore the certificate validation? (#120)

pandevice does not check certificates by default, so you should not get this error by default. Are you on Mac OSX, and if so, is it possible you're running into this TLS 1.0 issue?

http://pandevice.readthedocs.io/en/latest/readme.html#connect-to-pan-os-8-0-and-higher

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/PaloAltoNetworks/pandevice/issues/120#issuecomment-401671600, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AZwD3vHzufD4H8SzpRHsS10qEthGPahUks5uCaYbgaJpZM4U-oLG.

Seethaar avatar Jul 02 '18 08:07 Seethaar

Considering we use pandevice in-house on self-signed certs all-day every-day, I'm certain that it doesn't verify certs by default. I'm confident there is something in your OS python that is causing this problem.

  • What OS are you running and what version of the OS?
  • What version of python? python -V
  • What version of OpenSSL? python -c 'import ssl; print(ssl.OPENSSL_VERSION)'
  • Did this python come with your OS or did you install it?

btorresgil avatar Jul 02 '18 16:07 btorresgil

Hi Brian,

  • The OS is CentOS Linux release 7.5.1804
  • Python version 2.7.5
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/site.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/site.py
import site # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/site.pyc
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/os.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/os.py
import os # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/os.pyc
import errno # builtin
import posix # builtin
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/posixpath.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/posixpath.py
import posixpath # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/posixpath.pyc
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/stat.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/stat.py
import stat # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/stat.pyc
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/genericpath.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/genericpath.py
import genericpath # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/genericpath.pyc
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/warnings.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/warnings.py
import warnings # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/warnings.pyc
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/linecache.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/linecache.py
import linecache # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/linecache.pyc
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/types.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/types.py
import types # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/types.pyc
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/UserDict.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/UserDict.py
import UserDict # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/UserDict.pyc
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/_abcoll.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/_abcoll.py
import _abcoll # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/_abcoll.pyc
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/abc.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/abc.py
import abc # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/abc.pyc
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/_weakrefset.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/_weakrefset.py
import _weakrefset # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/_weakrefset.pyc
import _weakref # builtin
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/copy_reg.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/copy_reg.py
import copy_reg # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/copy_reg.pyc
import encodings # directory /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/encodings
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/encodings/__init__.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/encodings/__init__.py
import encodings # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/encodings/__init__.pyc
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/codecs.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/codecs.py
import codecs # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/codecs.pyc
import _codecs # builtin
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/encodings/aliases.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/encodings/aliases.py
import encodings.aliases # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/encodings/aliases.pyc
# /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/encodings/utf_8.pyc matches /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/encodings/utf_8.py
import encodings.utf_8 # precompiled from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/encodings/utf_8.pyc
Python 2.7.5 (default, Apr 11 2018, 07:36:10)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
dlopen("/home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/lib-dynload/readline.so", 2);
import readline # dynamically loaded from /home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib64/python2.7/lib-dynload/readline.so

import ssl; print(ssl.OPENSSL_VERSION) OpenSSL 1.0.2k-fips 26 Jan 2017

  • This python version was shipped with the OS.
  • I am running this is a virtenv

$ pip freeze certifi==2018.4.16 chardet==3.0.4 idna==2.7 pan-python==0.13.0 pandevice==0.6.3 requests==2.19.1 urllib3==1.23

  • The way I run it

fw = firewall.Firewall("192.168.0.1","admin","p@ssword") print fw.op("show system info", xml=True)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib/python2.7/site-packages/pandevice/firewall.py", line 180, in op
    return super(Firewall, self).op(cmd, vsys, xml, cmd_xml, extra_qs, retry_on_peer)
  File "/home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib/python2.7/site-packages/pandevice/base.py", line 3295, in op
    element = self.xapi.op(cmd, vsys, cmd_xml, extra_qs, retry_on_peer=retry_on_peer)
  File "/home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib/python2.7/site-packages/pandevice/base.py", line 3277, in xapi
    self._xapi_private = self.generate_xapi()
  File "/home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib/python2.7/site-packages/pandevice/firewall.py", line 191, in generate_xapi
    return super(Firewall, self).generate_xapi()
  File "/home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib/python2.7/site-packages/pandevice/base.py", line 3319, in generate_xapi
    kwargs = {'api_key': self.api_key,
  File "/home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib/python2.7/site-packages/pandevice/base.py", line 3271, in api_key
    self._api_key = self._retrieve_api_key()
  File "/home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib/python2.7/site-packages/pandevice/base.py", line 3405, in _retrieve_api_key
    xapi.keygen(retry_on_peer=False)
  File "/home/aramanan/code/acme-support-netscripts/PANDEVICE/env/lib/python2.7/site-packages/pandevice/base.py", line 3173, in method
    raise the_exception
pandevice.errors.PanURLError: URLError: reason: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)
  • Additionally running the same piece of code on a docker container, hits the same issue.

Any thoughts?

AR

Seethaar avatar Jul 05 '18 04:07 Seethaar

Thanks for sending this info, very helpful. I agree that it doesn't seem like a TLS 1.0 issue. pandevice doesn't actually make connections to the firewall since it is focused on abstracting the XML schema into objects. It relies on pan-python library for the connectivity. Let me run this by @kevinsteves, author of pan-python.

btorresgil avatar Jul 05 '18 19:07 btorresgil

@Seethaar What was the version of python in the docker container? Was it also 2.7.5?

btorresgil avatar Jul 05 '18 20:07 btorresgil

Also, can you test with one of these python docker images: https://hub.docker.com/_/python/ For example, try with tag 2.7.15-alpine3.7. Thanks!

btorresgil avatar Jul 05 '18 21:07 btorresgil

Thanks Brian,

Yes. I am using docker 2.7.5. I tried a docker pull on that image (alpine) and I realised it does not contain the 'pandevice' module by default.

I am trying to learn how to write a docker file to install that on that image. Will get back to you when I succeed.

But If you have the dockerfile already, Could I trouble your to share that please?

Thanks,

AR

From: Brian Torres-Gil [email protected] Sent: Friday, 6 July 2018 7:22:58 AM To: PaloAltoNetworks/pandevice Cc: Ayyappan Ramanan; Mention Subject: Re: [PaloAltoNetworks/pandevice] How can I ignore the certificate validation? (#120)

Also, can you test with one of these python docker images: https://hub.docker.com/_/python/ For example, try with tag 2.7.15-alpine3.7. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/PaloAltoNetworks/pandevice/issues/120#issuecomment-402856635, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AZwD3os-6YWxAq1_3yoS4lBJ-4GCq2jeks5uDoOygaJpZM4U-oLG.

Seethaar avatar Jul 06 '18 02:07 Seethaar

Rather learning dockerfile, I pulled alpine linux from docker hub and install python2.7.15 on it and imported pandevice module.

It just works perfectly! I can use this for the interim.

My Prod is Centos 7 though, is there way I could get that working ignoring the certificate warning?

Thanks a lot for your time Brian.

AR

From: Brian Torres-Gil [email protected] Sent: Friday, 6 July 2018 7:22:58 AM To: PaloAltoNetworks/pandevice Cc: Ayyappan Ramanan; Mention Subject: Re: [PaloAltoNetworks/pandevice] How can I ignore the certificate validation? (#120)

Also, can you test with one of these python docker images: https://hub.docker.com/_/python/ For example, try with tag 2.7.15-alpine3.7. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/PaloAltoNetworks/pandevice/issues/120#issuecomment-402856635, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AZwD3os-6YWxAq1_3yoS4lBJ-4GCq2jeks5uDoOygaJpZM4U-oLG.

Seethaar avatar Jul 07 '18 04:07 Seethaar

@Seethaar Thanks for testing. This supports @kevinsteves theory which is that your centos has some patched version of python 2.7.5, not the normal python 2.7.5. Python validates SSL certificates by default staring in python 2.7.9, so pan-python checks the version of python before making a connection. If it finds python < 2.7.9, it takes no specific action to validate certs. If it finds python >= 2.7.9 then it specifically does not validate a cert by default. It seems that your python 2.7.5 is patched to validate certs by default, which is different than normal python 2.7.5. So our recommendation as a long term solution is to upgrade to python 2.7.9 or higher, or use docker where you can leverage any python you want. Let me know if you have any questions on this.

btorresgil avatar Jul 09 '18 14:07 btorresgil

I was able to get around this issue by setting the following environment variable. It's obviously not very secure, but it works in a pinch when you cannot upgrade python.

export PYTHONHTTPSVERIFY=0

stealthllama avatar Dec 30 '18 01:12 stealthllama

After looking through pan-python, it has code to run with or without the requests module installed, and the code that ignores certificates is conditional based on whether or not requests imports without throwing an exception.

You might be able to override the issue by adding these lines to your script in Python3:

import urllib3

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

For Python 2, try using "urllib" or "'urllib2"

bsucevic avatar Nov 28 '19 02:11 bsucevic