2.0.1: Add certificate verification to requests

[Biboba]

trafficstars

Describe the bug When creating a "gis" with 'verify_cert' to True, I am getting warnings from urllib3 to add certificate verification.

To Reproduce Steps to reproduce the behavior:

from arcgis.gis import GIS
ent_gis = GIS(portal_url, portal_username, portal_password, verify_cert=True, use_gen_token=True)

error:

urllib3\connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host 'portal.company.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings

Expected behavior No warning should appear out of the box and best security practice should be implemented. More info: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings

Platform (please complete the following information):

• OS: Windows 10
• Browser [e.g. chrome, safari]
• Python API Version: 2.0.1

Thanks !

[achapkowski]

Please verify the system have a proper/valid cert. It switches to unverify if the generateToken endpoint can't be verified.

What version of portal are you hitting?

[Biboba]

Thanks for your reply. I do have a valid certificate issued by Sectigo. Everything seems to be properly configured (https://www.sslshopper.com/ssl-checker.html: everything is green). I am hitting 11.0 but faced also the issue on 10.9.1

[jpTipton]

If you run a test from requests, does it successfully return a result when it verifies the cert?

[Biboba]

Using urllib3, on the same Portal for ArcGIS, I do not have any warning:

import urllib3
http = urllib3.PoolManager()
r = http.request('GET', portal_url + '/sharing/rest?f=json')
print(r.status, r.data)
200 b'{"currentVersion":"10.1"}'

[achapkowski]

You would need to test with requests though urllib3 is the underlying library of requests, there are differences in certificate verification.

import requests
session = requests.Session()
response = session.get(<url>)
print(response.json())

[Biboba]

And here you are:

[achapkowski]

We'll take a look for the 2.1.0 release.

[nanaeaubry]

This has been fixed and will be in the next release

[vaclavstepan]

I am still having this issue with arcgis package 2.3.0 as installed using pip in Python 3.11 on a M2 Mac. (Python installed via Homebrew.) Should this be fixed there?

[nanaeaubry]

@jtroe Do you have any suggestions?

[achapkowski]

@vaclavstepan can you post your error messages?

[vaclavstepan]

My apologies, I've explored this a bit more, I should have been more specific. I am using OAUTH 2.0 credentials obtained via AGOL Content/New content/Application. So I have client_id and client_secret.

As I'm using it on my physically secured device, I thought to omit deploying a server-side component for token generation and use GIS() with named parameters client_id and client_secret, e.g.:

from arcgis.gis import GIS
client_id = '...'
client_secret = '...'

gis = GIS(client_id = client_id, client_secret = client_secret)
print("Logged in to " + gis.properties.portalName)

This does work, however, emits warning:

/Users/monkey/Library/Python/3.9/lib/python/site-packages/urllib3/connectionpool.py:1099: 
InsecureRequestWarning: Unverified HTTPS request is being made to host 'www.arcgis.com'. 
Adding certificate verification is strongly advised. 
See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(

I am now aware that there is a more correct, second approach, to generate token first and use it subsequently:

from arcgis.gis import GIS
import requests

# Get token for the app using the client credentials
def get_token():
    params = {
        'client_id': '...',
        'client_secret': '...',
        'grant_type': "client_credentials"
    }
    request = requests.get('https://www.arcgis.com/sharing/rest/oauth2/token',
                          params=params)
    response = request.json()
    token = response["access_token"]
    return token

token = get_token()

gis = GIS(token = token)

This gives no warning, and does work OK.