Support for Azure AD Service Principal with SQLAlchemy

[FaresBadrCA]

Problem Summary:

When using Azure AD Service Principal, a client-id and client-secret are used to obtain a short-lived token. The token may be used in the connection string:

uri = f"databricks://token:{DATABRICKS_TOKEN}@{DATABRICKS_HOST}:443?http_path={DATABRICKS_HTTP_PATH}"
engine = create_engine(uri)

However, there is no mechanism to ensure that connections do not stay in the connection pool past the token expiration.

Proposed Solution:

Allow client_id and client_secret as connection arguments, and ensure connections do not stay open past token expiration.

url = f"databricks://{DATABRICKS_HOST}:443?http_path={DATABRICKS_HTTP_PATH}"    
engine = create_engine(url, connect_args = {client_id: CLIENT_ID, client_secret: CLIENT_SECRET, scopes: SCOPES  } )

[amirhessam88]

any updates on this ?