exchangelib Better support for Modern Authentication

According to https://techcommunity.microsoft.com/t5/exchange-team-blog/upcoming-changes-to-exchange-web-services-ews-api-for-office-365/ba-p/608055, Basic Authentication is going away in Office365 as of ~October 13, 2020~ second half of 2021.

We should see if we can provide a better authentication user experience than fiddling with OAuth tokens, if possible. There's a library for ADAL at https://github.com/AzureAD/azure-activedirectory-library-for-python that may be useful.

May 03 '20 07:05 ecederstrand

I would suggest using MSAL over ADAL to prevent a 'future legacy' implementation. ADAL is supported but no longer under active development.

there is a supported and maintained MSAL library for Python :
https://docs.microsoft.com/en-us/azure/active-directory/develop/reference-v2-libraries#microsoft-supported-client-libraries

Sep 07 '20 10:09 Josverl

I'm sorry for the bump but is there any news on implementation?

Dec 17 '21 11:12 driesken

No, I don't have a personal need for this and am not actively working on this. As always, contributions are welcome.

Dec 20 '21 11:12 ecederstrand

Hi, just a polite request to implement this. It's really close to the term: https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-deprecation-in-exchange-online-september/ba-p/3609437

Thank you.

Sep 05 '22 08:09 ALutchko

exchangelib already has support for OAuth, so this issue is not really urgent anymore. If I understand MSAL correctly, it's just a more structured way of handling OAuth credentials.

Sep 05 '22 08:09 ecederstrand

Absolutely, I can confirm that exchangelib works fine with MSAL.

Sep 05 '22 08:09 ljnsn

@ljnsn Can you provide a short example or description here of MSAL in combination with exchangelib? Then I can add it to the docs and close this issue.

Sep 07 '22 07:09 ecederstrand

@ecederstrand Regarding OAuth: My organization's security policy is to disallow client shared secret, where we are only able to use client certificate for credentials. The OAuth lib used does not allow client certificate, where MSAL does. I would assume other organizations may also only allow client certificate credentials as well.

Note that O365 project also has the same issue (a branch was created with MSAL client certificate - see issue https://github.com/O365/python-o365/issues/570 )

Note that I already migrated to using Office365-REST-Python-Client ( Graph API / MSAL ) to get newer API and Client Certificate auth flow. Hopefully @ljnsn will post some code.

Sep 07 '22 16:09 mikkelcp1

Unfortunately, I cannot post actual code due to company policy, but we use MSAL with delegated device flow to retrieve an access token for a user and then use that token with OAuth2AuthorizationCodeCredentials provided by exchangelib. That's pretty much it. It's not certificate based though, so you do need client id and client secret.

Sep 08 '22 21:09 ljnsn

I'm closing this issue, at is seems nothing actually needs to change in this library. If someone wants to write up an example, feel free to post it here or open a PR against the docs so others can benefit.

Sep 12 '22 05:09 ecederstrand

An example on how to nicely pass a raw token to the library, without any "magic" that keeps it alive internally (usually msal handles renewal when you ask it to give you the token back from its cache and it expired), would be great...

Apr 21 '23 14:04 ThiefMaster

The general idea is to override the .refresh() method and implement your own token refresh code. See https://ecederstrand.github.io/exchangelib/#oauth-authentication

class MsalCredentials(OAuth2AuthorizationCodeCredentials):
    def refresh(self):
        # Add actual code here to get new token from small
        self.access_token = msal.get_token_from_cache(...)

It would be great if you can provide a concrete example for msal. I can then add that to the documentation.

Apr 24 '23 08:04 ecederstrand

Ah, that makes sense. Would it help if we changed this code, so it checks whether the Credentials object already has an access_token set and then avoid calling session.fetch_token()? Something like:

    def create_oauth2_session(self):
        session = self.raw_session(...)
        if self.credentials.access_token and not session.token:
            session.token = self.credentials.access_token
        elif not session.token:
            token = session.fetch_token(...)
            self.credentials.on_token_auto_refreshed(token)
        ...

Apr 24 '23 10:04 ecederstrand

This seems to work (copied together from multiple files but within my app it works fine):

import sys
from pathlib import Path

from flask import current_app
from msal import PublicClientApplication, SerializableTokenCache
from exchangelib import (
    OAUTH2,
    Account,
    Configuration,
    OAuth2AuthorizationCodeCredentials,
)
from oauthlib.oauth2.rfc6749.tokens import OAuth2Token


def get_msal_app():
    cache = SerializableTokenCache()
    cache_file = Path(current_app.config['EXCHANGE_PROVIDER_CACHE_FILE'])
    if cache_file.exists():
        cache.deserialize(cache_file.read_text())
    app = PublicClientApplication(
        current_app.config['EXCHANGE_PROVIDER_CLIENT_ID'],
        authority=current_app.config['EXCHANGE_PROVIDER_AUTHORITY'],
        token_cache=cache,
    )
    return app, cache, cache_file


def save_msal_cache(cache, cache_file: Path):
    cache_file.touch(mode=0o600)  # create with safe permissions if new file
    cache_file.write_text(cache.serialize())


def get_msal_token(*, force=False):
    app, cache, cache_file = get_msal_app()
    username = current_app.config['EXCHANGE_PROVIDER_ACCOUNT']
    if not (accounts := app.get_accounts(username)):
        return None
    # get cached token or use a refresh token
    assert len(accounts) == 1
    assert accounts[0]['username'] == username
    result = app.acquire_token_silent(
        scopes=['https://outlook.office.com/EWS.AccessAsUser.All'],
        account=accounts[0],
        authority=None,
        force_refresh=force,
        claims_challenge=None,
    )
    if not result:
        return None

    # save cache in case we refreshed the token
    if cache.has_state_changed:
        save_msal_cache(cache, cache_file)
    return result.get('access_token')


class MSALCredentials(OAuth2AuthorizationCodeCredentials):
    def refresh(self, session):
        # XXX i think we never get here since msal refreshes it and sessions
        # do not persist
        print('refresh called')
        # self.access_token = ...



# code to acquire the initial token
@click.command()
@click.option(
    '-f',
    '--force',
    is_flag=True,
    help='Get a new token even if one may already exist',
)
@click.option(
    '-d',
    '--dump-token',
    is_flag=True,
    help='Dump the token on stdout',
)
def get_exchange_token(force, dump_token):
    """Get a "modern auth" Exchange token."""
    app, cache, cache_file = get_msal_app()
    username = current_app.config['EXCHANGE_PROVIDER_ACCOUNT']
    if token := get_msal_token(force=force):
        print(f'Got access token for {username}')
        if dump_token:
            print(token)
        sys.exit(0)

    flow = app.initiate_device_flow(
        scopes=['https://outlook.office.com/EWS.AccessAsUser.All']
    )
    if 'user_code' not in flow:
        print(f'Could not create device flow. Error: {flow}')
        sys.exit(1)
    print(flow['message'], flush=True)
    result = app.acquire_token_by_device_flow(flow)

    if 'access_token' not in result:
        print(f'Error getting access_token: {result}')
        sys.exit(1)

    accounts = app.get_accounts(username)
    assert len(accounts) == 1
    assert accounts[0]['username'] == username

    save_msal_cache(cache, cache_file)
    print(f'Got access token for {username}')
    if dump_token:
        print(result['access_token'])




# code to use the token
username = current_app.config['EXCHANGE_PROVIDER_ACCOUNT']

credentials = MSALCredentials(
    access_token=OAuth2Token({'access_token': get_msal_token()})
)
configuration = Configuration(
    server='outlook.office365.com', auth_type=OAUTH2, credentials=credentials
)
uid_account = Account(username, config=configuration, autodiscover=False)
print(uid_account)

It looks like passing the access_token in the credentials directly already prevents the call to fetch_token from happening.

Apr 24 '23 12:04 ThiefMaster

Ok, so just creating a credentials like this works for you without any patches to exchangelib?

credentials = MSALCredentials(
    access_token=OAuth2Token({'access_token': get_msal_token()})
)

Apr 24 '23 12:04 ecederstrand

yeah, i reinstalled exchangelib to make sure i didn't have any leftover manual changes from previous attempts... I think it works because oauth2_session_params=self.credentials.session_params() includes the param to set the token on the session

Apr 24 '23 12:04 ThiefMaster

Great! If you can collect the minimal code needed to get a token from msal to use when creating an OAuth2AuthorizationCodeCredentials instance, then I'll add that to the docs.

Apr 24 '23 12:04 ecederstrand

Sure! FWIW, I think it would be pretty nice to have something like this for simple usecases where you have the token (regardless of whether it comes from msal or something else) and don't need any other oauth/refresh-related logic on the exchangelib side:

configuration = Configuration(server=server, auth_type=OAUTH2_TOKEN, credentials=OAuthTokenCredentials('token goes here'))

Apr 24 '23 14:04 ThiefMaster

I finally got some time to look into this today, and got it to work. There are many different flows possible with MSAL, but I opted for a version that supports a command-line Python script that authenticates via a browser window redirect and neither requires creating a certificate or a client secret in Azure, or storing secrets anywhere. This version gives delegate access to the connecting account, which is what you want to just access your own mailbox.

The following code is just the bare essentials to get the auth working. There's a more complete example at https://github.com/AzureAD/microsoft-authentication-library-for-python/blob/dev/sample/interactive_sample.py, and example code for other flows at https://github.com/AzureAD/microsoft-authentication-library-for-python/tree/dev/sample:

"""Prerequisite: Create an app in Azure with EWS.AccessAsUser.All permissions and a “Mobile and Desktop application”
Redirect URI set to http://localhost. Note down the client ID. See also:

https://msal-python.readthedocs.io/en/latest/#msal.PublicClientApplication.acquire_token_interactive
"""
from exchangelib import Configuration, OAUTH2, Account, DELEGATE, OAuth2AuthorizationCodeCredentials
import msal

config = {
    "authority": "https://login.microsoftonline.com/organizations",
    "client_id": "MY_CLIENT_ID",
    "scope": ["EWS.AccessAsUser.All"],
    "username": "[email protected]",
    "endpoint": "https://graph.microsoft.com/v1.0/users",
    "account": "[email protected]",
    "server": "outlook.office365.com",
}
app = msal.PublicClientApplication(config["client_id"], authority=config["authority"])

print("A local browser window will be open for you to sign in. CTRL+C to cancel.")
result = app.acquire_token_interactive(config["scope"], login_hint=config.get("username"))
assert "access_token" in result

creds = OAuth2AuthorizationCodeCredentials(access_token=result)
conf = Configuration(server=config["server"], auth_type=OAUTH2, credentials=creds)
a = Account(
    primary_smtp_address=config["account"],
    access_type=DELEGATE,
    config=conf,
    autodiscover=False,
)
print(a.root.tree())

May 23 '23 12:05 ecederstrand

Added to the documentation at https://ecederstrand.github.io/exchangelib/#msal-on-office-365

May 23 '23 12:05 ecederstrand