airflow icon indicating copy to clipboard operation
airflow copied to clipboard
verified publisher icon apache

Login delay causes false authentication error messages on UI Airflow >=3.1.1

Open anjo0511 opened this issue 1 month ago • 6 comments

Apache Airflow version

3.1.2

If "Other Airflow 2/3 version" selected, which one?

3.1.2, 3.1.1

What happened?

When login in with Azure Oauth SSO the initial authentication delay causes an error message to pop up on screen, when the authentication is successful 1 second after the error message dissapears.

Image Image

Logs:

api-server INFO: 10.9.155.17:60082 - "GET /api/v2/version HTTP/1.1" 200 OK api-server INFO: 10.9.155.17:34212 - "GET /api/v2/version HTTP/1.1" 200 OK api-server INFO: 10.9.155.17:34218 - "GET /api/v2/version HTTP/1.1" 200 OK api-server INFO: 10.128.1.206:43444 - "GET /api/v2/auth/login?next=***" 307 Temporary Redirect api-server Provider: None api-server DEBUG: Provider: None [views.py:633] api-server INFO: 10.128.1.206:43444 - "GET /auth/login/?next=***" 200 OK api-server INFO: 10.128.1.206:43444 - "GET /auth/static/appbuilder/css/fontawesome/fontawesome.min.css HTTP/1.1" 304 Not Modified api-server INFO: 10.128.1.206:43444 - "GET /auth/static/appbuilder/css/fontawesome/solid.min.css HTTP/1.1" 304 Not Modified ... api-server INFO: 10.128.1.206:43548 - "GET /auth/static/dist/flash.5583a9e0cf11f2be93da.css HTTP/1.1" 304 Not Modified api-server INFO: 10.128.1.206:43492 - "GET /auth/static/dist/main.3cf3be1a0c5439bb640d.js HTTP/1.1" 304 Not Modified ... api-server Provider: azure api-server DEBUG: Provider: azure [views.py:633] api-server DEBUG: Going to call authorize for: azure [views.py:646] api-server INFO: 10.128.1.206:43578 - "GET /auth/login/azure?next=***" 302 Found api-server INFO: 10.9.155.17:44558 - "GET /api/v2/version HTTP/1.1" 200 OK ... api-server DEBUG: Authorized init [views.py:676] api-server DEBUG: OAUTH Authorized resp: { 'token_type': 'Bearer', 'scope': 'email openid profile User.Read', 'access_token': '***', 'id_token': '***', 'userinfo': { 'email': '[REDACTED]', 'name': '[REDACTED]', ... } } [views.py:690] api-server DEBUG: Parsing JWT token for provider: azure [webserver_config.py:105] api-server DEBUG: User info from Azure: { 'email': '[REDACTED]', 'name': '[REDACTED]', 'roles': ['role1', 'role2', ...] } [override.py:2112] api-server DEBUG: _____________________role_keys start__________________________________ [webserver_config.py:115] api-server DEBUG: Parsed JWT token: { 'email': '[REDACTED]', 'username': '[REDACTED]', 'role_keys': [...] } [webserver_config.py:116] api-server DEBUG: Role Keys: [...] [webserver_config.py:117] api-server DEBUG: _____________________role_keys end____________________________________ [webserver_config.py:118] api-server DEBUG: User info retrieved from azure: { 'name': '[REDACTED]', 'email': '[REDACTED]', 'role_keys': [...] } [views.py:699] api-server DEBUG: No whitelist for OAuth provider [views.py:712] api-server DEBUG: Calculated new roles for user='[REDACTED]' as: [Public, Admin] [override.py:2043] api-server INFO: Updated user [REDACTED] [override.py:1449] api-server INFO: 10.128.1.1:38716 - "GET /auth/oauth-authorized/azure?code=***&state=***&session_state=*** HTTP/1.1" 302 Found api-server INFO: 10.128.1.1:38716 - "GET /dags/test_remote_logging/runs/manual__*** HTTP/1.1" 200 OK api-server INFO: 10.9.155.17:59236 - "GET /api/v2/version HTTP/1.1" 200 OK ... api-server INFO: 10.128.1.1:38716 - "GET /ui/config HTTP/1.1" 401 Unauthorized api-server INFO: 10.128.1.1:38716 - "GET /api/v2/auth/login?next=***" 307 Temporary Redirect api-server DEBUG: Provider: None [views.py:633] api-server DEBUG: Already authenticated [REDACTED] [views.py:635] api-server INFO: 10.128.1.1:38716 - "GET /auth/login/?next=*** HTTP/1.1" 302 Found api-server INFO: 10.128.1.1:38716 - "GET /auth/ HTTP/1.1" 302 Found api-server INFO: 10.128.1.1:38716 - "GET / HTTP/1.1" 200 OK api-server INFO: 10.128.1.1:38716 - "GET /ui/config HTTP/1.1" 200 OK api-server INFO: 10.128.1.1:38716 - "GET /static/i18n/locales/en/dag.json HTTP/1.1" 304 Not Modified api-server INFO: 10.128.1.1:38716 - "GET /ui/auth/menus HTTP/1.1" 200 OK ... api-server INFO: 10.128.1.1:50736 - "GET /ui/dashboard/historical_metrics_data?start_date=*** HTTP/1.1" 200 OK api-server INFO: 10.128.1.1:50740 - "GET /api/v2/assets/events?limit=6&order_by=-timestamp&timestamp_gte=***&timestamp_lte=*** HTTP/1.1" 200 OK api-server INFO: 10.128.1.1:50668 - "GET /api/v2/dags/~/dagRuns/~/hitlDetails?state=deferred&response_received=false HTTP/1.1" 200 OK api-server INFO: 10.128.1.1:50666 - "GET /api/v2/dags/~/dagRuns?state=running&state=queued HTTP/1.1" 200 OK api-server INFO: 10.128.1.1:50700 - "GET /ui/dags?limit=10&is_favorite=true HTTP/1.1" 200 OK ...

What you think should happen instead?

The ui should not render the error message and should wait for the login outcome.

How to reproduce

Setup Airflow 3.1.2 on python3.12

Operating System

Debian GNU/Linux 12 (bookworm)

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

Deploy using official helm chart to k8s on azure (aks)

Anything else?

This occurs each time a login happens, This is the apiserver_config.py used for the oauth towards azure, worked on airlfow 3.0.6 without issues.

#
## https://gist.github.com/wallyhall/915fedb4dfc766b61f442a32c95e1c29#file-apache_airflow_sso_howto-md
## https://gist.github.com/wallyhall/915fedb4dfc766b61f442a32c95e1c29?permalink_comment_id=5253211#gistcomment-5253211
#

from __future__ import annotations
import os

# Airflow 3.1.x and FAB provider >=2.6.0
#! https://gist.github.com/wallyhall/915fedb4dfc766b61f442a32c95e1c29?permalink_comment_id=5623661#gistcomment-5623661
from airflow.providers.fab.auth_manager.security_manager.override import FabAirflowSecurityManagerOverride
from airflow.utils.log.logging_mixin import LoggingMixin
from flask_appbuilder.security.manager import AUTH_OAUTH

basedir = os.path.abspath(os.path.dirname(__file__))

print("------------------------------------STARTING AUTH------------------------------------")

# ------------------------------------------------------------------------------
# General Flask AppBuilder / Airflow Auth Config
# ------------------------------------------------------------------------------
WTF_CSRF_ENABLED = True
WTF_CSRF_TIME_LIMIT = None

# ------------------------------------------------------------------------------
# Azure AD OAuth Settings
# ------------------------------------------------------------------------------
AAD_TENANT_ID = os.getenv("AAD_TENANT_ID")
AAD_CLIENT_ID = os.getenv("AAD_CLIENT_ID")
AAD_CLIENT_SECRET = os.getenv("AAD_CLIENT_SECRET")

AUTH_TYPE = AUTH_OAUTH

OAUTH_PROVIDERS = [
    {
        "name": "azure",
        "token_key": "access_token",
        "icon": "fa-windows",
        "remote_app": {
            "api_base_url": f"https://login.microsoftonline.com/{AAD_TENANT_ID}",
            "request_token_url": None,
            "request_token_params": {"scope": "openid profile email"},
            "access_token_url": f"https://login.microsoftonline.com/{AAD_TENANT_ID}/oauth2/v2.0/token",
            "authorize_url": f"https://login.microsoftonline.com/{AAD_TENANT_ID}/oauth2/v2.0/authorize",
            "client_id": f"{AAD_CLIENT_ID}",
            "client_secret": f"{AAD_CLIENT_SECRET}",
            "client_kwargs": {"scope": "openid profile email"},
            "jwks_uri": "https://login.microsoftonline.com/common/discovery/v2.0/keys",
        },
    }
]

# ------------------------------------------------------------------------------
# User registration and role sync
# ------------------------------------------------------------------------------
AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = "Public"
AUTH_ROLES_SYNC_AT_LOGIN = True

# This gets injected via Helm values usually, First you MUST create a role like"Admin with value Admin" in the AppRegistration "App Roles" section in the Azure Portal under Microsoft EntraID.
# Then groups MUST be linked from the Microsoft Entra ID "EnterpriseApplication" section in the Azure Portal under the "Users and Groups" section.
# Each groups or users MUST be assigned a role e.g.: Admin, Op, Viewer in the "Users and Groups"

# https://airflow.apache.org/docs/apache-airflow/2.4.3/security/access-control.html
# AUTH_ROLES_MAPPING = {
#     "airflow-admin": ["Admin"],
#     "airflow-user": ["User"],
# }
PLACEHOLDER_FOR_AUTH_ROLES_MAPPING


# ------------------------------------------------------------------------------
# Custom Security Manager
# ------------------------------------------------------------------------------
class AzureCustomSecurity(FabAirflowSecurityManagerOverride, LoggingMixin):
    """
    Custom FAB security manager for Azure AD OAuth.
    Compatible with Airflow 3.1.x (API server).
    """

    def oauth_user_info_save(self, userinfo):
        """
        Override Airflow 3.1+ user creation to reuse existing users by email.
        """
        email = userinfo.get("email")
        if not email:
            self.log.warning("OAuth userinfo missing email — cannot authenticate user.")
            return None

        existing_user = self.find_user(email=email)
        if existing_user:
            self.log.debug(f"User {email} already exists, reusing record instead of creating new one.")
            return existing_user

        self.log.debug(f"No existing user found for {email}, creating new user.")
        return super().oauth_user_info_save(userinfo)

    def get_oauth_user_info(self, provider, response=None):
        self.log.debug(f"Parsing JWT token for provider: {provider}")
        try:
            me = super().get_oauth_user_info(provider, response)
        except Exception as e:
            import traceback

            traceback.print_exc()
            self.log.debug(e)
            return None

        self.log.debug("_____________________role_keys start__________________________________")
        self.log.debug(f"Parsed JWT token: {me}")
        self.log.debug(f"Role Keys: {me.get('role_keys')}")
        self.log.debug("_____________________role_keys end____________________________________")

        #! Example output from: self.log.debug(f"Parse JWT token : {me}")
        # DEBUG - Parse JWT token : {'email': '[email protected]',
        # 'first_name': 'A',
        # 'last_name': 'J',
        # 'username': 'ebxxxxxxxxxxxxxxxxxxxxxxxxxxae7',
        # 'role_keys': ['airflow-user','airflow-admin']}

        return {
            "name": f"{me.get('first_name', '')} {me.get('last_name', '')}",
            "email": me.get("email"),
            "first_name": me.get("first_name"),
            "last_name": me.get("last_name"),
            "id": me.get("username"),
            "username": me.get("email"),  # email as username avoids duplicates
            "role_keys": me.get("role_keys"),
        }


# ------------------------------------------------------------------------------
# Register the Auth Manager for Airflow 3.1.x API Server
# ------------------------------------------------------------------------------
FAB_SECURITY_MANAGER_CLASS = "webserver_config.AzureCustomSecurity"
SECURITY_MANAGER_CLASS = AzureCustomSecurity

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

anjo0511 avatar Nov 06 '25 16:11 anjo0511

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

boring-cyborg[bot] avatar Nov 06 '25 16:11 boring-cyborg[bot]

Experiencing the same using fab 3.0.0

ido177 avatar Nov 08 '25 16:11 ido177

I got the same error, but after redirect, I got Internal Server Error

Airflow version: 3.1.2

Error log: https://pastebin.com/UCsFJnwf

vodkar avatar Nov 11 '25 09:11 vodkar

i got the same error on the UI but the log shows 2 pretty strange messages during the time:

  1. JWT token is not valid: Signature has expired [airflow.api_fastapi.auth.managers.base_auth_manager] loc=base_auth_manager.py:107
    • this one escalated to psycopg2.OperationalError: SSL connection has been closed unexpectedly and i had to restart the api-server
  2. Error authorizing OAuth access token: mismatching_state: CSRF Warning! State not equal in request and response.
    • this one happened after the server reboot. it seems to be harmless.

Airflow version: 3.1.2

oops, commented on the wrong issue, looks like mine was actually https://github.com/apache/airflow/issues/57065, https://github.com/apache/airflow/issues/57534 and https://github.com/apache/airflow/issues/57470. it was just fixed by https://github.com/apache/airflow/pull/58196

zachliu avatar Nov 11 '25 16:11 zachliu

When you go on the UI without being authenticated, you'll be redirected to your authmanager login page. There you login and normally, only after a successful attempt, you are redirected back to the airflow UI, via the next url, and with a set_cookie containing your token.

You should never be redirected to the UI while the Azure Oauth SSO the initial authentication delay occur.

Can you provide more context on how you are redirected to airflow UI without having the auth flow completed and the token successfully set as a secured cookie ?

pierrejeambrun avatar Nov 25 '25 15:11 pierrejeambrun

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

github-actions[bot] avatar Dec 10 '25 00:12 github-actions[bot]

Do you guys mind testing on latest release 3.1.4 and reporting back with additional information? That's not expected and would love some insight. Is that still happening on latest version?

Is that error screen coming right after the 'login' (admin + password form) ?

pierrejeambrun avatar Dec 11 '25 15:12 pierrejeambrun

Still happens on 3.1.5

amoskueez avatar Dec 21 '25 08:12 amoskueez