LDAPS Multiple Times Need to Signin

[caxefaizan]

Environment

Flask-Appbuilder version: Flask-AppBuilder==3.4.1

pip freeze output: adal==1.2.7 alembic==1.7.5 amqp==5.0.7 anyio==3.4.0 apache-airflow==2.2.3 apache-airflow-providers-amazon==2.4.0 apache-airflow-providers-celery==2.1.0 apache-airflow-providers-cncf-kubernetes==2.2.0 apache-airflow-providers-docker==2.3.0 apache-airflow-providers-elasticsearch==2.1.0 apache-airflow-providers-ftp==2.0.1 apache-airflow-providers-google==6.2.0 apache-airflow-providers-grpc==2.0.1 apache-airflow-providers-hashicorp==2.1.1 apache-airflow-providers-http==2.0.1 apache-airflow-providers-imap==2.0.1 apache-airflow-providers-microsoft-azure==3.4.0 apache-airflow-providers-mysql==2.1.1 apache-airflow-providers-odbc==2.0.1 apache-airflow-providers-postgres==2.4.0 apache-airflow-providers-redis==2.0.1 apache-airflow-providers-sendgrid==2.0.1 apache-airflow-providers-sftp==2.3.0 apache-airflow-providers-slack==4.1.0 apache-airflow-providers-sqlite==2.0.1 apache-airflow-providers-ssh==2.3.0 apispec==3.3.2 argcomplete==1.12.3 asn1crypto==1.4.0 attrs==20.3.0 Authlib==0.15.5 azure-batch==11.0.0 azure-common==1.1.27 azure-core==1.21.1 azure-cosmos==4.2.0 azure-datalake-store==0.0.52 azure-identity==1.7.1 azure-keyvault==4.1.0 azure-keyvault-certificates==4.3.0 azure-keyvault-keys==4.4.0 azure-keyvault-secrets==4.3.0 azure-kusto-data==0.0.45 azure-mgmt-containerinstance==1.5.0 azure-mgmt-core==1.3.0 azure-mgmt-datafactory==1.1.0 azure-mgmt-datalake-nspkg==3.0.1 azure-mgmt-datalake-store==0.5.0 azure-mgmt-nspkg==3.0.2 azure-mgmt-resource==20.0.0 azure-nspkg==3.0.2 azure-storage-blob==12.8.1 azure-storage-common==2.1.0 azure-storage-file==2.1.0 Babel==2.9.1 backports.entry-points-selectable==1.1.1 bcrypt==3.2.0 beautifulsoup4==4.10.0 billiard==3.6.4.0 blinker==1.4 boto3==1.18.65 botocore==1.21.65 cached-property==1.5.2 cachetools==4.2.2 cattrs==1.5.0 celery==5.2.1 certifi==2020.12.5 cffi==1.15.0 charset-normalizer==2.0.9 click==8.0.3 click-didyoumean==0.3.0 click-plugins==1.1.1 click-repl==0.2.0 clickclick==20.10.2 cloudpickle==1.4.1 colorama==0.4.4 colorlog==4.8.0 commonmark==0.9.1 croniter==1.0.15 cryptography==3.4.8 dask==2021.6.0 decorator==5.1.0 defusedxml==0.7.1 dill==0.3.1.1 distlib==0.3.4 distributed==2.19.0 dnspython==2.1.0 docker==5.0.3 docutils==0.16 elasticsearch==7.13.4 elasticsearch-dbapi==0.2.7 elasticsearch-dsl==7.4.0 email-validator==1.1.3 eventlet==0.33.0 filelock==3.4.0 Flask==1.1.2 Flask-AppBuilder==3.4.1 Flask-Babel==2.0.0 Flask-Caching==1.10.1 Flask-JWT-Extended==3.25.1 Flask-Login==0.4.1 Flask-OpenID==1.3.0 Flask-SQLAlchemy==2.5.1 Flask-WTF==0.14.3 flower==1.0.0 fsspec==2021.11.1 gevent==21.12.0 google-ads==14.0.0 google-api-core==1.31.4 google-api-python-client==1.12.8 google-auth==1.35.0 google-auth-httplib2==0.1.0 google-auth-oauthlib==0.4.6 google-cloud-appengine-logging==1.1.0 google-cloud-audit-log==0.2.0 google-cloud-automl==2.5.2 google-cloud-bigquery==2.31.0 google-cloud-bigquery-datatransfer==3.4.1 google-cloud-bigquery-storage==2.10.1 google-cloud-bigtable==1.7.0 google-cloud-build==3.7.1 google-cloud-container==1.0.1 google-cloud-core==1.7.2 google-cloud-datacatalog==3.6.1 google-cloud-dataproc==3.1.1 google-cloud-dataproc-metastore==1.3.1 google-cloud-dlp==1.0.0 google-cloud-kms==2.10.1 google-cloud-language==1.3.0 google-cloud-logging==2.7.0 google-cloud-memcache==1.0.0 google-cloud-monitoring==2.8.0 google-cloud-os-login==2.5.1 google-cloud-pubsub==2.9.0 google-cloud-redis==2.5.0 google-cloud-secret-manager==1.0.0 google-cloud-spanner==1.19.1 google-cloud-speech==1.3.2 google-cloud-storage==1.43.0 google-cloud-tasks==2.7.1 google-cloud-texttospeech==1.0.1 google-cloud-translate==1.7.0 google-cloud-videointelligence==1.16.1 google-cloud-vision==1.0.0 google-cloud-workflows==1.5.0 google-crc32c==1.3.0 google-resumable-media==2.1.0 googleapis-common-protos==1.54.0 graphviz==0.19.1 greenlet==1.1.2 grpc-google-iam-v1==0.12.3 grpcio==1.42.0 grpcio-gcp==0.2.2 gunicorn==20.1.0 h11==0.12.0 HeapDict==1.0.1 httpcore==0.13.7 httplib2==0.19.1 httpx==0.19.0 humanize==3.13.1 hvac==0.11.2 idna==3.3 importlib-metadata==4.8.2 importlib-resources==5.4.0 inflection==0.5.1 iso8601==1.0.2 isodate==0.6.0 itsdangerous==1.1.0 Jinja2==3.0.3 jmespath==0.10.0 json-merge-patch==0.2 jsonpath-ng==1.5.3 jsonschema==3.2.0 kombu==5.2.2 kubernetes==11.0.0 lazy-object-proxy==1.4.3 ldap3==2.9.1 libcst==0.3.23 locket==0.2.1 lockfile==0.12.2 lxml==4.7.0 Mako==1.1.6 Markdown==3.3.6 MarkupSafe==2.0.1 marshmallow==3.14.1 marshmallow-enum==1.5.1 marshmallow-oneofschema==3.0.1 marshmallow-sqlalchemy==0.26.1 msal==1.16.0 msal-extensions==0.3.1 msgpack==1.0.3 msrest==0.6.21 msrestazure==0.6.4 mypy-extensions==0.4.3 mysql-connector-python==8.0.27 mysqlclient==2.1.0 nox==2020.12.31 numpy==1.20.3 oauthlib==3.1.1 openapi-schema-validator==0.1.5 openapi-spec-validator==0.3.1 packaging==21.3 pandas==1.3.5 pandas-gbq==0.14.1 paramiko==2.8.1 partd==1.2.0 pendulum==2.1.2 platformdirs==2.4.0 ply==3.11 portalocker==2.3.2 prison==0.2.1 prometheus-client==0.12.0 prompt-toolkit==3.0.24 proto-plus==1.18.1 protobuf==3.19.1 psutil==5.8.0 psycopg2-binary==2.9.2 py==1.11.0 pyarrow==5.0.0 pyasn1==0.4.8 pyasn1-modules==0.2.8 pycparser==2.21 pydata-google-auth==1.3.0 Pygments==2.10.0 PyJWT==1.7.1 PyNaCl==1.4.0 pyodbc==4.0.32 pyOpenSSL==20.0.1 pyparsing==2.4.7 pyrsistent==0.18.0 pysftp==0.2.9 python-daemon==2.3.0 python-dateutil==2.8.2 python-http-client==3.3.4 python-ldap==3.4.0 python-nvd3==0.15.0 python-slugify==4.0.1 python3-openid==3.2.0 pytz==2021.3 pytzdata==2020.1 PyYAML==5.4.1 redis==3.5.3 redshift-connector==2.0.901 requests==2.26.0 requests-oauthlib==1.3.0 rfc3986==1.5.0 rich==10.16.0 rsa==4.8 s3transfer==0.5.0 scramp==1.4.1 sendgrid==6.9.2 setproctitle==1.2.2 six==1.16.0 slack-sdk==3.13.0 sniffio==1.2.0 sortedcontainers==2.4.0 soupsieve==2.3.1 SQLAlchemy==1.3.24 SQLAlchemy-JSONField==1.0.0 sqlalchemy-redshift==0.8.8 SQLAlchemy-Utils==0.37.9 sshtunnel==0.4.0 starkbank-ecdsa==2.0.3 statsd==3.3.0 swagger-ui-bundle==0.0.9 tabulate==0.8.9 tblib==1.7.0 tenacity==8.0.1 termcolor==1.1.0 text-unidecode==1.3 toolz==0.11.2 tornado==6.1 typing-extensions==3.10.0.2 typing-inspect==0.7.1 unicodecsv==0.14.1 uritemplate==3.0.1 urllib3==1.26.7 vine==5.0.0 virtualenv==20.10.0 watchtower==1.0.6 wcwidth==0.2.5 websocket-client==1.2.3 Werkzeug==1.0.1 WTForms==2.3.3 zict==2.0.0 zipp==3.6.0 zope.event==4.5.0 zope.interface==5.4.0

Describe the expected results

Login should happen on the first attempt itself.

Describe the actual results

In the logs of the webserver i see this {manager.py:1153} ERROR - {'result': -1, 'desc': "Can't contact LDAP server", 'ctrls': [], 'info': '(unknown error code)'} Multiple Simultaneous Clicks logs us in.

Steps to reproduce

This is the webserver file in apache-airflow.

#webserver_config.py

import os
from airflow.www.fab_security.manager import AUTH_LDAP

AUTH_TYPE = AUTH_LDAP
AUTH_LDAP_SERVER = "ldaps://corporate.example.com.au:636"
AUTH_LDAP_USE_TLS = False
AUTH_LDAP_ALLOW_SELF_SIGNED = False
AUTH_LDAP_TLS_CACERTFILE = "/usr/local/airflow/ldapcert/cert.crt"

# registration configs
AUTH_USER_REGISTRATION = True  # allow users who are not already in the FAB DB
AUTH_USER_REGISTRATION_ROLE = "Public"  # this role will be given in addition to any AUTH_ROLES_MAPPING
AUTH_LDAP_FIRSTNAME_FIELD = "givenName"
AUTH_LDAP_LASTNAME_FIELD = "sn"
AUTH_LDAP_EMAIL_FIELD = "mail"  # if null in LDAP, email is set to: "{username}@email.notfound"

AUTH_LDAP_SEARCH = "ou=WFL,dc=corporate,dc=example,dc=com,dc=au"  # the LDAP search base
AUTH_LDAP_UID_FIELD = "sAMAccountName"  # the username field
AUTH_LDAP_BIND_USER = "{{ LDAP_BIND_USER }}"  # the special bind username for search
AUTH_LDAP_BIND_PASSWORD = "{{ LDAP_PASSWORD }}"

# a mapping from LDAP DN to a list of FAB roles
AUTH_ROLES_MAPPING = {
    "CN=Airflow2-Admin,OU=Security,OU=Groups,OU=WFL,DC=corporate,DC=example,DC=com,DC=au": ["Admin"],
    "CN=Airflow2-Data-Engineers,OU=Security,OU=Groups,OU=WFL,DC=corporate,DC=example,DC=com,DC=au": ["Op"],
}

# the LDAP user attribute which has their role DNs
AUTH_LDAP_GROUP_FIELD = "memberOf"

# if we should replace ALL the user's roles each login, or only on registration
AUTH_ROLES_SYNC_AT_LOGIN = True

# force users to re-auth after 30min of inactivity (to keep roles in sync)
PERMANENT_SESSION_LIFETIME = 1800

[dpgaspar]

My first impression is that it could be something related with network, are you using a load balancer?

Have you tried reproducing the issue using:

openssl s_client -connect ldaps://corporate.example.com.au:636

from inside you Airflow web server?

[NikolaySokolov]

@caxefaizan I've found that the image for 2.3.0 (in my case specifically, we went from apache/airflow:2.2.5-python3.9 to apache/airflow:2.3.0-python3.9), turns out the underlying base image went from being Debian 10 to Debian 11 and with this change, the /etc/ldap/ldap.conf is no longer present which exhibits this exact behavior and the super cryptic error. After spending half a day trying to find a solution, all what's really needed is to add libldap-common to the docker image and Airflow starts working again with LDAP just fine.

So, hopefully this will help you and someone else in the future!

[sbernauer]

@caxefaizan this might be related to https://github.com/dpgaspar/Flask-AppBuilder/pull/1846. Yould you please give https://github.com/dpgaspar/Flask-AppBuilder/pull/1846 a try and see if it fixes you problem? You can get a idea on how to patch your flask installation here https://github.com/stackabletech/docker-images/pull/116