airflow icon indicating copy to clipboard operation
airflow copied to clipboard

Bugfix/mask sensitive values in cli

Open arnoldmr01 opened this issue 1 month ago • 4 comments

related: https://github.com/apache/airflow/issues/55947

The purpose of this PR:

Mask sensitive values in cli The sensitive values are:

secret_key
jwt_secret
asset_manager_kwargs
fernet_key
sql_alchemy_conn
sql_alchemy_conn_async
sql_alchemy_engine_args
remote_task_handler_kwargs
backend_kwargs
sentry_dsn
secret_key
secrets_backend_kwargs

What this PR changed:

  • Add two parameter hide_sensitive_values, and is_sensitive in def _write_value
  • Add logic to check if the value is sensitive. If it is, print the value as "< hidden >", if not, print the original value.
  • Add new cli flag --hide-sensitive, so user can determine whether to hide the sensitive values.

^ Add meaningful description above Read the Pull Request Guidelines for more information. In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed. In case of a new dependency, check compliance with the ASF 3rd Party License Policy. In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

arnoldmr01 avatar Nov 25 '25 05:11 arnoldmr01

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst) Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits. Apache Airflow is a community-driven project and together we are making it better 🚀. In case of doubts contact the developers at: Mailing List: [email protected] Slack: https://s.apache.org/airflow-slack

boring-cyborg[bot] avatar Nov 25 '25 05:11 boring-cyborg[bot]

This is a good idea but it's not going far enough.

We recently agreed on a different approach https://lists.apache.org/thread/3dhzqvpw4z2x0wfokpmdncsjwws86zcs

Part of this approach was that config list should not show values at all (just list configs) , then I think we should add flag to show non-sensitive values and mask sensitive, and third option should be show all.

Can you adapt please to the latest consensus ?

potiuk avatar Nov 25 '25 17:11 potiuk

@potiuk OK, I will modify it. Thanks for your review.

arnoldmr01 avatar Nov 26 '25 21:11 arnoldmr01

@potiuk With my comprehension, CLI * list should not show value. There should be two situations 1) --non-sensitive to only show the non-sensitive value. 2) --show-all to show all values. Am I right?

arnoldmr01 avatar Nov 27 '25 07:11 arnoldmr01

Here are the key traceback for the investigation

  File "/opt/airflow/task-sdk/src/airflow/sdk/api/client.py", line 182, in raise_on_4xx_5xx
    return get_json_error(response) or response.raise_for_status()
  File "/opt/airflow/task-sdk/src/airflow/sdk/api/client.py", line 178, in get_json_error
    raise err
airflow.sdk.api.client.ServerResponseError: Invalid auth token: Signature verification failed

jason810496 avatar Dec 20 '25 08:12 jason810496