Bugfix/mask sensitive values in cli
related: https://github.com/apache/airflow/issues/55947
The purpose of this PR:
Mask sensitive values in cli The sensitive values are:
secret_key
jwt_secret
asset_manager_kwargs
fernet_key
sql_alchemy_conn
sql_alchemy_conn_async
sql_alchemy_engine_args
remote_task_handler_kwargs
backend_kwargs
sentry_dsn
secret_key
secrets_backend_kwargs
What this PR changed:
- Add two parameter
hide_sensitive_values, andis_sensitiveindef _write_value - Add logic to check if the value is sensitive. If it is, print the value as "< hidden >", if not, print the original value.
- Add new cli flag
--hide-sensitive, so user can determine whether to hide the sensitive values.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst) Here are some useful points:
- Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
- In case of a new feature add useful documentation (in docstrings or in
docs/directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it. - Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the Airflow Coding style.
- Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits. Apache Airflow is a community-driven project and together we are making it better 🚀. In case of doubts contact the developers at: Mailing List: [email protected] Slack: https://s.apache.org/airflow-slack
This is a good idea but it's not going far enough.
We recently agreed on a different approach https://lists.apache.org/thread/3dhzqvpw4z2x0wfokpmdncsjwws86zcs
Part of this approach was that config list should not show values at all (just list configs) , then I think we should add flag to show non-sensitive values and mask sensitive, and third option should be show all.
Can you adapt please to the latest consensus ?
@potiuk OK, I will modify it. Thanks for your review.
@potiuk With my comprehension, CLI * list should not show value. There should be two situations 1) --non-sensitive to only show the non-sensitive value. 2) --show-all to show all values. Am I right?
Here are the key traceback for the investigation
File "/opt/airflow/task-sdk/src/airflow/sdk/api/client.py", line 182, in raise_on_4xx_5xx
return get_json_error(response) or response.raise_for_status()
File "/opt/airflow/task-sdk/src/airflow/sdk/api/client.py", line 178, in get_json_error
raise err
airflow.sdk.api.client.ServerResponseError: Invalid auth token: Signature verification failed