astro-sdk icon indicating copy to clipboard operation
astro-sdk copied to clipboard

Credentials from Connection getting printed in the Task Logs

Open manmeetkaur opened this issue 1 year ago • 0 comments

Describe the bug The problem is that the entire connection details including the password are getting printed in the Airflow Task Log when the DEBUG Logging mode is enabled.

Version

  • Astro: 7.3.0
  • OS: MacOS

To Reproduce Steps to reproduce the behavior:

  1. Write the DAG :
from datetime import datetime
from airflow import DAG
from astro import sql as aql
from astro.files import File
from astro.sql.table import Metadata, Table
from astro.sql import LoadFileOperator

imdb_file = File("https://raw.githubusercontent.com/astronomer/astro-sdk/main/tests/data/imdb_v2.csv")
imdb_table = Table(name="imdb_data", conn_id="snowflake")


with DAG(
    "calculate_popular_movies",
    schedule_interval=None,
    start_date=datetime(2000, 1, 1),
    catchup=False,
) as dag:

    load_file = LoadFileOperator(
        input_file=imdb_file,
        output_table=imdb_table
    )

    load_file 

    aql.cleanup()

  1. Create connection snowflake
  2. Enable the DEBUG Logging using ENV AIRFLOW__LOGGING__LOGGING_LEVEL=DEBUG in the Dockerfile
  3. Run the DAG
  4. You can search for parsers.py:240 in the Task Log for load_file task

Expected behavior The task will run successfully but in the Task log with DEBUG enabled you will see the entire snowflake connection details printed multiple times as seen in the screenshot.

Screenshots Screenshot 2023-03-28 at 2 44 56 PM

Additional context Add any other context about the problem here.

manmeetkaur avatar Mar 28 '23 09:03 manmeetkaur