gusty icon indicating copy to clipboard operation
gusty copied to clipboard

running backfill without disabling pickling runs .py tasks without __builtins__

Open machow opened this issue 3 years ago • 0 comments

I need to create a small reproducible example, but noticed while backfilling. Running backfill from the CLI like...

airflow backfill some_dag -s 2021-04-16T00:00:00+00:00 --reset_dagruns -i -t some_task

Seems to not include __builtins__ when running the task. This means that tasks I was running failed with...

NameError: name 'print' is not defined

When I don't pickle, the problem goes away, so I'm guessing it has to do with the way the .py file is imported using spec_from_file_location? For reference here is one way you could trigger this error:

exec(
    """exec('print("hey")')""",
    {
        '__builtins__': {'exec': exec}
    },
    {}
)

machow avatar Apr 27 '21 21:04 machow