airflow icon indicating copy to clipboard operation
airflow copied to clipboard

Param does not work with type=['array', 'null'], items={'type': 'number'} and None default

Open daniel-westerfeld opened this issue 1 year ago • 0 comments

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.8.2

What happened?

I am trying to pass a parameter to a DAG using Param. I need to pass an optional list of integers, hence, I tried using type=['array', 'null'], items={'type': 'number'} with None as default value. However, when I try to trigger the DAG and add a non-None array of numbers, the JSON config does not get updated and null is passed.

What you think should happen instead?

The non-empty array of integers should get passed to the DAG.

How to reproduce

Create a DAG with the mentioned Param, e.g.

from airflow import DAG
from airflow.models.param import Param

default_args = {
    'owner': '[email protected]',
    'start_date': datetime(2023, 12, 13),
    'retries': 1,
    'retry_delay': timedelta(minutes=5),
}

with DAG(
    'example_dag',
    default_args=default_args,
    schedule='0 1 * * *',
    tags=['news-dags'],
    params={
        'ids': Param(None, type=['array', 'null'], items={'type': 'number'}),
    },
) as example_dag:
    ...

Now try to trigger it with config and add a list of integer IDs. I noted, that it works when I specify type='array' but the default should be None. The behaviour of my DAG differs when ids are None or an empty list, so passing [] as a default and removing null is not an option.

Operating System

Debian bookworm

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==8.18.0 apache-airflow-providers-celery==3.6.0 apache-airflow-providers-common-io==1.3.0 apache-airflow-providers-common-sql==1.11.0 apache-airflow-providers-docker==3.9.1 apache-airflow-providers-ftp==3.7.0 apache-airflow-providers-http==4.9.1 apache-airflow-providers-imap==3.5.0 apache-airflow-providers-postgres==5.10.1 apache-airflow-providers-redis==3.6.0 apache-airflow-providers-sqlite==3.7.1

Deployment

Other Docker-based deployment

Deployment details

No response

Anything else?

Always when trying to use Param with the mentioned combination.

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

daniel-westerfeld avatar Dec 18 '24 13:12 daniel-westerfeld