Dynamic Task Mapping TriggerDagRunOperator breaks grid/graph view

[jkarun]

Apache Airflow version

2.8.1

If "Other Airflow 2 version" selected, which one?

2.6.1

What happened?

I am using dynamic task mapping of TriggerDagRunOperator to trigger sub-dags based on previous component responses. Recently I upgraded the airflow version to 2.8.1 and the graph/grid UI is not loading and gets an error popup. Earlier I was using version 2.6.1 and was able to view the graph UI.

Sample Code I used in version 2.6.1

   @task.python(queue='kubernetes', task_id='get_child_dag_details')
    def get_child_dag_details(**kwargs):
        """
        Fetch function pipeline details. This function MUST return list of dict value to run.
        sample output formate output:
            [
                {'trigger_dag_id': '', 'conf': {}},
            ]
        """
        try:
           # make REST API call to an external system and get the dag name and parameter
        except Exception as error:
            logging.error(error)
            raise
    
    champ_details = get_provisional_champion_details()
    
dynamic_dag_trigger = TriggerDagRunOperator.partial(
        task_id='trigger_sub_dag',
        wait_for_completion=True,
        poke_interval=30,
        allowed_states=['success'],
    ).expand_kwargs(champ_details)

   component_A >> champ_details >> dynamic_dag_trigger

After upgrading the airflow version graph UI throws the below error

What you think should happen instead?

DAG should load a graph or grid view in the UI without changing the working code.

How to reproduce

use airflow 2.8.1 and my sample code to reproduce the error.

Operating System

Mac

Versions of Apache Airflow Providers

No response

Deployment

Amazon (AWS) MWAA

Deployment details

No response

Anything else?

For a workaround, I set the operator_extra_links property of TriggerDagRunOperator to an empty list. After setting this property I can view the graph/grid view of the DAG in Airflow 2.8.1 version.

sample code:

    TriggerDagRunOperator.operator_extra_links=[]
    dynamic_dag_trigger = TriggerDagRunOperator.partial(
        task_id='trigger_sub_dag',
        wait_for_completion=True,
        poke_interval=30,
        allowed_states=['success'],
    ).expand_kwargs(champ_details)

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[josh-fell]

@jkarun Thanks for reporting. Does this cause a DAG import error as well or just an issue with the UI view?

[jkarun]

hi @josh-fell , No DAG import error. It's just an issue with the grid or graph UI view. I can run the DAG and the components are working fine.

[Taragolis]

Amazon (AWS) MWAA

Apache Airflow version 2.8.1

Just wondering is both of these information correct? As far as I know MWAA do not supports 2.8.1 yet: https://docs.aws.amazon.com/mwaa/latest/userguide/airflow-versions.html#airflow-versions-official

[cmarteepants]

@task.python(queue='kubernetes', task_id='get_child_dag_details')
    def get_child_dag_details(**kwargs):
       ...

Not to take away from the initial problem - but also want to confirm if this was from MWAA. I did not realize that MWAA supports celery worker queues or CeleryKubernetes Executor.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

[github-actions[bot]]

This issue has been closed because it has not received response from the issue author.

[barsa4ever]

Is it possible to reopen this issue?

[potiuk]

Is it possible to reopen this issue?

If you have more data, information and similar error - it's better to open a new issue and provide your data/ details - linking to that one. There seem to be confusion about where the issue comes from - so if you can create a new issue @barsa4ever and provide all the accurate details, log etc. that's way better than reopening an issue that has unanswered questions to the author. Then you will become an author and you will be able to answer questions and provide all the details and maybe even confirm that things are solved if it happens that someone solves it. .