`.airflowignore`'d DAGs visible in UI via `Assets`/`AssetEvents`

[jroachgolf84]

Apache Airflow version

main (development)

If "Other Airflow 2/3 version" selected, which one?

No response

What happened?

When DAGs that previously outlet/scheduled by Assets are added to .airflowignore, these DAGs can still be navigated to (via a DAG run).

Further Explanation

• dag_a outlets to asset_1
• dag_b has schedule=[Asset("asset_1")]
• dag_a is executed which triggers dag_b via asset_1
• Then, both of these DAGs are added to the .airflowignore file

However, going back into the Airflow UI to .../assets allows for navigation to a DAG run for both of the ignored DAGs.

What you think should happen instead?

If a DAG is included in the .gitignore file and is NOT visible in the UI, it should also remaining NOT visible (for any DAG runs) when going to triggering/scheduled DAG events.

How to reproduce

1. Author a dag_a and dag_b as shown below.
2. Leave .airflowignore empty.
3. Turn on dag_b.
4. Turn on dag_a.
5. Validate that an AssetEvent for asset_1 is generated.
6. Update .airflowignore to include dag_a and dag_b.
7. Restart local instance of Airflow.
8. Validate that no DAGs are present in localhost:28080/dags.
9. Navigate to the "Assets" view.
10. Click on asset_1. You should then see a single AssetEvent with "Source"" = dag_a and "Triggered Dag Run" = dag_b.
11. Click on the "Source" (dag_a). You should see a DAG run for the ignored dag_a.
12. Go back, and click on the "Triggered DAG Run" (dag_b). You should see a DAG run for the ignored DAG.

dag_a

from airflow.sdk import Asset, DAG, task


with DAG(
    dag_id="dag_a",
    schedule="@once"
) as dag:

    @task(outlets=[Asset("asset_1")])
    def task_1():
        print("Running dag_a::task_1")

    task_1()

dag_b

from airflow.sdk import Asset, DAG, task


with DAG(
    dag_id="dag_b",
    schedule=[Asset("asset_1")]
) as dag:

    def task_1():
        print("Running dag_b::task_1")

    task_1()

Operating System

MacOS

Versions of Apache Airflow Providers

NA

Deployment

Other

Deployment details

Running on the main branch, locally.

Anything else?

No response

Are you willing to submit PR?

• [x] Yes I am willing to submit a PR!

Code of Conduct

• [x] I agree to follow this project's Code of Conduct

[jroachgolf84]

cc: @potiuk, @jscheffl

[potiuk]

Likely wrong API queries @pierrejeambrun @ephraimbuddy @uranusjr

[pierrejeambrun]

All related entities to a stale DAG should be removed from the UI?

Runs related to a Stale DAG shouldn't be ever displayed. Should we expand that to asset events etc and all other entities ?

Because that would be weird to block the redirect for 'runs' of staled DAG, but still show asset events produced by staled DAG. (same for xcoms and such).

[potiuk]

All related entities to a stale DAG should be removed from the UI?

Runs related to a Stale DAG shouldn't be ever displayed. Should we expand that to asset events etc and all other entities ?

Because that would be weird to block the redirect for 'runs' of staled DAG, but still show asset events produced by staled DAG. (same for xcoms and such).

Yeah sounds reasonable. I definitely think it should be consistent, and hiding the stale dags and assets seems like best idea.