Added Feature: search dags by task id with suggestions

[pruoff]

This PR closes: #37106

Previously the DAG search bar could be used to search by DAG id and owner.

Changes in /home UI from user's perspective:

• DAG search with prefix "task: <task_id>" filters all DAGs by case-insensitive pattern-matching of the provided task_id.
• suggestions of task_ids are automatically shown when typing. Since the same task_id can be present in many dags, an italic " in <dag_id>" is shown following the task_id suggestion.
• actively limit the DAG search scope by prefixing "dag:" for dag_id search and "owner:" for owner search.
• autocompleted suggestions are prefixed by their type e.g. "owner: airflow"

[eladkal]

@pruoff can you rebase and resolve conflicts?

[jscheffl]

I checked and I have no auto-complete and suggestions when I type as described in the PR descriptions. Tested with Firefox and Chromium. Did I mis-undertstand the PR that suggestions are not there? Or are the suggestions browser specific?

Also when not prefixing dag: in the search the text entered always filters for the DAG ID. Search for tasks and owner though is working. But dag: prefix smells un-needed if this is the default. Or shall the default be to make a global search?

[pruoff]

@jscheffl

I checked and I have no auto-complete and suggestions when I type as described in the PR descriptions. Tested with Firefox and Chromium. Did I mis-undertstand the PR that suggestions are not there? Or are the suggestions browser specific?

A bug was introduced when rebasing, which caused the autocomplete to fail - see the FIX commit. The feature works for me now in both Firefox and Chromium.

Also when not prefixing dag: in the search the text entered always filters for the DAG ID. Search for tasks and owner though is working. But dag: prefix smells un-needed if this is the default. Or shall the default be to make a global search?

The default is to search by both dag id and owner. The dag: prefix resolves confusion in case a dag id and owner are named the same.

[pruoff]

Thanks for the fix, I can confirm it works now as advertised.

I approve from my side. I just kindly request to also alter the documentation a bit. Else the risk is high that nobody will be able to find and discover this cool freature - especially searching for tasks. Can you add / adjust documentation in docs/apache-airflow/ui.rst

Thanks a lot! I now also extended the UI docs.

[jscheffl]

@bbovenzi or @eladkal Can yo umake a 2nd pass review, then I thing we should merge this.

[pruoff]

Let's rebase.

Quite some effort this time. Included the display_dag_name logic

[pruoff]

Do we want to include searching by task_display_name in this PR or in a follow up?

Searching by task_display_name is already included and the PR does not change this.

[pruoff]

@ashb thanks for the review. I share your concern about resource consumption issues when loading all dags from the DB.

I tried to query the JSON via many ways. One would be: session.query(SerializedDagModel).filter(SerializedDagModel.__table__.columns.data['dag_id'].astext == dag_id) yet I always end up in NotImplementedError: Operator 'getitem' is not supported on this expression.

Can somebody support me here with a minimal working code snippet? In the repo, I cannot find any comparable code that queries the serialized dags in the DB.

[pruoff]

Here it finally is! I invested many hours in finding the correct methods. Asking for help did not create any responses neither here nor on Slack. There’s only one single occurrence of func.json_extract in the code that I found more or less by accident.

Tested with MySQL, Postgres, and sqlite; sqlite supports func.json_extract but nothing like func.json_keys so I kept the original code for that backend.

[bbovenzi]

Looking better! I think we should add an info tooltip telling users the new keywords search.

[ashb]

(@pruoff And sorry for not noticing the original ping! My GH notifications are a mess)