airflow icon indicating copy to clipboard operation
airflow copied to clipboard

feat(ui/logs)_#47619: add toggle for log grouping

Open davidfgcorreia opened this issue 7 months ago • 4 comments

  • Add button in header to enable or disable log grouping
  • Allow logs to be shown linearly or grouped
  • Adjust group offset and display https://www.youtube.com/watch?v=doCR4YM1Rek

^ Add meaningful description above Read the Pull Request Guidelines for more information. In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed. In case of a new dependency, check compliance with the ASF 3rd Party License Policy. In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

davidfgcorreia avatar May 28 '25 10:05 davidfgcorreia

Closes #47619

Looks like we have a merge conflict too

bbovenzi avatar May 29 '25 17:05 bbovenzi

Functionality-wise. I am wondering if a "Expand/Collapse All" would be better than flattening the logs? I feel that if the logs have groups, we should respect that rather than possibly remove important information.

bbovenzi avatar May 29 '25 17:05 bbovenzi

I’d like to apologize for the incorrect merge I made earlier. I realize it may have caused confusion or disrupted the workflow.

davidfgcorreia avatar May 31 '25 15:05 davidfgcorreia

I would rebase after https://github.com/apache/airflow/pull/51666 is merged. And there shouldn't be too many translations you need to add.

bbovenzi avatar Jun 12 '25 17:06 bbovenzi

Hi, I'm having some trouble running the UI tests. I'm following the instructions in the runner and using pnpm test, but when I try to render a component like this:

<AppWrapper initialEntries={["/dags/log_grouping/runs/manual__2025-02-18T12:19/tasks/generate"]} />

I'm not getting the expected mocks. instead, I get a fetch error:

stderr | Fetch.onError (file:/root/documentos/PIC/airflow/airflow-core/src/airflow/ui/node_modules/.pnpm/[email protected]/node_modules/happy-dom/src/fetch/Fetch.ts:668:37) Error: connect ECONNREFUSED 127.0.0.1:3000 at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16) { errno: -111, code: 'ECONNREFUSED', syscall: 'connect', address: '127.0.0.1', port: 3000 }

It seems like the tests are trying to make a real network request instead of using mocks. Any idea what might be causing this or how I can properly mock the backend for this route?

Thanks in advance!

davidfgcorreia avatar Jun 20 '25 22:06 davidfgcorreia

Hi, thanks for bringing this up.

This is actually expected behavior. During testing, a few unnecessary requests may still be triggered. However, they have already been bypassed in the setup and should not interfere with the tests. The bypass logic is handled in airflow-core/src/airflow/ui/testSetup.ts. If you're still seeing fetch errors that cause test failures, please let me know as there might be something else going on.

Feel free to ask more if there is anything need clarification.

guan404ming avatar Jun 22 '25 15:06 guan404ming

Hi, thanks for the clarification!

That makes sense good to know the unnecessary requests are already being bypassed in the setup.

However, I’m still running into an issue after rebasing. When I try to run the frontend tests, I get the following error:

Error: Failed to resolve import "i18next-http-backend" from "src/i18n/config.ts". Does the file exist? Plugin: vite:import-analysis File: /root/documentos/PIC/airflow/airflow-core/src/airflow/ui/src/i18n/config.ts:21:20 18 | */ import i18n from "i18next"; 19 | import LanguageDetector from "i18next-browser-languagedetector"; 20 | import Backend from "i18next-http-backend"; | ^ 21 | import { initReactI18next } from "react-i18next";

Do you have any idea what might be causing this? It seems like the module isn't being resolved correctly after the rebase.

Thanks again!

davidfgcorreia avatar Jun 24 '25 10:06 davidfgcorreia

'i18next-http-backend' is likely missing in your node modules. How are you trying to run the UI. Did you refreshed the dependencies (yarn install).

pierrejeambrun avatar Jun 24 '25 11:06 pierrejeambrun

Thanks! You're right . I hadn't run yarn install after the rebase.

Just did it now and that fixed the issue. Appreciate the quick help!

davidfgcorreia avatar Jun 24 '25 12:06 davidfgcorreia

Now I’m hitting another problem when the pre-commit runs. It seems to fail during the TypeScript check with the following error:

src/pages/Error.tsx:45:29 - error TS2339: Property 'env' does not exist on type 'ImportMeta'.

45 const isDev = import.meta.env.DEV; ~~~ This happens even though I’m not touching that file. Looks like the internal ts_compile_lint_ui.py script compiles a temporary tsconfig.json, and it might not be picking up the proper Vite type declarations?

Let me know if this is something I should fix on my end or if the pre-commit script needs an adjustment.

davidfgcorreia avatar Jun 24 '25 12:06 davidfgcorreia

We need to add /// <reference types="vite/types/importMeta.d.ts" /> to airflow-core/src/airflow/ui/src/vite-env.d.ts to solve this.

I don't know why this is an issue suddenly

pierrejeambrun avatar Jun 26 '25 12:06 pierrejeambrun

Should I still go ahead and push the changes, even with this error.

davidfgcorreia avatar Jun 26 '25 14:06 davidfgcorreia

You can add the line to the file and push, it will silence the error. (I have updated my previous message, I didn't realize that markdown formatting was messing up what I wrote)

pierrejeambrun avatar Jun 26 '25 14:06 pierrejeambrun

Ok, I’ll take care of that and push the change.

davidfgcorreia avatar Jun 26 '25 14:06 davidfgcorreia

Everything that was requested has been addressed. Is there anything else needed for this PR to be merged? @pierrejeambrun

davidfgcorreia avatar Jul 02 '25 11:07 davidfgcorreia