apache
Scrolling in logs does not work properly.
Apache Airflow version
3.0.2
If "Other Airflow 2 version" selected, which one?
No response
What happened?
I did a clean installation of Airflow 3.0.2 and when trying to scroll the logs the following behavior occurs:
Even though it seems to be at the end, I need a few seconds to manually move down until I reach the end...
What you think should happen instead?
No response
How to reproduce
Fix Scroll behavior
Operating System
Running docker compose on Debian 12
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else?
Always
Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Code of Conduct
- [x] I agree to follow this project's Code of Conduct
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
![boring-cyborg[bot] avatar](https://avatars.githubusercontent.com/in/50386?v=4 "Published by a pub.dev verified publisher"){kind=small}
It seems to be a common challenge with the virtualization library (@tanstack/virtual) when it handles items with dynamic heights.
Related PR: #50746
I found that even the official library example for dynamic rows shows a similar behavior when you scroll: TanStack Virtual - Dynamic Example
Thanks, @RoyLee1224 and @WilliamKarvat! We introduced the virtualized list to improve rendering efficiency. However, this is a kind of a known issue with @tanstack/virtual. If the library releases a fix, we'll definitely follow up and update our component accordingly.
Actually going to leave this as priority:medium. Oftentimes the most recent logs (ie the logs at the end) are the most important. so if this gets in the way we might need to rethink how we do the virtualization. Ideally, we get https://github.com/apache/airflow/pull/49470 merged soon.
Not that I think this should change the priority for this issue, but in my scenario the most important logs are always at the end. Anyway, let's wait for the lib updates, thanks.
@bbovenzi https://github.com/apache/airflow/pull/49470 should resolve this issue?
It's finally merged. So now we can update the UI to utilize the full streaming API
Looking forward to seeing this fixed. We have some jobs that run a couple hours and produce big log files and after the upgrade from 2.x to 3.x we can no longer scroll them in the UI easily.
@pierrejeambrun do you have some bandwidth to check this?
It's finally merged. So now we can update the UI to utilize the full streaming API
Unfortunately I don't this I will have time to work on that before the 3.0.4 cut off. This item is a considerable effort and there are a lot of other bug fixes and PR for 3.0.4 to review.
I'll get back to it if I can.
@pierrejeambrun @phanikumv I don't think we can make it in time for 3.0.5 too right? Should we move it to 3.1?
@guan404ming do you have some bandwidth to finish the UI part? https://github.com/apache/airflow/pull/49470 is already merged!
It's finally merged. So now we can update the UI to utilize the full streaming API
I think it is almost done but it could be better after the backend fix to support fully streaming for logs.
That should also help closing this, with a "jump to end" button that is fixed in the following PR, that should help preventing this. (manually scrolling will still behave like this because of virtualization though, but we would have a workaround with the "jump to end" button) https://github.com/apache/airflow/pull/54447
The virtualized scrolling has made searching in the logs impossible. It would be amazing to have an option to just load all the logs into the ui at once and bypass the streaming.
