dagu icon indicating copy to clipboard operation
dagu copied to clipboard

Timeline graph should show history

Open tapir opened this issue 9 months ago • 10 comments

The chart allows me to scroll to past days but there is no data shown. I would expect it to keep a certain amount of history so that I could compare run times from day to day

Thanks

tapir avatar Feb 27 '25 10:02 tapir

That sounds like a great idea. Does timeline refer to the chart displayed on the DAGs status page? Image Thanks!

yottahmd avatar Feb 27 '25 10:02 yottahmd

I was actually referring to the one in the mainpage where I can see all my jobs. But job specific timeline is also a good idea to implement this.

tapir avatar Feb 27 '25 10:02 tapir

Oh, I see. Just to clarify, at the moment, the timeline only shows the latest result for each DAG. Do you still think it’s useful to display detailed historical data for each run, even though only the most recent one is shown per DAG?

yottahmd avatar Feb 27 '25 12:02 yottahmd

Aaah gotcha. Didn't know that was the case. Then I guess it doesn't make sense but I think it would be nice to see every single run for each DAG and be able to filter them via status (show only success, fail etc...). Then it would make sense to show history too.

I understand if it's too much. Currently I think the dashboard timeline is not that helpful if it's only showing the latest.

tapir avatar Feb 27 '25 12:02 tapir

Yes, initially, the dashboard timeline was designed for DAGs that run on a daily basis (it has an option to display only today’s runs). Showing all historical data is possible, but performance issues may arise since Dagu stores history data on the local disk.

If there are many requests, we could develop a separate analytics dashboard to collect historical data from Dagu instances, allowing any complex queries, I think.

yottahmd avatar Feb 27 '25 12:02 yottahmd

I think historical timeline was discussed earlier as well:

https://discord.com/channels/1095289480774172772/1095291629344149544/1322389653449474070

@kriyanshii has implemented status wise search. If it is worth it, I think she would be happy to submit a PR for the same.

Regards Ghansham

ghansham avatar Feb 28 '25 12:02 ghansham

Oh thank you!

yottahmd avatar Feb 28 '25 15:02 yottahmd

thank you for the mention! I will submit the pr by weekend if @yottahmd want!

kriyanshii avatar Feb 28 '25 16:02 kriyanshii

Right now, I'm still in the process of making some necessary improvements related to persistence, and it seems we'll probably need to adjust the API to support specifying a time period for status-wise search, so perhaps it's better to hold off for now. But thank you so much for your kind offer!

yottahmd avatar Mar 01 '25 03:03 yottahmd

I also feel if we can improve the history data structure to include status in history filename, things may improve drastically. But let us keep this on our todo list as it will be a very useful feature.

Right now adding timeline for history runs may be doable.

ghansham avatar Mar 01 '25 04:03 ghansham