IPED
IPED copied to clipboard
sepinf-inc
Baseline draft of timeline graph to promote suggestions of additional implementations.
Baseline draft of timeline graph to promote suggestions of additional implementations. Make your appointments and suggestions.
Thank you very much @patrickdalla!!!
I'll try to do a basic test today to give you feedback, but I'm not sure if I'll be able to since I'm finishing some thinks to travel on vacation tomorrow. I'll be back next week.
Good vacation.
For anyone that wants to try: to select intervals on the chart, press shift, click with the mouse pointer on the desired date and drag till the end date.
To change the granularity of the date unit accumulation, click on the Domain Axis Label, and a popup menu will be shown.
Selecting and checking items can be done based on the corresponding bar clicked.
Select bookmarks and one chart will be plotted for each selected bookmark.
To hide or show specific Event Series on all plotted graph, click on the corresponding legend.
Some popup menu item will still be implemented.
To zoom in, press and drag mouse.
To pan on the domain, press ctrl+mouse and drag the chart.
This is a GREAT work, thank you @patrickdalla!
I kindly ask other users/devs to help validating the usage of this great feature. It possibly needs some optimizations, but we would like to finish the usage scenarios before @patrickdalla can proceed with the final adjustments/optimizations. It can be used on already processed 4.0 cases, as the needed index structures are already there, you just need to replace the [case]/lib folder with the one from this PR and update the conf/ResultsetViewersConf.xml file.
@patrickdalla something has gone wrong with the remote build, it failed, could you take a look? https://github.com/sepinf-inc/IPED/actions/runs/2571687668
This functionality is really fantastic. I would like to help test, where could I download a trial version?
I would like to help test, where could I download a trial version?
Hi @paulobreim, thank you for helping with tests, this is really important. Just to reinforce this isn't optimized yet, I suggest not testing this on cases with millions of items for now. I just pushed some compilation fixes, I think @patrickdalla forgot to push some changes. You can download a snapshot from: https://github.com/sepinf-inc/IPED/actions/runs/2572429182
If you want to try on a 4.0 case already processed, beyond replacing its whole [case]/lib folder as I said, you also need to replace the conf/ResultSetViewersConf.xml file into the case.
@patrickdalla I did some very very basic tests, this is a really awesome work! Some preliminary suggestions from my side:
- some progress bar and cancelation button while timeline chart is loading would be good as it can take seconds with medium size cases (for now), freezing the UI;
- an explicit option/button to clear the applied range filter on the new tab, like in other panels/filters, would allow user to clear just the date range filter, possibly done by mistake by him
- change new Tab title color to Red when filtering like other filters would make sense
- I noticed the "esconder no gráfico" option is inverted
- decreasing unused blank area on the left of Y axis would be good
Proposals for discussion:
- changing the default selection behavior to filter instead of selecting items on table or maybe show some popup so user could choose the action? I think filtering would be much faster.
- add explicit input boxes/popup so user can precisely define start/end of timestamp range selection?
TODO:
- more exhaustive testing and possible fixes
- localization to other languages
- optimizations
I would really appreciate more opinions from other devs...
PS: the new tab should implement the ClearFilterListener, so it would be cleared like other filters when user click on the "Clear Filters" global button
Selecting and checking items can be done based on the corresponding bar clicked.
This is really nice!
Proposals for discussion:
Other: the zoom on the X axis is really nice! Maybe it could be also applied to Y axis? When zooming in a lot edited: and changing X scale to be more and more granular, bars become very small, making it difficult to see height differences... The Y scale change animation may have some abrupt changes when you already see high and small bars at the same time and zoom in the small ones, maybe some scale smoothness change could be applied...
I forgot to tell other devs, @patrickdalla is going to work most his work time dedicated to this project for the next months after some negotiation between chiefs, so almost a 2x increase in permanent dedicated people :-)
Select bookmarks and one chart will be plotted for each selected bookmark.
This is also really really nice to search for correlations between different event groups, thank you! e.g. different sets of events from different users/actions (independent of event type), or from different hosts...
Originally I thought about plotting them on the same chart, using different colors and making some kind of overlapping, this could be good to see correlations. But your approach allows to still see the different event types/colors.
- possible fixes
When zooming out the chart till the end, the UI freezes indefinitely.
- possible fixes
Sometimes, after zooming in after applying some range filter, trying to select a new interval to apply another range filter seems to use the previous zoom/scale and everything in the chart is selected.
- optimizations
When changing X axis scale to be more granular, not sure, but seems the whole dataset is clustered again. Maybe just items in the visible area of the chart could be clustered again before being plotted, or within some neighborhood to allow horizontal dragging/translation
Other: the zoom on the X axis is really nice! Maybe it could be also applied to Y axis? When zooming in a lot edited: and changing X scale to be more and more granular, bars become very small, making it difficult to see height differences... That animation may have some abrupt changes when you already see high and small bars at the same time and zoom in the small ones, maybe some scale smoothness change could be applied...
Or maybe just allowing user to change the Y axis granularity/interval manually like in the X axis would be enough to workaround those very small bars.
suggestions
Bars are rendered with a thin vertical left bright line:
I suggest trying to remove them, because when the zoom is far away, seems the chart has some gaps, but it hasn't:
Rounding issues also make those bright lines to be rendered at different places depending on the zoom:
I'm trying to implement filters on the evenType filter, but it doen't work. Even if I write the filter on the search inputbox.
Just one comment, I think the user could use the lower-left Metadata filter panel to filter by timeEvent, so maybe there is no need to make current graph event type filter more complex, it's fine to me as is.
well, it is already done :-)
Em sex., 1 de jul. de 2022 07:54, Luis Filipe Nassif < @.***> escreveu:
Just one comment, I think the user could use the lower-left Metadata filter panel to filter by timeEvent, so maybe there is no need to make current graph event type filter more complex, it's fine to me as is.
— Reply to this email directly, view it on GitHub https://github.com/sepinf-inc/IPED/pull/1193#issuecomment-1172264913, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG247S4GG7CTRL4TFNMIIODVR3L5VANCNFSM5Z7UFT5A . You are receiving this because you modified the open/close state.Message ID: @.***>
I saw you reopened. In fact, when I commented about the "timeEvent" query, I had already found the solution, and wanted to cancel the comment, but I clicked "close with comment". I am not so used with this site, sorry.
Nassif, I think I have found an error in the table data change event dispatch. Now, the updateFileListing method is calling "App.get().resultsModel.fireTableDataChanged()" before the search is really completed. It doesn't interfere, for now, on the final result as ResultTableRowSorter also calls "App.get().resultsModel.fireTableDataChanged()" after the sorting, and so the search, is complete. But I need to differentiate between real data content change and only sorting change, as the timeline doesn't need to change on row sorting change. So I created a TableModelEvent subclass RowSorterTableDataChange, and all sorting events are dispatched as an object of this class. But as I found that "error" of the first paragraph, I need to change this. So I removed "App.get().resultsModel.fireTableDataChanged()" and put it on UICaseSearcherFilter.done(). It worked for the timeline, and it seems that it didn't affect other viewers.
Nassif, I think I have found an error in the table data change event dispatch. Now, the updateFileListing method is calling "App.get().resultsModel.fireTableDataChanged()" before the search is really completed. It doesn't interfere, for now, on the final result as ResultTableRowSorter also calls "App.get().resultsModel.fireTableDataChanged()" after the sorting, and so the search, is complete. But I need to differentiate between real data content change and only sorting change, as the timeline doesn't need to change on row sorting change. So I created a TableModelEvent subclass RowSorterTableDataChange, and all sorting events are dispatched as an object of this class. But as I found that "error" of the first paragraph, I need to change this. So I removed "App.get().resultsModel.fireTableDataChanged()" and put it on UICaseSearcherFilter.done(). It worked for the timeline, and it seems that it didn't affect other viewers.
Hi @patrickdalla. If I remember correctly, another table data change event is also fired before the search to clean up table data, so if some issue occurs or if the search is canceled, no old data remains on table. But I understood you refer to another table data change event, I agree with you, sorting changes shouldn't fire table data change events, thanks for finding and improving this!
I needed a method to get a result set from all configured ui filters, except the filters configured by the own timeline. Something very similar to what the class UICaseSearcher do, but this class contains coupled some code to changed the UI base on the result, what I didn't need. So I've created a copy of this class called CaseSearchFilter and removed the UI changing code. I have created an interface called CaseSearchFilterListener to be add in a CaseSearchFilter to execute code in some phases of the execution of the search. It worked to the timeline. I think that, to keep this repeated code centralized, we could use it on the prior use case, in the class AppListener updateFileListing method, implementing a different listener to do the UI changing after the search result. What do you think?
I have also created an interface called IQueryFilterer, that has 2 methods, getQuery and hasFiltersApplied. We could use it, so the app search for all registered objects that implements IQueryFilterer to get the respective query to be applied on the final result set of CaseSearchFilter. I have implemented the timeline this way, so take a look and tell me what do you thing to adapt other classes?