Add analysis progress bar

[ebugden]

trafficstars

Adding a progress bar in the view panel would help users know:

• Is the analysis working? Seeing how fast things are going would help users notice unusual stalls, crashes, etc.
• How long will the analysis take? This is useful if the analysis will take more than a couple seconds to complete (ex. more than 10-15 seconds).

If analyses are functional and typically take a few seconds, a progress bar is not that important.

Do analyses typically take more than a few seconds to complete?

It depends on file size. It seems like a progress bar would be relevant in corner cases (beginners, tracing complex applications) more than than in a typical workflow. Beginners can easily generate large trace files by accident. Some tracing applications (ex. HPC) can also easily generate large files.

• How large are the trace files that TC users are typically analyzing?
  • Depends on tracing experience: Most of the time, the trace files do not exceed 1GB if you enable just what you need, and are searching through 1 minute or less of trace. But if you're new to tracing it's quite easy to accidentally get large files.
  • Depends on application: With HPC programs or deep learning programs, we can generate >10 GB of trace in a couple of minutes.
• How long do analyses typically take on files this size? (ex. Couple minutes?)
  • If your trace is less than 1 GB, the analysis and indexing take less than 10 seconds
  • Above 3GB is a good place to warn users that it will require a few seconds to process the trace. Usually most analysis will work, but if some of them are not implemented correctly it may crash.

Context above from @arfio.

Implementation

The trace compass server is not designed to show the progress of an analysis so this would require some serious renovations.

Easier alternatives to an analysis progress bar:

• #428
• #424

Details from comment below:

bhufmann: the Trace Compass server has not implemented a way to know how far a particular analysis (and data provider) is progressed. It is feasible, but the change in Trace Compass to support progress of analysis is significant. It needs a Trace Compass framework enhancement and each analysis (data provider) needs to use this enhancement. BTW, because of that the Trace Compass Eclipse application doesn't show the progress either and only shows a spinning progress.

bhufmann: TSP responses currently return a status COMPLETED, RUNNING or FAILED. For RUNNING, we could return an optional percentage number that could be used in a progress bar in the front-end. Front-end would have to poll the back-end repeatedly to get the progress in the form of the percentage number.

Once we have a percentage, to display as a bar maybe we can reuse code from the bar that shows the progress of traces being merged?

Bar could be placed here:

This is a specific case of #56

[bhufmann]

#340 doesn't show an actual progress bar. I shows a spinning wheel to indicate that analysis is running. Re-opening this issue.

[ebugden]

Would a progress bar be quick to add? (Right now, does the server already periodically update the front end on its progress?) Could we imitate what is done for the "Merging traces" progress bar?

[thefinaljob]

Proposed Solution for Implementation

1. Modify tsp response so that it sends a numeric value as a response for the status, as a calculation of the work done so far in the backend
2. Use this response to fill in a progress bar with the numeric value from the tsp response
3. Destroy the progress bar upon completion and load the chart

Would this be a good way to proceed? @bhufmann @ebugden

[ebugden]

@arfio @MatthewKhouzam @tahini Could one of you also weigh in on this? I'd like to have approval from two committers before implementing especially if it involves changes to the tsp response.

@thefinaljob Could you point to where you are thinking of making changes in the server code?

[ebugden]

If #54 is still an issue, it could be relevant to this task. @tahini could you confirm if this is relevant?

[bhufmann]

Proposed Solution for Implementation

1. Modify tsp response so that it sends a numeric value as a response for the status, as a calculation of the work done so far in the backend

2. Use this response to fill in a progress bar with the numeric value from the tsp response

3. Destroy the progress bar upon completion and load the chart

Would this be a good way to proceed? @bhufmann @ebugden

@ebugden, the TSP responses currently return a status COMPLETED, RUNNING or FAILED. For RUNNING, we could return an optional percentage number that could be used in a progress bar in the front-end. Front-end would have to poll the back-end repeatedly to get the progress in the form of the percentage number.

Please note, that the Trace Compass server, has not implemented a way to know how far a particular analysis (and data provider) is progressed. It is feasible, but the change in Trace Compass to support progress of analysis is significant. It needs a Trace Compass framework enhancement and each analysis (data provider) needs to use this enhancement. BTW, because of that the Trace Compass Eclipse application doesn't show the progress either and only shows a spinning progress.

[ebugden]

Would there be a way to implement where new data providers (and maybe select old ones) would have a progress bar, but not all existing data providers would need to be changed?

My understanding is that analysis of large files can take minutes. It would be important to give the user a reference for how long analysis will take otherwise they'll likely think the analysis is stuck or broken.

[ebugden]

@arfio: For me an indexing loading bar is very useful and is easier to implement. A loading bar for the analyses is harder to implement and we can look at that later.

Update: An indexing progress bar is not an alternative to an analysis progress bar since it would give no information about whether an analysis is progressing/functional (indexing is independent of any analysis).

~~After some discussion, we're thinking of first implementing a progress bar for indexing #422:~~

• Easier to implement
• ~~Will still give users a reference for whether server processing is progressing (and how quickly it's going)~~

Since an analysis progress bar requires more extensive changes we'll come back to it later. @arfio May explore how tricky this is to implement.

[ebugden]

Moving to low priority since:

• If it's clear whether analyses are functional or not (This should be well taken care of by: #460, #406, #296)
• And if when they do, they typically take a few seconds (According to arfio: Most of the time, the trace files do not exceed 1GB if you enable just what you need, and are searching through 1 minute or less of trace. And if your trace is less than 1 GB, the analysis and indexing take less than 10 seconds)

a progress bar is not that important.

[MatthewKhouzam]

Upon reviewing this, most modern platforms eschew the progress bar in favor of a spinner. (e.g. Gitpod) I think we need to follow their model here. I would suggest this is wontfix