UX and UI Revisions to Test Queue and Candidate Test Plan pages

[isaacdurazo]

Some of the changes proposed in issue #648 have an impact on the Test Queue and Candidate Test Plan pages. The majority of them relate to visual inconsistencies. There are also some opportunities to simplify the UI and UX.

Before putting together a revised mockup for these two pages, I spent some time reviewing the impact of these changes. I ran into some questions as well as some ideas I would like to discuss:

Test Queue

1. Do we need to keep the "Manage AT Versions" and "Add Test Plans to the Test Queue" disclosures on this page? These disclosures are already under Data Management

2. We should rename the “Report Status” column to “Overall Status” to make it consistent with the Data Management page

3. The “Update Test Plan” button’s functionality only applies to a single AT/Browser combination under a given Test Plan. Shouldn’t this action apply to the Test Plan as a whole? Meaning for all the AT/Browser combinations under a given Test Plan?

4. Do we want to keep the “Mark as Candidate” button on this page or do we want to only have it on the Data Management page?

Candidate Tests

1. I have notes about the “Mark as..” Button. This button currently moves a Test Plan to Recommended or Draft.
   • We don’t have the option to move a Test plan back to Draft in the new Test Plan Status Summary table, why have it here?
   • Similar to my question about the “Mark as Candidate” button in the test queue, do we need to keep this functionality on this page?
2. I think we should move the Review Status Summary table to the top so we have a high-level view first.
3. The current statuses under the Review Status column might not be consistent with the statuses in the new Test Plans Status Summary.

[jscholes]

Test Queue

Do we need to keep the "Manage AT Versions" and "Add Test Plans to the Test Queue" disclosures on this page? These disclosures are already under Data Management.

I don't think we need the first one ("Manage AT Versions"), but an "Add Test Plans to the Test Queue" button on the Test Queue page makes sense to me as a shortcut of sorts. If there are reasons it should only live on the Data Management page, I'm also okay with its removal, but having it just do the same thing seems fine.

We should rename the “Report Status” column to “Overall Status” to make it consistent with the Data Management page

I think this makes sense, because the test queue tracks test plan versions, not reports. However, I want a gut check from @mcking65 on this one.

The “Update Test Plan” button’s functionality only applies to a single AT/Browser combination under a given Test Plan. Shouldn’t this action apply to the Test Plan as a whole? Meaning for all the AT/Browser combinations under a given Test Plan?

Also interested in @mcking65's thoughts here. Currently, we update the version of a test plan for a specific test plan run, but it might be more convenient to update the versions of a test plan targeted by multiple test plan runs at once. However, we have to be careful in cases where the queue contains multiple test plan runs targeted at different versions of the plan already.

Do we want to keep the “Mark as Candidate” button on this page or do we want to only have it on the Data Management page?

There are times when we complete a set of results for a given test plan run, and want to publish those results to a candidate report without publishing them all. If the Data Management page also allows that flow, I'm okay with this functionality beinv removed from the Test Queue.

Candidate Tests

We don’t have the option to move a Test plan back to Draft in the new Test Plan Status Summary table, why have it here?

I'm not sure if it should be in the summary table or not, but it is possible that we publish something as candidate by accident, I suppose? I feel like the app UI should let us revert all changes we make in the same UI, just in case.

Similar to my question about the “Mark as Candidate” button in the test queue, do we need to keep this functionality on this page?

I don't understand what that button would actually do here. If something is already in candidate, and present on Candidate Tests, how can it be marked as candidate again? IIRC, this came up as a point of confusion live on a call with Vispero today.

I think we should move the Review Status Summary table to the top so we have a high-level view first.

This sounds good, paging @mcking65 for thoughts.

The current statuses under the Review Status column might not be consistent with the statuses in the new Test Plans Status Summary.

Not sure what you mean by this.

[isaacdurazo]

Thanks for your response @jscholes. To add more context regarding:

The current statuses under the Review Status column might not be consistent with the statuses in the new Test Plans Status Summary.

The new Test Plans Status Summary table presented in #648, uses strings such as "Review Completed DATE" and "Approved DATE" for R&D Version, Draft Review, Candidate Review, and Recommended Versions columns. In the current version of the Candidate page, under the "Review Status" column, we use: "Review in Progress" and "Ready for Review". Should we try to make these more consistent?

[ccanash]

Hi @mcking65 , would you be available to review this this week? We are working on the mockups and want to start implementation in the next sprint. Thanks

[mcking65]

@isaacdurazo wrote:

Test Queue

1. Do we need to keep the "Manage AT Versions" and "Add Test Plans to the Test Queue" disclosures on this page? These disclosures are already under Data Management

Agree with James that we might want to keep the ability to add to test queue since this is the test queue page. It is not necessary to keep manage AT versions. That said, adding to the test queue from the report status dialog is likely going to be more efficient. Making changes to these disclosures is P2 or lower priority because only admins see them.

2. We should rename the “Report Status” column to “Overall Status” to make it consistent with the Data Management page

It is two different kinds of data so I think the names should be different.

Overall status on the test management page is about the status of the test plan, e.g., how far it has advanced through the working mode. The report status column on the test queue page is about the status of report generation for a single AT/Browser combination for a specific version of a test plan.

3. The “Update Test Plan” button’s functionality only applies to a single AT/Browser combination under a given Test Plan. Shouldn’t this action apply to the Test Plan as a whole? Meaning for all the AT/Browser combinations under a given Test Plan?

You are right that this button is now "out of place" because it is not properly scoped. This button was kind of a stop gap solution because we didn't have the test management page.

If version X of a test plan has one or more runs in the queue, and if version X+1 becomes available, then we would update the tests for all runs of that plan at one time; we should not be able to update a subset of runs.

With the test management page updates, this happens when version x+1 is advanced to the same phase as version X. For example, if version X is in Draft phase and version X+1 is in R&D Complete, the way we would update the runs of version X that are in the test queue is to advance version X+1 to draft. This will sunset version X and update the runs of that plan to use the tests as written in version X+1. Test results for tests that have not changed will be copied into new runs for version X+1.

The net is that the update test plan button will eventually need to be removed from the tables on the test queue page as it will become obsolete.

4. Do we want to keep the “Mark as Candidate” button on this page or do we want to only have it on the Data Management page?

We will not advance plans to candidate from the test queue; that will only happen from the test management page. This is another button with incorrect scope. We don't advance an individual AT/Browser report to candidate, we advance a version of a test plan to candidate.

However, we still need that button; it just needs to be changed slightly. It should be named "Mark as Final". I described the functionality in issue 648.

When the admin deems the test plan results to be sufficient, the admin can mark the run final. This does two things:

1. Marks the report as final in the system.
2. Removes the run from the test queue.

Final simply means that work to generate the report is done, so the report can be used by the reporting system. How it is used depends on the phase of the test plan version that was used to generate the report.

• If the test plan version is recommended, the final report is available from the reports page.
• If the test plan version is candidate, the final report is available on the reports page and the data is shown when viewing the candidate test plan from the candidate test page.
• If the test plan version is draft, the final report can be accessed from the report status dialog on the test management page.

Candidate Tests

1. I have notes about the “Mark as..” Button. This button currently moves a Test Plan to Recommended or Draft.

   • We don’t have the option to move a Test plan back to Draft in the new Test Plan Status Summary table, why have it here?
   • Similar to my question about the “Mark as Candidate” button in the test queue, do we need to keep this functionality on this page?

We should remove that button.

2. I think we should move the Review Status Summary table to the top so we have a high-level view first

Maybe someday. It is at the bottom mostly because it is more of an admin table. I vote for not spending any cycles on that table for now. We may want to improve this table in other ways too if we have the cycles. This kind of change isn't even close to P2 yet.

3. The current statuses under the Review Status column might not be consistent with the statuses in the new Test Plans Status Summary.

Right, but not even a P2 priority yet.

[isaacdurazo]

@mcking65 It appears that the only change required for the Test Queue at the moment is to rename the "Mark as Candidate" button to "Mark as Final" Correct?

I'm wondering if we should also incorporate into the "Report Status" column, status messages such as: "Required Reports Not Started", "Required Reports In Progress", and "Required Reports Complete", etc from the Data Management page. Do you have any thoughts on that?

[mcking65]

@isaacdurazo asked:

@mcking65 It appears that the only change required for the Test Queue at the moment is to rename the "Mark as Candidate" button to "Mark as Final" Correct?

Yes, that is all that is necessary.

However, one very simple update that would add clarity and consistency would be to also make the rendering of the test plan version in the test plan column match the rendering on the test management page. Currently, the test queue page shows the version with format:

Published Apr 13, 2023 at 4:22:06 pm UTC

This should change to:

V2023-04-13

@isaacdurazo asks:

I'm wondering if we should also incorporate into the "Report Status" column, status messages such as: "Required Reports Not Started", "Required Reports In Progress", and "Required Reports Complete", etc from the Data Management page. Do you have any thoughts on that?

Those status values don't align with the data in the test queue. Each row in the test queue represents runs for one report for the test plan. Those status values describe the status of all reports for a test plan, so they use the plural, "reports".

The report status column on the test queue page should eventually be improved. However, I don't think we should invest in designing that yet as we probably want to make some larger changes to the table structure. The table structure is not well-aligned with how we work.

[mcking65]

@isaacdurazo

I spent some more time reviewing the candidate tests page. I think there are some stopgap changes that should be made. We will eventually revisit the overall design of the page, but these changes could help delay the need for that larger effort.

@ccanash, you may want to break these into separate tasks so they can be planned and sequenced appropriately.

• [ ] P0: Add test plan version to the test plan column.
• [ ] P0: Remove Change report status button from test plan column, per @isaacdurazo recommendation.
• [ ] P1: Remove Change Target Date button from test plan column after change target date function is available on test management page, per suggestion by @isaacdurazo.
• [ ] P1: Move Candidate Phase Start Date and Target Completion Date out of test plan column into separate columns.
• [ ] P2: Change page title from "Candidate Tests" to "Candidate Review".