Mockups for Test Plan Management, Test Queue, and Candidate Test Plans

[isaacdurazo]

Test Plan Management, Test Queue, and Candidate Test Plans Mockups

Test Plan Management - Text-version Mockup

From this new page, Admins can control all aspects of Test Plans, meaning some features have moved from other pages to this one.

Manage Assistive Technology Versions

This Disclosure component was previously displayed in the Test Queue. It's now been moved to this page.

Add Test Plan and Assistive Technology combinations

Previously displayed in the Test Queue as "Add Test Plans to the Test queue", this Disclosure component has now been moved to this page.

Status Summary table

This table displays all Test Plans currently added to the ARIA AT App. From here, Admins can control their Phase from a high level, meaning a Test Plan and all its AT and browser combinations are moved from one phase to another at once. The details of this table are these:

• Test Plans column: Lists all Test Plans that have been added to the ARIA AT App
• Phase column: Lists a dropdown for each test plan to modify its Phase: Draft, Candidate, and Recommended
• Status column: Displays the high-level status in which a Test Plan is. Depending on the Phase a Test plan is in, the status could be different. Here are the details. Please note some scenarios could be missing.
  • Draft:
    • 0 Testing Rounds Completed
    • X of Y Testing Rounds Completed
  • Candidate
    • 0 Reviews Completed
    • X of Y ReviewsCompleted
  • Recommended
    • Approved

Assistive Technology Table

This table displays all Test Plans under a particular AT and the different Browsers in which these are being tested. The Test Plan Management page includes multiple Assistive Technology Tables, one for each Assistive Technology recognized by ARIA-AT.

• Test Plans column: Every cell in this column has a Test Plan name, the phase in which it is next to it, and the published date underneath. There's also a plain text "Update" link (which should be a button but wanted to make it look less prominent) next to the date. The Publish date and Update buttons have been moved from the Test Queue to this page.
• Browser and Status column: There could be more than one of these columns depending on how many Browsers we have available. This column displays the status in which a particular Test Plan being run with a particular AT and Browser combination is The statuses depend on the phase and could be the following:
  • Draft:
    • Testing Round in Progress
    • Testing Round Completed
  • Candidate
    • Review in Progress
    • Review Completed
  • Recommended
    • Approved

Test Plan Management - Visual Mockup

Test Queue - Text-version Mockup

This page is now primarily for Testers, with all Admin actions moved to the Test Plan Management page. A few more changes have been made to improve the UX. Here are the details:

1. "Manage AT Version" and "Add Test Plans to the Test Queue" disclosures have been moved to the new Test Plan Management page.
2. Details of the new AT/Browser table
   • Test Plan Column:
     • Displays the name of the Test Plan as a link if the user is assigned or as plain text if they are not
     • The Phase is displayed next to the Test Plan name. We used to have a dedicated column called "Report Status" for this, but using the word status is confusing. There's really no need to have a column for this anymore because we don't control it from here but from the Test Plan Management page.
     • "Open Run as...", "Delete Test Results for..." and "Remove Test Plan", have been moved to be displayed under the Test Plan name. For sighted users, we can have these hidden by default and appear on hover, which could make the UI look cleaner.
   • Testing Status. There are different statuses a Test Plan could be in. A Test plan can have multiple statuses at once. These statuses pertain only to "testing" and not to other kinds such as "reviewing". Here are the details:
     • No Testing Round started yet
     • X Testing Rounds in progress
     • Testing Round Completed
     • X Testing Rounds Completed
     • X Conflicts
   • Testers. This column doesn't have any changes

Test Queue - Visual Mockup

Candidate Test Plans - Text-version Mockup

This page hasn't changed much. Here are the details:

1. The "Review Status Summary" table has been moved to the top so the user can take a look at a high level first and be consistent with the similar summary table we have on the Test Plan Management page
2. The "Mark as..." button has been removed. That Admin action is now on the Test Plan Management page.

Candidate Test Plans - Visual Mockup

Afterthoughts

These new mockups represent major changes to the ARIA AT App and the Working mode. There are still some questions that remain unanswered that should be discussed. Here a few I was running into while I worked on these mockups with Seth last year:

1. What happens in the following scenario? I add Disclosure Nav Example with JAWS. It then moves from Draft to Candidate, then later I add Disclosure Nav Example with VO, which theoretically should start as a Draft. Would both Test Plans be in different phases? They can't. So in a way, the Test Plan review process should be like a train leaving the station. after the first group of ATs is published as Recommended, any additional ATs that we add should start as recommended too.

2. What are the minimum requirements to move a test from "draft to candidate"? Do we need to specify an X number of finished testing rounds with no conflicts?

3. What happens if we need to update a Test Plan that has been moved to recommended? Should it go back to Draft?

4. Similarly, what about the scenario where we might need to add more test results to a Test Plan that has moved to Candidate or Recommended?

There are probably more things I'm not considering, hence I think a broad discussion should be had about these mockups to see how we can address these questions and their impact.