Test format: add additional column to allow test reordering without loss of results
Currently, each test is identified in tests.csv using a test number (starting at 1) and a task string. FOr example:
1,Trigger an alert in reading mode,"JAWS,NVDA",reading,trigger alert,...
(note: other columns omited)
In this example, the test number is 1, and the task string is "trigger alert". If we needed to insert a test before this one, we would number that new test as 1, and renumber the example and all subsequent tests accordingly.
However, this has the downside of the ARIA-AT App being unable to track this change. Specifically, the renumbered tests would look like completely new ones, because there is no suitable identifier that will persist. The task string is not suitable, as it is shared across multiple tests to allow commands to be grouped appropriately in commands.csv.
So, in short, any existing results would no longer be applicable, and all renumbered tests would need to be retested.
Proposed Solution
Add a new column to tests.csv, representing a fixed test ID. The current first column can be renamed to testNumber or similar, and the new column can be named testId instead.
In terms of ID format, it is not proposed that this be specifically prescribed; anything that the app code is able to compare will suffice. For instance, a computer-generated UUID, or even just numbers.
Open question: does this new ID need to exist in commands.csv at all?
CC @richnoah / @s3ththompson