Specification
Specification copied to clipboard
Out of order IDs mixed with missing IDs
TAPv14 says:
Test Points may be output in any order, but any Test Point ID provided must be within the range described by the Plan.
TAPv14 also says:
If there is no number, the harness must maintain its own counter until the script supplies test numbers again.
How do these two interact? For example, how should a harness process the following document:
1..4
ok 2
ok
ok 1
ok 3
I can see several reasonable choices for the implied ID of that second test point:
- 1 because that was the lowest ID not yet affiliated with any test points when the second test point was observed
- 2 because the ID-less test point was the second test point observed
- 3 because that is one greater than the highest affiliated test point ID when the second test point was observed
- 4 because that was the only ID not affiliated with another test point at the conclusion of the TAP document
My opinion: If the Nth test point in the TAP document does not have an ID, then the harness gives it ID N even if ID N is already affiliated with another test point. Thus, the above stream would be equivalent to:
1..4
ok 2
ok 2
ok 1
ok 3
Rationale:
- It is the easiest for a harness to implement.
- A test point's ID is immediately known when the test point is observed.
- The ID is guaranteed to be within the plan's range unless there are too many test points.
- It would effectively require TAP document producers to always include the ID with their test points if the test points might be written out of order.
Thoughts?