bikeshed
bikeshed copied to clipboard
tabatkins
Generalize WPT
Some specs (e.g. AOM specs) don't use Web Platform Tests for the tests. It'd be good to have an element like WPT, i.e. that creates a TESTS block (with styles, can be opened and closed, etc.), but whose contents would be free-form text.
Do you actually want free-form text? In email discussions you talked about wanting to link test URLs to assertion IDs in the spec - I can format those nicely, with verification that the IDs exist.
(You can put notes into your test blocks with <wpt title> today, if you just want to provide some context.)
The email said free-form because I wasn't really aware of <assert>. Now that I know it, I'd like a mix of free-form (maybe through title) and URL + assert id references.
Initially what I had in mind was something like:
<testing>
The following links point to invalid files exercising the "shall" above. They should be reported as invalid by validation tools. Writers conformant to this specification should not produce them.
- http://.../invalid_color_primaries.obu
- http://.../invalid_transfer_characteristics.obu
- http://.../invalid_matrix_coefficients.obu
</testing>
Now I think more of:
<assert>A valid bitstream shall have X.</assert><assert>A valid bitstream shall not have Y</assert> and <assert>shall not have Z</assert>
<testing title="The following links point to valid files exercising the "shall" above. Validation tools should not produce any error when processing them.">
http://.../valid_color.obu [#X6789]
</testing>
<testing title="The following links point to invalid files exercising the "shall" above. They should be reported as invalid by validation tools. Writers conformant to this specification should not produce them.">
http://.../invalid_color_primaries.obu [#Y1234]
http://.../invalid_transfer_characteristics.obu [#Y1234]
http://.../invalid_matrix_coefficients.obu [#Z4567]
</testing>