brimdata
In-line doc example tests that are not shown to the reader
The opening text of #2901 describes how it was a stopgap effort to give us some test coverage on docs examples where we want to show abbreviated Zed syntax or spare the reader from having to see all the inputs/outputs just so a test can be executed by mdtest in its present form. In a discussion with @nwt, we agreed that a better solution would be to have the test-not-shown-in-the-doc actually still live as part of the doc itself, but not be shown as part of the rendered page. This would help with maintenance since having the test code very close to the relevant docs text maximizes the likelihood that a developer that's fixing the broken test will also remember to update the corresponding docs text. Unfortunately it seems like markdown itself doesn't provide a great way to achieve this, but @nwt sensed he could cobble something together that leverages in-line HTML, since that way we could use comment blocks.
When we address this issue, I'll plan to modify the tests that were added as part of #2901 to take advantage of it, and assuming #2857 has not been addressed by then, I could also modify the two unprotected examples linked from that issue to use this approach (and perhaps decide this is "good enough" and close #2857).
A specific variation of this came up recently in. In brief, we had an unprotected example in the docs where named ZSON named type syntax had changed from port=(uint16) to port=uint16. In https://github.com/brimdata/zed/pull/5000#issuecomment-1904518089 made this suggestion on how we might address:
maybe for these kinds of tests we want to add a mdtest boomerang flag that 1) verifies it can serialize the value and 2) deserialize it to zson and 3) ensures the input/output values match?