Use Case: Expose Comments Channel In Zig For Tooling Interface

[exxjob]

Reopening since... https://github.com/ziglang/zig/issues/14656 was closed, citing https://github.com/ziglang/zig/issues/1099. The two proposals are mutually exclusive with different use cases. This issue strongly resembles https://github.com/ziglang/zig/issues/14656 which should provide a point of reference.

https://github.com/ziglang/zig/blob/2252dcc508c53dc302ec1f919c4792a7f00ce125/lib/std/zig/Parse.zig#L182-L183

Autodoc & zig fmt are first party tools using Zig comment forms. Regular comments should be handled under a flag, for the user to interface with. This will facilitate congruous and thetical Zig extensibility.

1. Issue Tracker Integration: Raise TODO comments to an issue tracker, remove in correspondence to their status. Browse and highlight issues in source assigned to a team member, or by other filters.
2. License Management: lookup and process byzantine rights expressions by schematic or adaptably. https://github.com/ziglang/zig/issues/14342#issuecomment-1751552275
3. Inverted Ziglings: solutions are written in whole, with a CLA generating problems from custom comment tokens. This is far more submission friendly, and eminently allows for test evaluation as opposed to string equivocation.
4. Shallowly embedded verification systems, potentially promoting away safety checks. Deeply embedded verification systems, especially by third party, isn't necessarily preferable or practical.
5. Flags for behavioral subtyping (instead of volatile assertion boilerplate) in automated code review for team and cross-team development settings.

It's a streamlined solution that further discourages platforming less sustainable and less intuitive configurations. I had in mind a similar idea for annotations like in https://github.com/ziglang/zig/issues/14656, and using that for doc comments as well. I'm flaky on how this should work but I think it's well worth the discussion. It's "just enough freedom" so there's still one way to do things, but you can do more things now.

[exxjob]

This doesn't call for an extensible parser per se, ultimately comment form tooling to leverage the AST declaratively should be viable with common components (i.e. no forks or unnecessary propagations inc. passes). I'd consider Aro as well.

[ianprime0509]

For what it's worth, it's already pretty straightforward to detect comments in an Ast: https://github.com/ziglang/zig/blob/2252dcc508c53dc302ec1f919c4792a7f00ce125/lib/std/zig/render.zig#L3094-L3157 The space between two adjacent tokens consists of either whitespace or comments, and Zig only has one type of comment (line comment with //), so it's enough to look between adjacent tokens, scan for //, and follow each comment until the end of the line.

Additionally, doc comments are already handled by the tokenizer and contribute to parsing (it's illegal to put doc comments in unexpected locations), so any tooling needing to use doc comments can also follow the same sort of logic used in render.zig to process them.

[exxjob]

Modularizing this is an emender. This would be a significant redundancy in colossal codebases.

[exxjob]

Closing in favor of discussion in https://github.com/ziglang/zig/issues/14656 I was originally going to suggest exactly that, then got a bit lost, with the rationale I made here, though maybe still poorly explained