Improving doctag support and theme design in general

The problem

GitHub's doctag highlighting is so nice and making me jealous. I suppose we're quite close actually, but in thinking about this it brought up a lot of questions about how we classify doctors.

GitHub

Us

So lets tear apart a single doctag:

 * @param {string | HighlightOptions} optionsOrCode - the code to highlight

And if we look closely we find:

• * comment decoration
• The whole line starting with @param is technically documentation. (block scope)
• @param the "tag" itself -{} punctuation around the type
• string | HighlightOptions the type itself
• optionsOrCode variable name
• - punctuation
• the code to highlight description

Currently we break that down line this:

• parent doctag
• type inside that
• variable inside that
• no way to target the description

This is "ok" until say you want to apply a background to perhaps just the tag portion... it seems we could perhaps be more granular here...

Also there is an issue with theme "leaking".

I've been thinking a lot about new theme designers and what type of experience they should have. Do we make it easy or hard to design themes, etc... how "fiddly" are we, etc... Right now if you don't even know what a doctag is and you design a theme without support then you probably get your doctag types and variables colored entirely unexpectedly with whatever color you used in the outside source. Is this desired behavior?

Or if you do design with support... you add a doctag rule... did you remember the fact that variable and type could be nested within? See #3167 for another example of this unintentional "leaking" of CSS.

I wonder if all of these shouldn't be sub scopes instead to prevent this leakage - or perhaps the leakage is sometimes desirable? To me (with a theme designer hat on) I would NOT expect the type inside a comment doctag to automatically be highlighted exactly the same as a type inside my code proper.

Some possible scopes:

• block.doctag (TextMate would call this meta.doctag or something)
• doctag.tag
• doctag.type
• doctag.variable
• doctag.description
• The {} in type would either not be included or labeled as punctuation.

I make the first a block/meta scope (vs just doctag because (like TM) I think in general we'd like to discourage people highlighting entire scopes, but they can still be useful for targeted rules.

For reference TextMate does this with it's top-level scopes (entity.name, storage.type, variable), so perhaps we're not doing the wrong thing after-all. Though it still feels like the behavior where doctag wraps everything should perhaps be changed.

• comment.block.documentation (the larger block scope)
• @param - storage.type.class.jsdoc
• string - entity.name.type.instance.jsdoc
• codeToHighlight - variable.other.jsdoc
• the description is given no specific scope

There is actually no scope for the "whole doctag line"... comment.block.documentation is applied to the entire /** comment, irregardless of doctors...

Other ideas

Perhaps this is solved with better docs, or some sort of theme design tool with examples? IE, a minimal version of "demo" that has to look "passable" for multiple languages, etc...

CC @highlightjs/theme-maintainers @highlightjs/core

Perhaps what is needed is a canonical "template theme" with comments explaining all the comment scopes and someone goes thru and kind of "fills it out" as they build their theme. Then regardless of naming/scoping it's clear what needs to happen and any "caveats" can be explained?

Candidly, a template theme punchlist would be helpful.

I can't speak for other theme maintainers, but I'm pulled in a lot of different directions lately and don't have a lot of time to follow the nuance of the theme's construction.