Discuss: Improving testing / integration / usage for 3rd party grammars

[joshgoebel]

This is a continuation of #2328 (linking for reference). Where-as that discussion was more about the file layout/structure of a 3rd party grammar and related conventions - this can be a discussion of how we continue to improve the tooling around working with 3rd party grammars - both as a user or developer.

[joshgoebel]

If everyone is happy as things are then I may just put an auto-close clock on this one.

[joshgoebel]

My original thought was to make it easier for grammar maintainers to "build" outside of our project - ie, not need their grammar checked out into our extras folder. So perhaps you'd just set an ENV variable (or something to point to us) or perhaps we'd provide some CLI tooling or something... but overall there seems to be little demand for this and I'd say out of every other thing we could possibly spend time on there are much more important things.

So I'm closing this issue for now.

[joshgoebel]

Continued from https://github.com/highlightjs/highlight.js/issues/3008#issuecomment-789007258:

The disagreement is about whether the repository is the means of distribution, along with what I consider the normal distribution channels like GitHub Releases or packages on the npm registry.

@frangio I'm aware. Some of our end users probably have no idea what a "npm registry" is... the desire is to make the binary as EASY to download (and use) as possible. Someone could potentially:

• Add dist (as we recommend) and let us build it (for guaranteed compatibility and consistency)
• Use GitHub releases
• Publish the CDN distributable as part of the NPM package
• Which would also auto-host it via CDN, win win.

The bigger push here is that all grammars provide not ONLY a Node.js module, but also a CDN distributable [which includes the registration hook] as well and consistent instructions for using it easily (which MANY already do). The CDN distributable should only require a single <script> tag to "install". This serves the purposes of our many users doing simple website/CDN installs. (over 100 million per month)

These users have no build systems. They perhaps don't know what npm is, nor would they care.

We of course encourage all 3rd party grammars to publish to npm so that anyone who uses a more complex build pipeline can integrate however they choose.

---

dist is our choice because that's what our built-in build system does (making it super easy for maintainers) and we want to see the install experience for all 3rd party grammars be consistent (a one liner script tag in the simplest case). It's also already a convention many grammars are already following.

[frangio]

I understand your points and if the suggestions work for the target audience that's great. I'm not criticizing at all. It was a misunderstanding on my part to think the proposed setup was a requirement where it's just a suggestion.

[joshgoebel]

It was a misunderstanding on my part to think the proposed setup was a requirement where it's just a suggestion.

So far (since making providing of a CDN distributable itself a requirement) no one new has opted to do anything other than use our own build scripts and place it in dist. I would suppose because that's what we've made super easy - and because maintaining a custom build system, GitHub Action, etc. is not everyone's cup of tea.

The first time anyone shows us something else in the concrete rather than abstract I'm sure we'll take a close look and see if it seems easy enough to use compared to everything else... though if more than a few confused users ever show up and ask "why do all the 3rd party grammars have dist folders but these 2" I'm not sure how sympathetic I'd be to the outliers.

So I'd say it's a very strong recommendation at this point. :-)