Rethink sublanguages

[isagalaev]

There are many subtly different ways of embedding one syntax into another:

• The host language clearly defines boundaries, between which we can use a completely independent highlighting process. It's the case with <script>..</script> in HTML.
• The host language defines boundaries, but the closing boundary can only happen at the top level of the sublanguage. This the case of PHP in HTML, where ?> inside a PHP comment wouldn't end the sublanguage.
• The sublanguage ends when it parses out a single top-level mode. This is the case of JSX.

Current implementation of subLanguage only works for the first case. And the skip attribute only fixes a few common cases.

Current idea is to implement sublanguages as sub-parsers embedded in the host parser tree.

[joshgoebel]

Current idea is to implement sublanguages as sub-parsers embedded in the host parser tree.

This is on my radar for the future... obviously you have some thoughts on this, but I'm not getting much of a feel for your thoughts with just that one-liner.

And chance when you have some time you might be able to expand that one liner to 2-3 short paragraphs kind of laying out at least your big-picture thoughts (if you can still remember them).

Right now to fix JSX I'm thinking it really needs to not be a sublanguage at all and just an actual part of the JS language. Because you can do things like each line can either be HTML or a JS comment... so you can't just simply switch to HTML mode for a block... because:

var x = (<div>
// this is Javascript code, a comment that I can use to annotate the HTML with JS comments, fun fun
</div>)

So now html would need to know about JS comments... I wonder if your ideas solve any of this (it's ok if they don't). This really isn't one language embedded into another exactly it's like a big mixing bowl where they just randomly throw JS and HTML together in strange ways. :)