microsoft
Behavior of $base and $self in injection grammars
As far as I can tell, $base and $self currently do not work as expected in external injection grammars. Both seem to resolve to the injection grammar itself.
Instead, here's how I think they should work:
$base in an injection grammar should always be resolved to root grammar (it should have the same behavior as using $base in a normal grammar file that is included from another grammar)
The expected behavior of $self is less clear. I think it would make most sense for $self to resolve to the grammar being targeted by the injection, although you could argue that it could be resolved to the injection grammar itself
I'm not sure how this should work, we should just align with whatever TextMate does.
I'm following up because of the same use case as @mjbvz. It appears $self correctly resolves to the injection grammar itself. The metaphor is self in Ruby. $base doesn't seem to do anything at all. It should resolve to the grammar being targeted by the injection. The metaphor is super in Ruby. I can't imagine a use case for targeting the root grammar. In any case the only documentation I'm aware of is:
https://www.apeth.com/nonblog/stories/textmatebundle.html#:~:text=the%20value%20of%20an%20include%20rule%20can%20also,embedded%2E
Relevant code: https://github.com/microsoft/vscode-textmate/blob/a53ae3c12bef5499a7dc548b3058ee64bfcbd222/src/grammar.ts#L122-L125
@texastoland The usecase for targeting root grammars was for superset languages E.g. Objective C++ includes C++ which includes C. So when C uses $base it re-activates the root (Objective C++) pattern set. The C family and Latex (Tex, Latex, Katex) are the only two I know of that can even attempt to nest that way. But just because thats what it was designed for doesn't mean its good, and one of the first things we did to C++ was rip out all uses of $base because it is terrible. I don't think $base should ever be used.
While $base has special functionality, $self is purely for convenience. Arguably $self should never be used because it would be more clear to actually state the set of patterns that was being reused instead of just saying "$self" and making other devs look up or guess the behavior.
I agree something like $super would be more useful. Not terribly useful because it would need to be a grammar/injection that was unaware of the parent, yet still wanted to include the parent grammar inside itself. If the grammar knows what language it is being injected into, it can simply specify that language by name instead of trying to use something like $super or $base.
@jeff-hykin Thanks for your explanation of the reasoning! In any case $base has no effect in my current injection grammar.
I follow your logic about explicit scope names. I'd argue very little about TextMate grammars is documented or even consistent. I've been writing regexes for 15 years yet struggling for more than a week to fix a bug I expected to spend a half day on.
There are more bugs I could track here but I'd really prefer some update on https://github.com/microsoft/vscode/issues/216#issuecomment-913872371 (I know you're even more invested in that topic).
@texastoland I 100% agree.
Matter123 and I spent years on a library (which I finally published just last week) to make it way less painful. The documentation is still sparse, especially for things like $base and $self, but Textmate is easily one of the worst least-documented tools I've ever used. 90% of what I've learned has been through reading other people's textmate grammars and trial-and-error.
@texastoland
As an example, I went ahead and created a demo of your f# templates syntax using the library. Hopefully that'll be useful
- main/main.rb is the important file
- autogenerated/template.fsharp.injection.tmLanguage.json is the output you could compare to your tmLanguage
Here's a link to that demo repo: https://github.com/jeff-hykin/fsharp-injections-demo (you can ignore the readme and other stuff, most of it is leftovers from a template/skeleton I use)
@jeff-hykin Thanks that was fast! That's the syntax I'm currently rewriting. As I mentioned there are a few tricks to do it much more succinctly than either JSON or Ruby with plain YAML. Understandablly you're working at a bigger scale with C and Bash. But I've been stopped by weird behavior (or just lack of documentation) and a few annoying bugs in this repo. I only reported this 1 because $base literally does nothing in my grammar.
@texastoland I'll continue the topic over here since this is off topic from the original issue
https://github.com/jeff-hykin/fsharp-injections-demo/issues/1