Support inheritance and/or imports

[mitranim]

Currently SBNF doesn't seem to support inheritance or imports between syntaxes (unlike .sublime-syntax with extends). For some syntaxes, this would be quite a useful feature.

Currently SBNF requires all definitions to be present at compile time, so it's unclear to me whether it could reasonably support extending .sublime-syntax syntaxes. But importing another SBNF syntax seems quite doable.

The use case exists; I happen to have a group of syntaxes where 2 inherit from 1. Also, depending on how import paths are resolved, we might consider the case where a new syntax would import common definitions from a "library" syntax. For example, one might want to reuse C-style comments, numbers, and strings, which are practically copy-pasted between languages anyway.

Open questions:

• File-relative paths?
• CWD-relative paths?
• Importing from default packages, should they happen to ship with .sbnf sources?

[predragnikolic]

I would prefere composition over inheritance. To pick only the parts that you need vs. getting everything even if you don't need them.

[mitranim]

@predragnikolic Unless I'm missing something, there aren't many different ways to do this. In .sublime-syntax, extending a syntax puts its definitions into your scope, but yours take priority. Because the syntax uses only what's explicitly specified in prototype and main, if you have those of your own, your syntax is unaffected unless you choose to refer to the definitions from the syntax you extended, which you're free to do.

If .sbnf ends up implementing its own imports, one meaningful choice would be whether to splurge definitions into your scope unqualified, like in .sublime-syntax, or use a namespace prefix which would make it easy to tell where a definition came from. Qualified imports would also prevent imported definitions from automatically giving you prototype or main if you didn't have one.

[BenjaminSchaaf]

The trouble I see with using .sublime-syntax's extends is that it's built using contexts which don't have a direct translation to rules. Because of that you end up having to think in terms of the contexts SBNF generates rather than using a BNF.

On the other hand statically importing rules from another syntax may lead to performance issues. It's my understanding that extends doesn't create a full copy of the extended syntax, is that correct @wbond? This would be something unachievable when importing rules from another syntax.

[wbond]

The extends functionality copies all contexts from the base syntax and then merges contexts from the current syntax over the base syntax, keyed on the context name.

Implementing such a thing in sbnf I imagine would be 100% up to sbnf since it is compiling down to Sublime-syntax, and the context names are auto generated.

[BenjaminSchaaf]

@wbond thanks for confirming that. So it looks like we don't lose anything from doing "static linking" style imports. This would be my preferred approach to this as I'd argue it's the most logical programming-style solution. Just throwing a syntax out there, I'm thinking of something like this:

import './C.sbnf' as c

prototype = c@prototype ;

...

[wbond]

It all depends on how you handle overrides and if includes are transitive.

Maintaining derived syntaxes via include statements was unmaintainable in how sublime-syntax handled that. The key thing is being able to modify included transitive contexts, such as push targets.

[mitranim]

Relative paths with explicit extensions get my 👍. Would prefer namespacing via c.prototype or c:prototype, banning the corresponding character from rule names. Not entirely sure how it would be possible to import from built-in syntaxes. In the theoretic case of using SBNF in Packages, syntaxes in Packages could use ../OtherLang to refer to each other, but authors of external syntaxes would have to use file paths specific to their local setup, which seems doable but fragile.

Good question about override handling. sublime-syntax allows child variables to override parent variables, affecting the parent's contexts. This allows interesting things, like children that use different delimiters, identifier rules, keywords, and so on. But it also creates the danger of unintented collisions, arguably even greater if we allow rule interpolation. Namespacing would imply that the child's rules don't override the parent's rules, which is safer but less powerful. Explicitly listing the overrides would be safe and powerful, but potentially verbose, and complicates SBNF a bit more. I'm probably missing other important scenarios, and not ready to offer an opinion right now, but this is something we should consider.

Not sure what was meant by include statements and being able to modify push targets... are there such features in sublime-syntax?

[wbond]

Inheritance allows overriding context, along with appending and prepending rules.

[mitranim]

Looking at the SBNF Prolog example, it seems we want to be able to say "everything from syntax X, overriding rule Y", or more generally, "everything from syntaxes A, B, C, overriding rules D, E, F". The simplest way that comes to my mind is through unqualified imports, mimicking .sublime-syntax.

name: Prolog-SWI

import './Prolog.sbnf' as *

multi-line-comment{comment.block} = '...';

There might be better approaches. Just making a point that some syntaxes will want this feature, which is already possible with .sublime-syntax.

If we want to avoid unqualified imports, one idea that comes to my mind is allowing locally-defined rules to be namespace-qualified. This "modification" would only affect the current file.

name: Prolog-SWI

import './Prolog.sbnf' as super

prototype = super.prototype;

main = super.main;

super.multi-line-comment{comment.block} = '...';

[BenjaminSchaaf]

I'd much prefer the second suggestion as it's much more explicit and works with multiple imports including a "diamond inheritance" situation. My suggestion would be to make imports part of the meta-programming in SBNF. Using some example syntax:

prolog := import[`./Prolog.sbnf`]

prototype = prolog.prototype;
main = prolog.main;
prolog.multi-line-comment = `...`;

# A meta-programming example
include[GRAMMAR] = '```' ( ~GRAMMAR.main )? ~'```' ;
js = include[import[`./JavaScript`]] ;

# Could also make the import "implicit"
# ie. not having a separate value type for grammars and instead reuse strings
prolog := `./Prolog.sbnf`
main = `./Prolog.sbnf`.main ;
prototype = prolog.prototype ;

[mitranim]

Will mentioned that .sublime-syntax supports optionally prepending or appending to parent context when overriding it. The SBNF suggestions above don't explicitly address this. I suspect that | might be sufficient, but I'm not entirely sure. The following examples require SBNF to treat super.some_context as the original parent context in RHS. This might mean that every occurrence of super.X for any given X must be the original, non-overridden version, not just in RHS of the override, but in the rest of this file, except for the LHS where it's assigned?

super := import[`./some_parent_syntax.sbnf`]

# Replacing.
super.some_context = `some_child_alternative` ;

# Prepending.
super.some_context = `some_child_alternative` | super.some_context ;

# Appending.
super.some_context = super.some_context | `some_child_alternative` ;