Proposal: Semantic color aliases (both official and theme defined)

[joshgoebel]

TLDR: Many themes are impossible to represent in Base16 because we insist on binding multiple semantic meanings into singular colors: IE, "variables AND diff deleted MUST always be the same color". It's baked right into the spec. 🙁 You'll never find a Base16 theme where that's not true, but you'll find many themes in the larger world where that's not true.

This results in many themes ported to Base16 being broken or becoming "hollow shells" of what they intended to be. I think this is a bad outcome and we should be more flexible. Look at the Nord example at the top of #10... the Base16 Nord on the left honestly doesn't even look like Nord at all.

---

I think this deserves it's own topic though it's an offshoot of #10... I've quickly come to believe than many themes coming into Base16 from the "outside" can't possibly be properly represented in Base16 because of Base16's insistence on "semantic binding"... ie binding multiple semantic meanings into a single base16 color:

• "a variable MUST be the same color as Diff Deleted"
• "and also the same color as an XML tag"
• "and also the same color as..."

It's a bit larger problem than "too much red"... We don't have to look any further than the Nord examples given to see how fast this falls completely apart:

• https://github.com/chriskempson/base16/issues/162
• #10

Nord is very clear about the intention of it's palette (emphasis mine):

Nord4: (a very light grey) For dark ambiance designs, it is used for UI elements like the text editor caret. In the context of syntax highlighting it is used as text color for variables, constants, attributes and fields.

Nord11: (a slightly subdued red) Used for UI elements that are rendering error states like linter markers and the highlighting of Git diff deletions. In the context of syntax highlighting it is used to override the highlighting of syntax elements that are detected as errors

So immediately Nord is impossible to properly represent in Base 16 because it's a violation of the "semantic binding" of base08.

base08 - Variables, XML Tags, Markup Link Text, Markup Lists, Diff Deleted

---

Proposal

I propose we entirely unbind the semantics with aliasing... and that all the current semantic labels become aliases. We many also need aliases for mapping colors into terminal color space... (if the desire is to TRY and match default terminal colors at all).

scheme: "Nord"
# our own user defined aliases for convenience
nord0: "#2E3440"
nord1: "#3B4252"
nord2: "#434C5E"
nord3: "#4C566A"
# now the "official" semantic alias
diff_added: nord14
diff_deleted: nord11
variable: nord4
error: nord11
# ... and finally, required for compability...
base00: nord0
base01: nord2
base02: nord4
base03: nord3
# terminal colors often have fixed meanings
term09: nord11  # terminal color 9 is bright red, etc...

For backwards compatibility all the "official" alaises could map to their "semanticly bound" colors, as they do now... so if you didn't specify error or diff_deleted or variable they would all point to base08 (redish in many themes).

This would certainly require the aid/assistance/buy in of the template maintainers (to benefit from this improvement), but I think they'd welcome it... I'll speak for myself as the maintainer of Highlight.js template that I'd love to see this - and Nord is just one example of why.

Originally posted by @joshgoebel in https://github.com/base16-project/base16/issues/10#issuecomment-1170386189

[joshgoebel]

For builders I think the changes would be pretty simple:

• Simply allow/welcome the additional YAML keys
  • (we might consider name-spacing or nesting for the official ones)
• enforce maximum of 16 literal hex colors (we're still base16)
• Require base00 thru base15 (for compatibility)
• a color may be either a literal hex color or an alias to another alias or literal
• For sanity check (typos, etc) make sure every label is referenced at least once (ie nord44 would be an instant error)

[forrestli74]

I think the basic idea is pretty sound. I assume only base 0-15 is required. Since we want to make sure any template works for any scheme (The whole point of base16), we probably want to define a superset for all possible name aliases and all aliases should default to a base color. If what I'm saying is ok, aliases like nord4 should be ignored by templates.

Another lower level question is what syntax we want to use. An alternative syntax is to use yaml anchor

I think it's more expressive and easier for builders to implement. It even opens the door for having more than 16 colors.

[joshgoebel]

aliases like nord4 should be ignored by templates.

Yes that's the idea. They'd only exist as a convenience for scheme authors. I think in many other cases one might use the baseXX aliases to define the semantic aliases or else define named color aliases like red, green, etc.

[JamyGolden]

In https://github.com/base16-project/base16/issues/10 although I did title it The "red" colour prominence in base16 editor colourschemes for non-error code, I suggested that I suspect the problem was systemic; and the goal of the issue was to create discussion around that (most probably) systemic problem to find a path forward, so I'll close that ticket now since it achieved the goal.

@joshgoebel I like this proposal. I suggest also including diff_modified to go with diff_added/diff_deleted.

[joshgoebel]

I suggest also including diff_modified

Sure. I purposely wasn't being exhaustive... obviously an official list would need to be created if we are to go this direction... and it might involve MORE than what we currently list in our styles doc, but those we list there (including Diff Change) should certainly be included I'd imagine.

[joshgoebel]

I'd say:

• first get buy-in on the whole proposal at a high-level
• create an official list
• run that list by template maintainers (who will likely suggest additions)
• come up with an initial master list for v0.11 of the spec

We obviously will have to decide how general/specific we want to be since some scopes in some editors get crazy like: variable.inside_comment.constant.builtin (entirely made up, but I've seen scopes go deep)...

Looking at some popular editors and the most popular scopes/contexts to theme might help us insure we have a good working list. (i mean what we have now isn't terrible, but it's probably missing a few items at least)

[forrestli74]

By the way, base9 has something similar which could use as a guide for designing a list of possible aliases: https://github.com/base9-theme/base9-core/blob/main/src/semantic.json

[actionless]

Proposal I propose we entirely unbind the semantics with aliasing...

such en-complication of format would require to create a simple linting tool based on some of the builders, to run it on CI, which would check what all those references are resolvable and provide as a result at least minimal base16 scheme names

also i think it would be better to prefix named color references with some special characters to avoid confusing situation with old-style colors which are not prefixed with '#' and not to break backward-compatibility with old schemes by disabling colors without '#' prefix

[joshgoebel]

require to create a simple linting tool based on some of the builders

I'm not sure why it wouldn't just be the builder... CI:

• select a template that exercises all spec features
• builder builds all schemes (and reports errors)

We get it for free as soon as the builder properly supports these checks for building (which it must to do it's job).

also i think it would be better to prefix named color references with some special characters to avoid confusing situation with old-style colors which are not prefixed with '#'

An example here might help, I'm not sure I follow what you're suggesting fully.

[actionless]

An example here might help, I'm not sure I follow what you're suggesting fully.

ffffff is it "variable" name or color?

[joshgoebel]

ffffff is it "variable" name or color?

Well it could technically be either... if there was an ffffff variable, it's a variable, otherwise it's a color - that was how I imagine the scheme would work by default - any aliases that can't be resolved further are treated as colors. I'm not sure this presents a problem in the real world usage unless you can come up with a more compelling example.

I could of course invent ficticous examples, with my Bace scheme:

bace00: ...
bace01: ...

---

But we could probably also disallow variables with 6 digit hex names, and then we avoid this edge case entirely. Though I don't think a lot of people were likely to run into it in the first place.

Or force someone to add # to the hex codes where they'd be ambiguous...

# no ambiguity now
bace00: #bace01
bace01: #bace00

I'm kind of sad we made it optional personally. If that's what you were originally suggesting that's not so bad.

[actionless]

i was lazy exercising myself to come up with a 6-letter word containing only letters a-f, but great what you found better example yourself

i think it would be better to just prefix variables with smth like @ or other character which doesnt require escaping in yml, to avoid spec-ing and implementing in the builders additional resolving algorithm for deciding whatever some identifier is color or "variable"

Or force someone to add # to the hex codes

that's not worth of breaking backward-compatibility

[joshgoebel]

that's not worth of breaking backward-compatibility

It wouldn't since no aliases exist yet, there are no naming conflicts yet. The # would only be necessarily in the case of ambiguity - which is going to be RARE.

i think it would be better to just prefix variables

You mean on the value side, like this?

# reminder, # is optional 
bace00: "#bace01"
bace01: "#bace00"
bace02: "@bace00"
bace03: "@bace01"

Someone suggested the < YAML back-reference feature but honestly if we can stick with super-simple YAML I think that's preferable for the widest inter-op.

[actionless]

i meant what totally disabling colors without # would break backward-compatibility

adding resolution algorithm to determine whatever some unprefixed identifier is color or "variable" - won't break backward-compatibility but will en-complicate both spec and builder logic

[joshgoebel]

totally disabling colors without # would break backward-compatibility

I was never suggesting that.

adding resolution algorithm ... builder logic

It's quite trivial I assure you... few lines. But even if it was slightly harder - I think the main focus should be on usability for theme and template authors, not the backend implementation details.

will en-complicate both spec and

And could be explained in one or two sentences tops (and perhaps an example or two). It might be it's "too magical"... but I'll let others weigh in and decide... Using a prefix is "ok"... just I think not necessary in 99.9999999% of cases.

[actionless]

It's quite trivial I assure you... few lines.

it's ok when updating existing builder, but every of those "two lines" here and there would be a PITA if implementing builder from scratch. especially as it could be avoided without loosing the same functionality

[actionless]

i think it's also would be useful to do a research which of currently available yaml parsers do support < - if it's wide enough across different languages - it would be better decision than implementing that at builder at all

[joshgoebel]

but every of those "two lines" here and there would be a PITA if implementing builder from scratch

I've written such things from scratch before, it's simply not that complex. Truly. 💻

[joshgoebel]

@belak Kind of waiting for your high-level thoughts before I whip up a quick proof of concept and fix Nord to see how all this might work in practice. :)

[actionless]

but why to do it at all if same problem could be solved without doing it?

your motivation also totally not correlating with your previous decision to remove template-index from spec because "it was too complicated"

but writing same two lines for allowing ambiguous variable identifiers instead of specific ones is not, is "easy, not complex"

[joshgoebel]

but why to do it at all if same problem could be solved without doing it?

It has to be done one way or the other - which requires some amount of complexity, I just think my way might be a bit more elegant... sometimes a bit of magic is nice.

your previous decision to remove template-index

Not my decision... and entirely different. Each thing must be judged on it's own merits. I think aliases are a feature that would benefit MOST syntax template authors where-as I think "build the world" benefits a very, very small number of users.

allowing ambiguous variable identifiers i

I don't allow them, they (in the very rare chance they are encounted in the wild) they throw a clear error. :)

[actionless]

you yourself came up with idea about "building the world" - and keep repeating that term - why you keep attributing your fantasies to my name? it seems like you purposely ridiculing the problem and pretend like none of real motivation behind this exists and keep repeating this weird thing you came up yourself about building the word, because you simply can't tell "yes, we did mistake and accidentially removed the thing which we not fully understood in the past"

[joshgoebel]

you yourself came up with idea about "building the world" - and keep repeating that term - why you keep attributing your fantasies to my name?

I think it's actually from BSD... https://people.freebsd.org/~rodrigc/doc/handbook/makeworld.html - I'm not attributing it to you, it's just an easy expression that sticks in my head and I think makes sense for "build all schemes/all templates"...

it seems like you purposely ridiculing the problem and pretend like none of real motivation behind this exists

I do believe that building the world is a niche case needed by a very small number of users... that is not ridicule... and you haven't really said anything yet to suggest otherwise (that I recall) or to show that a lot of users were using that feature.

[actionless]

could you show where i motivating having theme index by need of build all templates at once? my builder CAN'T even do that - you somehow came to that point solely from your fantasy and just keep arguing against it. plus coming with other ridiculous excuses about simplifying, about some imaginary user complains about "unease" and lots of other empty words - all to cover up your past mistake and pretend like it was reasonable decision

[joshgoebel]

I implemented a proof of concept as a simple direct alias system (as originally proposed) just to see the effort required and it's a very clean ~30 line patch... all the complexity lands inside a resolveAliases function which chases down aliases to their raw values.

All values can potentially be aliases other than the reserved keys of scheme and author...

This doesn't yet include the "no more than 16 colors" cross-checking (since now you can have unlimited aliases) but it could be added fairly easily if this moves forward.

[actionless]

~30 line patch

assert.almostEqual(2, 30)

:3

[joshgoebel]

You're the one whose been quite vocal here about the removal of the theme index... if you're only building 2-3 themes (as a maintainer) you don't need a global index. The global index's main utility (as I imagine it) was for "building world".

all to cover up your past mistake and pretend like it was reasonable decision

Again, not my decision and mistake or not, that is an opinion. Please if you want to discuss this further go to #3 rather than hijacking this issue.

[actionless]

template index is needed as template index

the global index's main utility (as I imagine it) was for "building world".

so instead of reading my motivation in my messages in related topic and arguing against them you decided to argue against what, as you can imagine, would be my motivation? this is just wrong at all levels

Again, not my decision and mistake or not, that is an opinion. Please if you want to discuss this further go to https://github.com/base16-project/base16/issues/3 rather than hijacking this issue.

i mentioned this in this topic only because it's hypocrytical motivating removing template index because of "simplification" but adding much bigger chungus of code almost for nothing

[joshgoebel]

assert.almostEqual(2, 30)

😄 True (in absolute terms) - and perhaps I shouldn't have said "two lines of code" or whatever I said... BUT... I'm always thinking in terms of top-down design, abstraction, and well hidden complexity... and from that perspective it is literally a 1 line patch that really touches a single location in the codebase (other than a few constants) and adds only a single simple and easy to understand function.

    const hexDefinitions = Object.keys(data).reduce((accumulator, key) => {
+      resolveAliases(key, data);

You can't get a lot better than that for the MAJOR new functionality this provides.

[davidscotson]

A possibly relevant area to compare with is the recently standardized forced-colors spec in browsers.

It tries to to standardize some basic semantic names that are under the control of the user (and/or the user's OS, e.g. dark mode may be activated at sundown). So you as a user can specify white on black, black on white, high-contrast, low-contrast (many of the examples talk about high-contrast but it's important to give the user control over this) or really anything to suit your own needs and any site that tries to follow the spec will do it's best to honor the users wishes.

https://drafts.csswg.org/css-color/#valdef-system-color-canvas

It hits lots of the same issues, if you don't know which colors are used (since those are under user control), how can you know the output will be readable. They suggest mostly setting the colors in foreground/background pairs, but also expect certain colors to be readable against certain other colors (e.g. Link on Canvas) rather than always setting a pair of colors.

An accessabile base16 theme that picks these up as well for consistency would be neat too, but referring to it mostly as a source of similar ideas.