zed

zed copied to clipboard

Check for existing issues

• [X] Completed

Describe the feature

Semantic Tokens allow for extra helpful highlighting that the language server might have better context over.

For example, in Go many things are well highlighted via Semantic Tokens which are all explained here.

My personal favorite is being able to see formatting directives easily in strings:

which is not currently supported in zed:

Thanks!

If applicable, add mockups / screenshots to help present your vision of the feature

No response

These are different requests, but this one feels like its in a similar category:

• https://github.com/zed-industries/zed/issues/5345

I've been trying to port my custom One Dark theme to Zed, everything went pretty smoothly, except for more complex code. Unfortunately some sections end up being almost unreadable compared to VSCode, where semantic tokens are available. VSCode on the top, and Zed on the bottom:

note how customizations are missing for mutable, macros, and enum members.

Here's another example of mutable vs non mutable semantic token highlighting:

more useful than one might initially think since it applies to variables / parameters and method calls.

It would also be neat if semantic tokens in Zed supported rules the same way VSCode does using globs/stars ("*.mutable": "#aabbcc") but that's not as important.

bump

This is the only blocker for me

This really helps to navigate the code. We are waiting impatiently.

"editor.tokenColorCustomizations": {
"comments": "#9D9D9D",

"textMateRules": [
	{
		"name": "Rust function definition name",
		"scope": [
			"meta.function.definition.rust entity.name.function.rust",
		],
		"settings": { "foreground": "#FF90F0" }
	},
	{
		"name": "Rust function definition params",
		"scope": [

			"meta.function.definition.rust variable.other.rust",
			"meta.function.definition.rust keyword.operator.key-value.rust",
			"meta.function.definition.rust punctuation.brackets.round.rust",
		],
		"settings": { "foreground": "#F06BDD" }
	},
],
},

bump

Is anybody working on this? It is genuinely the only thing stopping me from daily driving zed.

Is anybody working on this? It is genuinely the only thing stopping me from daily driving zed.

First, I understand that people want their top bug squashed or feature implemented. But when this kind of level of intensity shows up in bug reports, it can be hard for many of us to take at face value. Is this really a deal-breaker issue? How much of it is exaggeration?

So, for those that feel strongly about this issue, please help me understand:

• why is this bug a high priority?
• what are the sub-components of the problem? (is this issue blocked by something else?)

(Caveat: I'm not with the Zed organization.)

Gladly.

why is this bug a high priority?

First, I wouldn't call it a bug, but a missing and much-needed feature. Regardless, semantic tokens are an important feature for a few reasons:

• Code is much, much easier to understand when things are highlighted with the correct color regardless of context. It's much easier to track where different types and values are referenced. Consider a Rust module where an enum variant called A is used. Reading the code in Zed, one would assume that A is actually a constant, and not an enum variant, since that is the highlighting Zed currently gives it.
• Semantic token customization allows people to shape the way code is presented to them, which is important for accessibility as well as for everyday use. In VSCode, I have unsafe function calls set to underline, and mutable calls set to bold. That helps me understand what the code is doing without checking the signature of every function.
• Semantic tokens are supported in nearly every major editor with LSP support. (Even Lapce has semantic token support, iinm.) Why not Zed?

what are the sub-components of the problem? (is this issue blocked by something else?)

I am not familiar with Zed's internals, but the language server provides semantic tokens (if supported). It would be a matter of replacing tree-sitter highlighting with semantic tokens whenever they are available. I don't think anything is blocking this.

Yes, I don't think anything is blocking the implementation here per se, it's just a significant piece of work. Right now we have one entity called HighlightMap that ties together results of tree-sitter highlight query and colors from theme. I imagine that in order to implement semantic highlighting we would have to allow HighlightMap to take in multiple sources of highlights (and not just tree-sitter matches); I am pretty positive that we could use such facility to implement rainbow brackets as well, so there's value in generalization beyond semantic tokens. Just to chime in though:

Semantic token customization allows people to shape the way code is presented to them, which is important for accessibility as well as for everyday use. In VSCode, I have unsafe function calls set to underline, and mutable calls set to bold. That helps me understand what the code is doing without checking the signature of every function.

While I agree that this would be great to have, I'd leave it out of the initial implementation, as theme customization makes things quite a bit trickier and semantic tokens have value even without customization. Once again, this is cool, but it's not essential for the first version.

I would really love to see this happen and I might push it myself at some point, but this really is not a trivial change to make; I've just described issues with altering HighlightMap and I've just skipped over integrating the protocol wholesale; that part will be quite cumbersome too. :)

Adding just a bit more to the conversation, a feature I would like implemented is the underlines for mutable bindings (variables, methods, etc) just like this comment above here. They want different colors, I'd like underlines just like VSCode, more or less the same thing.