Add semantic token modifiers to callable variables.

[heejaechang]

this is related to https://github.com/microsoft/pylance-release/issues/5785

basically, user could have something like this

func: Callable[...] = ...

it is a variable, but it can be used like function

also, user could have something like this

class Foo():
     def __call__(self): ...

func = Foo()
func()

here func is a variable again but it can be used like function

we could add semantic token modifier to these variables so users who want to distinguish them can use editor.semanticTokenColorCustomizations to do so.

[erictraut]

If my understanding is correct, today's semantic token provider generates tokens based on semantic categories only, not on type information. What you're proposing would require the token provider to perform full type analysis of every identifier at every point in the code flow since the type of an identifier can change based on narrowing, assignments, etc. It also means that the color of an identifier might now vary from one place in the code flow to the next. And it would require you to decide what to do with types that involve unions between, for example, a callable type and a non-callable type.

[heejaechang]

how we will implement or how much perf cost it will have, we need to dig in but before that, we probably want to see how many people will upvote this.

[heejaechang]

I briefly looked, and we get decls for all identifiers (basically names), but we only get the type of the decl, not at each specific position, which will cause us to do the narrowing and other things Eric mentioned above.

So, if we do it based on the declared type, there won't be any extra cost. However, if we want it to be precise and correct at each position where identifiers are used, there will be extra performance cost.

That said, since the semantic token provider and checker (the one that produces diagnostics) will share the same cache, and since the checker will call getType on every identifier anyway, in reality, the overall cost should be the same. It would be just a matter of who does it first and gets the blame.

For the same variable/parameter being colored differently if we decide to do it per each position, since we just mark the token with extra data and people who want it need to opt in, it might be okay?

But I am not here to draw conclusions but to provide more context so people can discuss with more information in hand.

[maflAT]

This is related to https://github.com/microsoft/pylance-release/issues/5612 Currently the same symbol can be treated either as a function or a variable depending on how it is first declared.