Razor semantic tokens range handler can return incomplete tokens outside of the requested range

[gundermanc]

Describe the bug: I recently discovered an issue in Visual Studio for Windows 'Razor LSP' scenario where large ranges of semantic tokens can 'disappear' as the user scrolls when polling is disabled.

This issue seems to arise from different interpretations of a language server protocol spec ambiguity.

Version used: e.g. VS2022 17.3 Preview with polling disabled.

Expected behavior: Razor server should only return semantic tokens within the requested range OR when 'extending' the range should include all tokens in between the first and last returned token.

This was clarified by Dirk in the LSP protocol: https://github.com/microsoft/language-server-protocol/issues/1447#issuecomment-1106171637.

Actual behavior: Razor server returns '@{' from outside the requested range and all tokens that intersect the requested range. Tokens between the '@{' and the start of the requested range are omitted. Prior to my fix in VS, this lead to blocks of color disappearing when scrolling.

I worked around this from the VS side but recommend also fixing the issue in the Razor server: https://devdiv.visualstudio.com/DevDiv/_git/VSLanguageServerClient/pullrequest/394818.