monaco-editor icon indicating copy to clipboard operation
monaco-editor copied to clipboard

[Bug] maxTokenizationLineLength message does not make sense for JSON tokenizer

Open CommonGuy opened this issue 2 years ago • 2 comments

Reproducible in vscode.dev or in VS Code Desktop?

  • [X] Not reproducible in vscode.dev or VS Code Desktop

Reproducible in the monaco editor playground?

Monaco Editor Playground Link

https://microsoft.github.io/monaco-editor/playground.html?source=v0.44.0#XQAAAAJvAQAAAAAAAABBqQkHQ5NjdMjwa-jY7SIQ9S7DNlzs5W-mwj0fe1ZCDRFc9ws9XQE0SJE1jc2VKxhaLFIw9vEWSxW3yscwzeGe2_1uhs0-3YGTBOsVJ4S4FwfCDQb5-2uUHXTVAefbVSd9IzaZ_vv9VR9KugsKiuHAg9KQ6ZwEgg65_cAgZ7FYyNxeWYeKJhXGybBcv8k5zZE6VqryJsIdUnidc5Y6K1guqADFHgmK9d9buXYFQ8pSvlFJXvdJ4L6uJlIPp7nFL6ntQqL6fMVp4kC38OvU-qP_A-LyF-cEV2uE2yuNQXBBKfqHh9XwBQlHDCGODul-9BOtxsvhIDj4a_YbO2L-A0YBl5yYHm88OQ2GO2Xu40kjYatxjerA4W1iXfRosAkKyLVAK9CelZJVnv73Tuoy

Monaco Editor Playground Code

const value = /* set from `myEditor.getModel()`: */ `{ "test": "this is my custom json" }`;

// Hover on each property to see its docs!
const myEditor = monaco.editor.create(document.getElementById("container"), {
	value,
	language: "json",
	automaticLayout: true,
	maxTokenizationLineLength: 4,
});

Reproduction Steps

No response

Actual (Problematic) Behavior

The line gets tokenized and highlighted. When hovering over the line, it actually says that tokenization was skipped.

Expected Behavior

Tokenization should be skipped. Or at least the info on hovering should be consistent with the tokenization.

Additional Context

This already has been reported in https://github.com/microsoft/monaco-editor/issues/3025.

However, I couldn't find a monaco version that works correctly.

CommonGuy avatar Nov 19 '23 08:11 CommonGuy

This is because json uses its own tokenizer. It works for typescript

hediet avatar Dec 14 '23 10:12 hediet

I get the technical reason behind this and I'm fine that this setting doesn't work for JSON.

However, the information displayed on hover is incorrect in this case, as tokenization works

CommonGuy avatar Dec 14 '23 10:12 CommonGuy