Uncommenting code that was commented in VS Code leaves unexpected space at start of line

[debonte]

Describe the bug Uncommenting (Ctrl+K, Ctrl+U) in VS only removes the #, but commenting (Ctrl+K, Ctrl+C) in VS Code adds # plus a space. If you comment in VS Code and then uncomment in VS you're left with an unexpected space at the start of the line.

This may not be important if we don't think users are round-tripping between PTVS and VS Code. I ran into this because I copy-pasted a block of code from VS Code to VS.

Steps to Reproduce

1. Open/create .py file in VS Code
2. Comment out some lines using Ctrl+K, Ctrl+C
3. Save the file
4. Open the file in Visual Studio
5. Uncomment the lines using Ctrl+K, Ctrl+U

Expected behavior Affected lines are the same as they were before step 2.

Actual behavior Affected lines have an unexpected extra space character at the beginning of the line.

Additional context and screenshots

Configuration information (If you are providing a diagnostics file (see below), skip this section) VS Version: main 32111.27 (17.2.0 Preview 1.0) PTVS version: 17.0.21344.1 Python version: 3.10

[debonte]

Moved to VS -- https://dev.azure.com/devdiv/DevDiv/_workitems/edit/1463290

[debonte]

VS Editor team returned the issue to us. See AzDO link above for their recommendation around language-configuration.json.

[StellaHuang95]

Some updates:

Oleg proposed to delete PTVS command handlers and provide the same language-configuration.json we use in VS Code.

To provide the language configuration rules, I did the following steps:

• Added the following lines to VS-Platform\src\Productivity\TextMate\VSWindows\Setup\TMLanguage.pkgdef:

[$RootKey$\TextMate\LanguageConfiguration\GrammarMapping]
"source.python"="$PackageFolder$\Starterkit\Extensions\python\syntaxes\language-configuration.json"
[$RootKey$\TextMate\LanguageConfiguration\ContentTypeMapping]
"Python"="$PackageFolder$\Starterkit\Extensions\python\syntaxes\language-configuration.json"

• Copied the language-configuration.json from vscode repo, and pasted it to VS-Platform\src\Productivity\TextMate\VSWindows\Setup\Starterkit\Extensions\python\syntaxes
• Deleted the PTVS PythonCommentSelectionCommandHandler

As a result, I was able to comment and uncomment python code in VS. But I also realized that it doesn't solve the original bug, which is uncommenting Python code that was commented in VS Code leaves unexpected space at start of line. And the reason is because VS only adds "#" to the start of each line, while VSCode always adds "#" and a space. It's not python specific, I also found the same issue in C++. 

So the short term fix would be having PTVS add extra space when commenting python code. The long term fix would be on the VS side. Etienne said he will assign this bug to him and take care of it in 17.6 Preview 2 or 3. I will also ship our language config with TextMate on our end.

Some tips from Etienne:

• I was trying to add the changes manually in C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\CommonExtensions\Microsoft\TextMate to test my theory, Etienne suggested running devenv /updateconfiguration after patching and before trying the change.
• %temp%/CommonLanguageLog.txt sometimes gives you useful information such as why language configuration file cannot be opened.
• To debug textmate:

1. in VS-Platform repo:Run init.cmd in a command prompt.
2. Make sure your VS has the C++ workload installed.
3. Open src/Productivity/TextMate/TextMate.VSWindows.sln in VS
4. Set Microsoft.VisualStudio.TextMateGrammars as start-up project.
5. F5 to build and debug.
6. Here is a good place to put a breakpoint to start.

[StellaHuang95]

Etienne said he will take care of it in 17.6 Preview 2 or 3. I submitted a PR on my end to ship my language config with TextMake.

PR: https://dev.azure.com/devdiv/DevDiv/_git/VS-Platform/pullrequest/453429

Once he makes his change, I will remove the comment/uncomment command handler from PTVS, which will fix the issue in the long term.

[StellaHuang95]

Track the second park of the fix here: https://github.com/microsoft/PTVS/issues/7402