[C#] Deeply nested parens causes editor hang.

[aeschli]

From @staticvoidmain on August 2, 2018 16:52

Issue Type: Bug

Problem

Too many nested parens on a single line cause C# file to hang.

Additional Info

• I recently reversed some VB to C# and pasted it into code, resulting in a complete hang of the vscode process.
• It's fine if these are separated into multiple lines as in the example, however if it's all on a single line you'll get a hang.
• It's definitely the syntax highlighter, and seems to be specific to C#.
• Tested against insiders, and with no extensions on the current build.
• Tested the same line with other c-like language file extensions (js, ts, cpp, java) and they handle it without a crash.
• Oh, also, if you drop the (object) casts off it's fine as well.

Steps To Reproduce

1. New file > evil_line.cs
2. Paste these contents

// don't try to join these lines...
var message = Conversions.ToString(
	Operators.ConcatenateObject(
		Operators.ConcatenateObject(
			Operators.ConcatenateObject(
				Operators.ConcatenateObject(
					Operators.ConcatenateObject(
						Operators.ConcatenateObject(
							Operators.ConcatenateObject(
								Operators.ConcatenateObject(
									Operators.ConcatenateObject(
										Operators.ConcatenateObject(
											Operators.ConcatenateObject(
												Operators.ConcatenateObject(
													Operators.ConcatenateObject(
														Operators.ConcatenateObject(
															Operators.ConcatenateObject(
																Operators.ConcatenateObject(
																	Operators.ConcatenateObject(
																		Operators.ConcatenateObject(
																			Operators.ConcatenateObject(
																				Operators.ConcatenateObject(
																					Operators.ConcatenateObject(
																						(object) strMsg,
																						Interaction.IIf(
																							Strings.Len(strMsg) > 0, 
																							(object) "\r\n", 
																							(object) null)), 
																					(object) "\r\n"),
																				(object) "name '"), 
																			(object) name), 
																		(object) "'"), 
																	(object) "\r\n"), 
																(object) "age '"), 
															(object) age), 
														(object) "'"),
													(object) "\r\n"),
												(object) "dob '"),
											(object) dob), 
										(object) "'"), 
									(object) "\r\n"), 
								(object) "favoriteColor '"), 
							(object) favoriteColor), 
						(object) "'"), 
					(object) "\r\n"), 
				(object) "favoriteFilm '"), 
			(object) favoriteFilm), 
		(object) "'")
	);

3. Join Lines

VS Code version: Code 1.25.1 (1dfc5e557209371715f655691b1235b6b26a06be, 2018-07-11T15:43:53.668Z) OS version: Windows_NT x64 10.0.14393

System Info

Item	Value
CPUs	Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (4 x 2400)
GPU Status	2d_canvas: enabled flash_3d: enabled flash_stage3d: enabled flash_stage3d_baseline: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on native_gpu_memory_buffers: disabled_software rasterization: unavailable_software video_decode: enabled video_encode: enabled vpx_decode: enabled webgl: enabled webgl2: enabled
Memory (System)	12.00GB (7.34GB free)
Process Argv	C:\Program Files\Microsoft VS Code\Code.exe
Screen Reader	no
VM	100%

Extensions (12)

Extension	Author (truncated)	Version
xml	Dot	2.3.1
EditorConfig	Edi	0.12.4
tslint	eg2	1.0.34
vscode-great-icons	emm	2.1.39
code-beautifier	mic	2.1.0
mssql	ms-	1.4.0
python	ms-	2018.7.1
cpptools	ms-	0.17.7
PowerShell	ms-	1.8.2
team	ms-	1.136.0
debugger-for-chrome	msj	4.8.2
vscode-javascript-booster	sbu	0.10.1

Copied from original issue: Microsoft/vscode#55687

[aeschli]

From @vscodebot[bot] on August 2, 2018 16:52

(Experimental duplicate detection) Thanks for submitting this issue. Please also check if it is already covered by an existing one, like:

• An extension's default plain object setting is displayed as [object Object] (#45987)
• [json] Outline View displays array of objects as duplicated keys of object (#53605)

[aeschli]

From @alexandrudima on September 12, 2018 15:6

Repro steps:

• create file.txt with the contents:

var message = Conversions.ToString( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( Operators.ConcatenateObject( (object) strMsg, Interaction.IIf( Strings.Len(strMsg) > 0, (object) "\r\n", (object) null)), (object) "\r\n"), (object) "name '"), (object) name), (object) "'"), (object) "\r\n"), (object) "age '"), (object) age), (object) "'"), (object) "\r\n"), (object) "dob '"), (object) dob), (object) "'"), (object) "\r\n"), (object) "favoriteColor '"), (object) favoriteColor), (object) "'"), (object) "\r\n"), (object) "favoriteFilm '"), (object) favoriteFilm), (object) "'") );

• change the language to C#

[aeschli]

From @alexandrudima on September 12, 2018 15:12

Using the steps described in https://github.com/Microsoft/vscode/issues/57593#issuecomment-417276769

The bad regex is:

(?x)
(?<return-type>
  (?<type-name>
    (?:
      (?:ref\s+(?:readonly\s+)?)?   # ref return
      (?:
        (?:(?<identifier>@?[_[:alpha:]][_[:alnum:]]*)\s*\:\:\s*)? # alias-qualification
        (?<name-and-type-args> # identifier + type arguments (if any)
          \g<identifier>\s*
          (?<type-args>\s*<(?:[^<>]|\g<type-args>)+>\s*)?
        )
        (?:\s*\.\s*\g<name-and-type-args>)* | # Are there any more names being dotted into?
        (?<tuple>\s*\((?:[^\(\)]|\g<tuple>)+\))
      )
      (?:\s*\?\s*)? # nullable suffix?
      (?:\s*\[(?:\s*,\s*)*\]\s*)* # array suffix?
    )
  )\s+
)
(?<interface-name>\g<type-name>\s*\.\s*)?
(\g<identifier>)\s*
(<([^<>]+)>)?\s*
(?=\()

That is coming from this rule -- https://github.com/Microsoft/vscode/blob/667e4c2a1a85946344cd1194348d4f2e65fa53cc/extensions/csharp/syntaxes/csharp.tmLanguage.json#L1220

That line was changed last via https://github.com/Microsoft/vscode/commit/076c08493034201ab4e7c56b6258246def7afa5e

[DustinCampbell]

Same issue as #101

[DustinCampbell]

@aeschli : At the moment, with regex we can choose to be accurate for 99.99% of C# code but exponentially slow for pathological cases, or we can be inaccurate and fast for 100% of C# code. This is a very difficult choice to make. Is there any hope of a non-regex engine for syntax highlighting in VS Code?

[alexdima]

@DustinCampbell Yes, we eventually want to add API for setting tokens from an extension. That would allow for AST-based tokenization or for semantic tokenization.

At the same time (or quite likely even sooner), I am also considering moving the TM tokenization to a separate thread (a web worker which can now load native node modules such as oniguruma needed for regex) such that these hangs don't lead to a complete VS Code crash. We could implement recovery from such a slow regex by killing the web worker and then avoiding a range of lines next time, etc... or perhaps disabling tokenization in the problematic file.

[DustinCampbell]

Both of the ideas you describe would be awesome!

[JoeRobich]

This case seems to be an editor issue perhaps. When inspecting tokens and scopes we see the correct scope however it is not rendering in the correct color.

[wise0704]

VSCode 1.81.1

Only the first ToString is highlighted.

Main branch: Open -> any input

Nothing is highlighted on open, but works after any input change (not just enter).

[staticvoidmain]

Had honestly forgotten I opened this and just saw the update from a few days ago. Decided to open up my old example in the latest VSCode and the issue persists, however, what I initially thought was the parenthesis seems to also factor in the line length.

Oddly the threshold is not a power of 2 length like 1024, it's right at 1002 characters on the line that performance falls off.

Also, this was just some ugly VB.Net to C# conversion that I thought exhibited interesting editor behavior, if you guys want to just close this out feel free.