Incorrect tokenization of multiline comments with whitespace

[alexr00]

Checklist

• [ ] This problem exists even with the setting "C_Cpp.enhancedColorization": "Disabled"
• [ ] This bug exists for C
• [x] This bug exists for C++
• [ ] This bug exists for Objective-C
• [ ] This bug exists for Objective-C++

The code with a problem is:

#define TCP_NOTSENT_LOWAT	 25 /* Limit number of unsent bytes in

				       write queue.  */

#define TCP_CC_INFO		 26 /* Get Congestion Control

				       (optional) info.  */

#define TCP_SAVE_SYN		 27 /* Record SYN headers for new

				       connections.  */

#define TCP_SAVED_SYN		 28 /* Get SYN headers recorded for

				       connection.  */

It looks like:

** screenshot + theme name preferable **

It should look like:

Comments should span multiple lines.

Originally from @duzeev in https://github.com/microsoft/vscode/issues/107565

[jeff-hykin]

I think this is unfixable without the tree sitter.

The #define uses the while pattern to ignore ranges (ranges like a { with no }), but in doing so, it ignores the range of a comment too.

[matter123]

@jeff-hykin Apologies for not being involved for the last 6 months.

The reason this breaks is that :single_line_macro matches as the line does not end in \

It should be possible to have :single_line_macro exclude lines that have /* that is not later succeeded by */. This would cause the macro matching to fall back to multi-line macro matching where the comment would be properly matched.

[jeff-hykin]

You're good @matter123 , I've been gone for awhile, and I probably will be intermittent at best. I've gone back to just using master since it's too much effort with the branches when I have such short time.

Thanks for the comment, I forgot there was a single line macro. Still sounds like a hack to me so I'll let you implement that one 😆. If there was a string with /* inside of it, seems like that would be a problem.