Detection of a "Begin String" phrase or "End String" phrase for syntax highlighting does not work when there is a backslash in front of the "Begin String" phrase or "End String" phrase

[KSXia]

Description

Start with a syntax highlighting rule similar to the following screenshot. For this demonstration, I will create the syntax highlighting rule in the String category, but this bug appears to occur regardless of the syntax highlighting category the syntax highlighting rule is created in. The syntax highlighting rule should have the Regular Expression option disabled, have a phrase set for the Begin String option, and have a phrase set for the End String option. I'll refer to the phrase set for the Begin String option as the Beginning Phrase and the phrase set for the End String option as the Ending Phrase.

The Beginning Phrase and Ending Phrase can either be a single character or a set of multiple characters. For this demonstration, I will set the Beginning Phrase to ab and Ending Phrase to xyz, but this bug appears to occur with any characters, including symbols and numbers. While I will set the Beginning Phrase and Ending Phrase to be different in this demonstration, this bug will still occur if the Beginning Phrase and Ending Phrase are the same.

For this demonstration, I will disable the Ignore Case option, but this bug still occurs when the Ignore Case option is enabled.

For this demonstration, I have set the Syntax Kind to Code, but this bug seems to still occur when the Syntax Kind is set to General.

If I type the Starting Phrase and Ending Phrase and add a \ before the Ending Phrase, the syntax highlighting seems not to detect the Ending Phrase, so there is no syntax highlighting. (In all of the screenshots, I have added some text after the string and in the next line after the string to show whether the syntax highlighting continues beyond the string, but this bug still occurs without these texts.)

If the \ is instead added before the Starting Phrase, a similar situation occurs in which there will be no syntax highlighting.

Normally, if I type the Ending Phrase twice, the syntax highlighting will end after the first instance of the Ending Phrase.

However, if there is a \ before the first instance of the Ending Phrase, that first instance of the Ending Phrase seems to be ignored, and the syntax highlighting will instead end after the second instance of the Ending Pharse.

This behavior also occurs if there is text between the Starting Phrase and the first instance of the Ending Phrase.

This behavior also occurs if there is text between the first and second instance of the Ending Phrase.

The syntax highlighting will also continue across lines.

Normally, if there are multiple instances of the Starting Phrase before the Ending Phrase, the syntax highlighting starts from the first instance of the Starting Phrase.

However, if a \ is added before the first instance of the Starting Phrase, that first instance of the Starting Pharse seems to be ignored, and the syntax highlighting will instead start from the second instance of the Starting Phrase.

To Reproduce

1. Go to settings and create a new syntax. I'll refer to this new syntax as Demonstration Syntax.
2. In any highlighting category, create a new highlighting rule. I will use the string highlighting rule, but this bug seems to occur with all of the highlighting categories.
3. Ensure the Regular Expression option is disabled for this new highlighting rule.
4. I disabled the Ignore Case option, but this bug still seems to occur when the Ignore Case option is enabled.
5. Add a character or multiple characters in the Begin String option. I added the ab character, but this bug seems to occur with any singular character or set of multiple characters added to the Begin String option.
6. Add a character or multiple characters in the End String option. I added the xyz character, but this bug seems to occur with any singular character or set of multiple characters added to the End String option.
7. Create a new file.
8. Set the syntax to Demonstration Syntax.
9. Type in the phrase you set to the Begin String option.
10. Type in the phrase you set to the End String option.
11. Type in a \ before the phrase you set to the 'End String' option. (If you used the same settings for the Begin String and End String option as me, you should have typed in ab\xyz).
12. You should be able to observe the bug now: the text you just typed in will not be highlighted.
13. At the end of the text you already typed in, type in the phrase you set to the End String option. (If you used the same settings for the Begin String and End String option as me, you should have typed in ab\xyzxyz).
14. Observe how the highlighting does not end at the first instance of the phrase you set to the End String option.
15. Delete all of the text you have typed in.
16. Type in the phrase you set to the Begin String option.
17. Type in the phrase you set to the End String option.
18. Type in a \ before the phrase you set to the 'Begin String' option. (If you used the same settings for the Begin String and End String option as me, you should have typed in \abxyz).
19. Observe how the text you just typed in will not be highlighted.
20. Right after the Begin String phrase you have typed in, type in the phrase you set to the Begin String option. (If you used the same settings for the Begin String and End String option as me, you should have typed in \ababxyz).
21. Observe how the highlighting does not start at the first instance of the phrase you set to the Begin String option.

Expected behavior

The expected behavior is described in the description.

CotEditor version

4.8.7 (657)

macOS version

14.5

Additional context

I initially discovered this bug when writing strings that ended in a \.

[1024jp]

This behavior is by design.

[KSXia]

May I ask what the reasoning for this behavior is?

[1024jp]

Sorry for my delayed response. I’ve been contemplating your suggestion once more.

CotEditor has been using the previous specification for years. However, as you rightly pointed out, it seems more suitable to omit considering the escape character for simple begin-end string pairs that aren’t regular expressions. In light of this, I will discontinue the current implementation of skipping escaped words in the upcoming version of CotEditor, version 5.0.7.

Thank you for bringing this to my attention.