RTF formatter creates incomplete RTF code sequences

[wifhm]

trafficstars

Hi,

I use pygments for converting code sniplets to RTF. I observed a difference between the version 2.4.2 and 2.5.1+. In the newer versions some RTF codes are missing, especially round and curly brackets are ignored.

See the following Java code line which I want to highlight:

 for ( int i = 0; i < nl.getLength(); i++ ) {

Here, the Python code:

lexer = get_lexer_by_name("java") 
formatter = RtfFormatter(style = "xcode")
print(highlight(code, lexer, formatter))

Dependent on which version I install, I get two different results:

 2.4.2:   
{\cf7 for} {\cf6 (} {\cf7 int} {\cf6 i} {\cf6 =} {\cf3 0}{\cf6 ;} {\cf6 i} {\cf6 <} {\cf6 nl}{\cf6 .}{\cf8 getLength}{\cf6 (}{\cf6 )}{\cf6 ;} {\cf6 i}{\cf6 +}{\cf6 +} {\cf6 )} {\cf6 \{}\par

and

2.5.1 and higher :  
{\cf7 for} ( {\cf7 int} {\cf6 i} {\cf6 =} {\cf3 0}; {\cf6 i} {\cf6 <} {\cf6 nl}.{\cf8 getLength}(); {\cf6 i}{\cf6 +}{\cf6 +} ) \{\par

You see the newer version lacks some RTF color codes. In comparison to current SVG and JPEG and older RTF output the first round bracket in Java's for ( has the same color than the for statement, but it must be a different one (as in version 2.4.2).

The problem concerns other styles and languages, too.

[birkenfeld]

The token type of those brackets got changed to Punctuation in 2.5. Color codes are not lacking there - the tokens are just unstyled by the selected style. If you use a different style, they might have some color tag assigned.

I don't see why in the 2.5.1 output you show, the round bracket would have the color of the for: the scope of the \cf7 ends before. Indeed, when I generate the full output and open the RTF with Libreoffice, the bracket is black while the "for" is purple.

[jeanas]

Indeed, when I generate the full output and open the RTF with Libreoffice, the bracket is black while the "for" is purple.

Yes, same result for me.

[wifhm]

Okay, I did some more checks and it seems to be a problem in MS Powerpoint. When inserting the v2.5.1 generated RTF text block via clipboard to a MS Word 2019 document everything looks fine. However, when inserting the same clipboard content to a Powerpoint 2019 text object the mentioned problem occurs (see my manual marks in the following image):

A v2.4.2 generated RTF text block on the other side looks great in PP 2019, too.

[birkenfeld]

Hm, that is unfortunate. Since RTF is mainly used with MS things, and we can't expect MS to fix this bug on their own, I guess we should introduce a default fgcolor of black?

[wifhm]

Perhaps someone else can check if my observation is repeated in another MS PP installation. I use PP 2019 (with updates) on Windows 10 Professional / version 10.0.19044 Build 19044