YAML multi-line strings inconsistencies

[ItalyPaleAle]

trafficstars

In recent updates to Chroma, yaml multi-line strings (starting with |) seem to be rendered inconsistently.

Example:

steps:

  # Sets FOO to be "some value" in the script and the next ones
  - bash: |
      FOO="some value"
      echo "##vso[task.setvariable variable=FOO]$FOO"

  # Using the $() syntax, the value is replaced inside Azure Pipelines before being submitted to the script task
  - bash: |
      echo "$(FOO)"

  # The same variable is also present as environmental variable in scripts; here the variable expansion happens within bash
  - bash: |
      echo "$FOO"

This renders as:

As you can see, if there's a single line in the multiline string, the number of spaces at the beginning is increased (to match the | character's position). The color is also different. Adding more than one line with the same indentation (even if empty), fixes that.

While the resulting, rendered YAML is valid, it looks odd

[silverwind]

It seems that whitespace is rendered twice. Simple repro:

key: |
  value

HTML of second line:

<span style="color:#d88200">  </span>  <span style="color:#ae81ff">value</span>

The 2 spaces are rendered highlighted once inside the first <span> and then again unhighlighted following that span.

[alecthomas]

Unfortunately YAML is horrifically difficult to parse, and Chroma's lexer is consequently not that robust. You can see the complexity in the Pygments lexer.

[silverwind]

I think it would be a good test case to verify whether the textContent of the HTML output matches the input code exactly. With such a test, bugs like double whitespace rendering should not be possible anymore.

[jmooring]

I think this was resolved with https://github.com/alecthomas/chroma/pull/724. I am now unable to reproduce the problem.