Syntax highlighting in Github breaks after ``` without close in documentation comment

[parlough]

This may be an issue with linguist, but I haven't had a chance to further investigate yet.

You can see an example in the markdown repository here: https://github.com/dart-lang/markdown/blob/cc57a08156356d7fc8c06c73de972c58ddb39a87/lib/src/block_parser.dart#L421

[slightfoot]

Just experienced this issue today with: https://github.com/arquivolta/desktop/blob/main/lib/platform/win32/arch_to_rootfs.dart

@kevmoo any ideas why this might be an issue?

[DanTup]

Odd - I just tested this in VS Code, and although the comment is highlighting incorrectly, the "code" formatting does end at the end of the comment, so the subsequent code is highlighted correctly:

So there may be some difference between VS Code and GitHub/Linguist. I'm not sure which is the correct behaviour.

[DanTup]

I had a quick look at this, and while VS Code is doing what I want, I'm not convinced it's correct. My interpretation of the textmate grammar spec seems to suggest what GitHub is doing is correct. I've filed https://github.com/microsoft/vscode/issues/189940 about this to see if VS Code can provide a reason for why it's correct (in which case I'll file with Linguist). If not, and it's deemed a bug in VS Code we can try to find a way to handle this better (which might be to only support triple-backticks in dartdocs at the start of a line, and consider another line with the same or a line that doesn't start with a comment the end).

[DanTup]

Based on discussions in https://github.com/microsoft/vscode/issues/189940 and https://github.com/github-linguist/linguist/issues/7015 the consensus seems to be that this is a GitHub bug and VS Code's behaviour with the grammar is correct.

It seems that GH may not be actively maintaining their TextMate grammar support (they're trying to move everything to TreeSitter) so this issue may go unfixed until GitHub moves Dart to tree-sitter (I don't know if/when that would happen - I see a few community projects for Dart in tree-sitter, but it's not clear how complete they are).

Edit: Latest comments on https://github.com/microsoft/vscode/issues/189940 suggest GH is correct and actually VS Code is at fault. If VS Code agree to update their implementation to match GH, then we will also have to update the Dart grammar.

[DanTup]

The conclusion in the issues above was that VS Code is wrong, and therefore this grammar is also wrong (which is a shame, because it will become more complex now we can't rely on while to close nested patterns).

I'm working on a change to use end and never use while, however I hit an issue in the string scanner we use for tests here while testing it. I've opened a PR to try and get a fix there (though there are some open questions). I'll come back to this once a fix has landed.

[DanTup]

I expect this to be fixed by 5b362281ed55724bb2ed95a03a3d65c2a7c03349 and e1ac5c446c2531343393adbe8fff9d45d8a7c412, but I don't know on what schedule GitHub updates the grammars.

[DanTup]

Seems like this has been updated on GitHub, because the code linked and shown in the screenshots above now looks like this (the code under the comments that contain uneven backticks are no longer coloured like comments).

@parlough's example:

@slightfoot's example: