Ignoring comment lines

[Kazzz-S]

Hello,

Thank you for providing a useful extension, which improves my productivity. This is related to #13. In Python world, I'm regularly working with CSV-like files containing multiple comment (header) lines as below.

# 1st comment line
# 2nd comment line
# 3rd comment line
# :
# next line is the beginning of CSV data...
column_label1,column_label2,column_label3
data1,data2,data3
:
:

It will be nice if I can ignore (and see in one color) those comment lines with additional options like:

"rainbow_csv.ignore_comment_line": true,
"rainbow_csv.comment_line_regexp": ^[#].*

Thank you and best regards, Kazzz

[mechatroner]

Hello, @Kazzz-S ! Thank you very much for your feedback!

I think "comment_line_regexp" option is a a reasonable solution to this problem. We don't even need "ignore_comment_line" variable because the presence of "comment_line_regexp" already indicates that user wants to ignore comment lines.

I was thinking about a different way to do this, which has some pros and cons:

"rainbow_csv.skip_preheader_lines": true

so if this option is set to true, the parser would just skip all lines at the beginning which don't have the same number of fields as the majority of lines in the file.

Anyway a PR that implements either yours or mine approach would be very welcome.

[mechatroner]

BTW, why "SetVirtualHeader" is not suitable for you?

[Kazzz-S]

Hello @mechatroner

I think "comment_line_regexp" option is a a reasonable solution to this problem. We don't even need "ignore_comment_line" variable because the presence of "comment_line_regexp" already indicates that user wants to ignore comment lines.

Thank you for your suggestion. I agree with you :-)

---

BTW, why "SetVirtualHeader" is not suitable for you?

"SetVirtualHeader" works fine to see info about a column on mouse hover.

However, in my use case, I need nicely colored images of CSV files for writing operation manuals of different tools (software and hardware). If a CSV file has many heading comment lines starting with '#' and a comment line contains multiple commas (number is unpredictable), then the one line is displayed in different colors. I just want to avoid this by ignoring comment lines. This is the reason behind.

With best regards, Kazzz

[mechatroner]

Oh, I see. I thought that the main reason is that it is annoying to use "SetVirtualHeader" for each new file.

So, I think it could be possible to "undo" syntax highlighting for pre-header lines using setDecorations VSCode API function, but this needs some investigation.

[enerjazzer]

Please make this option available not only for pre-header lines. Most CSV parsers accept # comments anywhere in the file (to skip the remainder of the line or to skip the line altogether). I need to deal with CSV files which has comments before each block of data, not only before the header.

[yzhang-gh]

Workaround: using Highlight extension with setting

{
    "highlight.regexes": {
        "(^#.*)": {
            "decorations": [
                {
                    "color": "#546e7a"  // Your comment color
                }
            ],
            "filterLanguageRegex": "csv"
        }
    }
}

[enerjazzer]

It's not just the question of highlighting. Unsupported comment lines also break hover tooltips (if the comments are above the first line) and CSV Lint.

[mechatroner]

Comment lines support is now available with version 1.1.1 thanks to @larsonmars ! There is still no special coloring of comment lines though, so the issue will remain open.

[lucky-wolf]

Not sure if this is a good place for this - but I cannot get the Rainbow_csv:Comment_prefix to work.

I love the idea of a regex I love the idea of being able to skip comments anywhere in the file

I mostly just need to skip comments in the header - but currently - setting Comment_prefix to '#' doesn't quite do enough:

• it looks to be failing to work at all because it highlights the line same as all others - stuff before first comma is white, then blue, then... but it's a comment, so it should not be coloring except as comment-color
• I get errors when the line after the comment happens to be blank (why does it care?)

I'm running VSCode if that matters?

If there's a better place for support - please point me to it!!!

Love the extension overall!!!

[mechatroner]

@lucky-wolf You can check if it is working or not by hovering your cursor over the comment line: instead of the column info it will show "Comment" or something like this. Setting Comment_prefix also allows Rainbow CSV to efficiently run separator autodetection, affects Align/Shrink and Multi-cursor edit logic. I am also planning to add support of Comment prefix to RBQL. Unfortunately it is not possible to change color of lines with the comment prefix to the comment color because of some VSCode limitations ( I know about the "decorations" feature but it didn't work well in my tests ). I hope this will change in the future and comment lines would be correctly highlighted.

[lucky-wolf]

that's a bummer that VSCode makes it hard/impossible to do that!

Yeah, I originally thought it wasn't detecting a comment at all - but later realized that it was mis-coloring it - and the error I was seeing in the tooltip - which mentioned "header" and "wrong number of columns" actually was referring to the blank line in between the comment and the real header line.

# my comment, with commas, which looks really silly, but it is a comment so that's great!

reel1,reel2,reel3,reel4,reel5,reel6,basewin,bonuswin,sidebetwin
0, 3, 25, 36, 99, 5, 0, 0, 0

Basically, it was that blank that was generating an error and the fact that the coloring was as-if it were a regular line confused me into thinking it was upset and not detecting the # properly.

Thanks for the response - I hope my explanation aids you going forward - and handling blanks & comments throughout this extension would / will be a huge boon!

Thank you for the extension! It's a great tool!

[Timmmm]

Can you not change the comment colour using semantic highlighting?

[mechatroner]

@Timmmm, I can! And I actually already implemented this in the "rfc" branch along with many other changes. I am planning to publish a new version in about 2 weeks.

[mechatroner]

I just published a new release 3.0.0 which should highlight the comments with the "comment" syntax. Please let me know if there are any issues with it.

[Timmmm]

Works like a charm, thanks!