CSV highlighting broken

[alextes]

Describe the bug

CSV highlighting appears broken

To Reproduce

Create a data.csv file. Paste in the following data:

address,etherscan_tag,name,details,twitter_handle,category
0x3fC91A3afd70395Cd496C647d5a6CC9D4B2b7FAD,Uniswap: Universal Router,Uniswap,Universal Router,Uniswap,defi
0x1c479675ad559DC151F6Ec7ed3FbF8ceE79582B6,Arbitrum: Sequencer Inbox,Arbitrum,,arbitrum,l2
0x000000000000Ad05Ccc4F10045630fb830B95127,Blur.io: Marketplace,,,blur_io,nft
0x0a252663DBCc0b073063D6420a40319e438Cfa59,,XEN Crypto,,XEN_Crypto,meme
0x6b75d8AF000000e20B7a7DDf000Ba900b4009A80,MEV Bot: 0x6b7...A80,,,,mev
0xA9D1e08C7793af67e9d92fe308d5697FB81d3E43,Coinbase 10,,,coinbase,cex
0x902F09715B6303d4173037652FA7377e5b98089E,Layer Zero: Relayer V2,,,LayerZero_Labs,l1-bridge

The highlighting looks like:

Expected behavior

Something which doesn't highlight numbers or even better highlights alternate columns.

Output of :checkhealth nvim-treesitter

nvim-treesitter: require("nvim-treesitter.health").check()

Installation ~
- OK `tree-sitter` found 0.20.8 (parser generator, only needed for :TSInstallFromGrammar)
- OK `node` found v18.2.0 (only needed for :TSInstallFromGrammar)
- OK `git` executable found.
- OK `cc` executable found. Selected from { vim.NIL, "cc", "gcc", "clang", "cl", "zig" }
  Version: Apple clang version 14.0.3 (clang-1403.0.22.14.1)
- OK Neovim was compiled with tree-sitter runtime ABI version 14 (required >=13). Parsers must be compatible with runtime ABI.

OS Info:
{
  machine = "arm64",
  release = "22.5.0",
  sysname = "Darwin",
  version = "Darwin Kernel Version 22.5.0: Thu Jun  8 22:22:20 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T6000"
} ~

Parser/Features         H L F I J
  - bash                ✓ ✓ ✓ . ✓
  - c                   ✓ ✓ ✓ ✓ ✓
  - cpp                 ✓ ✓ ✓ ✓ ✓
  - csv                 ✓ . . . .
  - dhall               ✓ . ✓ . ✓
  - dockerfile          ✓ . . . ✓
  - fennel              ✓ ✓ ✓ . ✓
  - git_config          ✓ . . . .
  - git_rebase          ✓ . . . ✓
  - gitcommit           ✓ . . . ✓
  - gitignore           ✓ . . . .
  - go                  ✓ ✓ ✓ ✓ ✓
  - gomod               ✓ . . . ✓
  - gosum               ✓ . . . .
  - javascript          ✓ ✓ ✓ ✓ ✓
  - json                ✓ ✓ ✓ ✓ .
  - jsonc               ✓ ✓ ✓ ✓ ✓
  - lua                 ✓ ✓ ✓ ✓ ✓
  - make                ✓ . ✓ . ✓
  - markdown            ✓ . ✓ ✓ ✓
  - markdown_inline     ✓ . . . ✓
  - python              ✓ ✓ ✓ ✓ ✓
  - rust                ✓ ✓ ✓ ✓ ✓
  - scss                ✓ . ✓ ✓ .
  - sql                 ✓ . . ✓ ✓
  - toml                ✓ ✓ ✓ ✓ ✓
  - tsx                 ✓ ✓ ✓ ✓ ✓
  - typescript          ✓ ✓ ✓ ✓ ✓
  - vim                 ✓ ✓ ✓ . ✓
  - vimdoc              ✓ . . . ✓
  - yaml                ✓ ✓ ✓ ✓ ✓

  Legend: H[ighlight], L[ocals], F[olds], I[ndents], In[j]ections
         +) multiple parsers found, only one will be used
         x) errors found in the query, try to run :TSUpdate {lang} ~

Output of nvim --version

NVIM v0.10.0-dev-5003+g7c4d317a6-Homebrew
Build type: Release
LuaJIT 2.1.0-beta3
Run "nvim -V1 -v" for more info

Additional context

No response

[clason]

What does :Inspect show (on the comma and the digits)?

[amaanq]

I didn't account for hex numbers, thanks for the report

[ObserverOfTime]

I wonder if highlighting alternate columns is actually possible. :thinking:

[theHamsta]

I wonder if highlighting alternate columns is actually possible. 🤔

yes, please! :slightly_smiling_face:

[amaanq]

I'm not sure I understand what you mean by alternate columns, an example would be great

[clason]

alternate captures after each comma

so

a,b,c

gets captured as

@string.odd,@string.even,@string.odd

(ceterum censeo csv has to be the most pointless parser yet; kudos.)

[ObserverOfTime]

No, pem is the most pointless.

[clason]

That one has more queries, so no.

[alextes]

@amaanq

I'm not sure I understand what you mean by alternate columns, an example would be great

The best way to work with CSV I've seen so far is from: https://github.com/cameron-wags/rainbow_csv.nvim. I called this alternate column highlighting because the core feature to me is easily separating rows. Thinking about it more, alternate is probably still confusing to look at with many columns. You'd need a fairly large set of colors to keep things clear in unaligned modes. Which it seems people call "rainbow" highlighting.

The above is what I use, but then Treesitter supporting CSV turned it into what you see in the OP. I have highlighting disabled now in my config for CSV but thought I'd report the bug.

One step further you can start using alignment. You can align, edit, and remove whitespace again, or leave it aligned, orrr call it a day and open an editor built to handle CSVs. I like the happy middle above, although I'd take a plugin which uses rainbow highlighting and a "magical overlay" not actual whitespace, to align columns, like below.

[clason]

Yeah, that's out of scope for this project and needs to be a plugin.

[ObserverOfTime]

csv.vim is good.

[boydkelly]

Yes I find that I'm having to turn off treesitter for tsv/csv. Colored highlighting for columns is very practical in many cases. Rainbow csv provided Column0/1/2 etc, and somewhere recently I noticed CSVeven and CSVodd highlight groups (or similar). Also back in the day with plugins like airline we could get the column header in the status line, which is super practical for large csv files. But that seems like its not possible at this time with treesitter.

With default theme currently:

Treesitter disabled (csv.vim plugin):

Treesitter enabled:

[boydkelly]

Yeah, that's out of scope for this project and needs to be a plugin.

But is highlighting out of scope for Treesitter?

[lucario387]

But is highlighting out of scope for Treesitter?

Highlighting that cannot be done with just queries, yes

[boydkelly]

But is highlighting out of scope for Treesitter?

Highlighting that cannot be done with just queries, yes

ok then so what is the solution here? I don't want to be a complainer / insensitive to all the amazing work done here. Its just that treesitter for csv/tsv does not seem to provide any useful featues (at this point?). If all it does according to the TSModuleinfo is highlighting, but the entire file is the same highlight, I would say its broken or not useful at this point in time.

(ok my bad here as in TSModuleIno csv doesn't even claim to highlight. The table shows x's across the board. So I guess its safe to say that its a work in progress that may provide something some day?)

[clason]

not useful at this point in time.

No claim of usefulness was made; all parsers and queries here are offered as-is. (It does syntax highlighting, but only minimally -- which you might not even see with your colorscheme.)

The issue here is that the queries here only do syntax highlighting, which this issue is not about. So this additional highlighting is out of scope and needs to be implemented as a separate plugin (which by all means can use the parser we provide).

And just to put it bluntly: this is not vim-polyglot, which bundles all manner of random filetype plugins and features. We have a much more focused scope, which I believe is a big benefit.