intellij-csv-validator icon indicating copy to clipboard operation
intellij-csv-validator copied to clipboard

Support for escape characters; not quotes

Open TheGoesen opened this issue 4 years ago • 1 comments

Some csv uses escape charactes instead of quotes. So Instead of "Hello, World", "1" The csv will be like Hello \, World,1

Most notable example would be mysql default output. https://dba.stackexchange.com/questions/46584/escaped-by-clause-in-load-data-infile https://dev.mysql.com/doc/refman/8.0/en/load-data.html

My understanding is that this plugin doesn't support this escaping strategy at all? This feature might be non trivial to implement using a precompiled lexxer. I humbly suggest evaluation of a more flexible backend like commons-csv. This would also solve the ever so present "cant use my own custom record seperator"

TheGoesen avatar Feb 11 '20 16:02 TheGoesen

@TheGoesen Thats a quite interesting escaping strategy - first time seeing it, every day is a learning. Thanks for pointing this out. And so yes, this plugin does not support it.

I somewhat like this kind of escaping control characters, but as you already mentioned, it might be some effort. I have to evaluate it in more detail.

I humbly suggest evaluation of a more flexible backend like commons-csv. This would also solve the ever so present "cant use my own custom record seperator"

Let me just cite myself:

Parsing CSV (once) with adjustable settings is a different scope than providing an editor with a defined language, syntax validation, non-text editor etc.

IntelliJ works with PsiElements, which must be supported by the lexer. For parsing a file, the framework requires therefore a Lexer implementation. Of course its possible to have an own lexer implementation based on a certain CSV library (btw. common-csv doesn't seem to be the best choice regarding performance), but until now I didn't consider it to be a reasonable effort. My gut feeling is that a "pre-compiled" parser/lexer should perform better than a dynamic and very flexibel one. Thats the trade-off.

However, I would be happy to learn that this assumption is wrong, but I myself do not have the capacity to validate the performance/flexibility ratio. Contributions are welcome!

SeeSharpSoft avatar Feb 11 '20 18:02 SeeSharpSoft