q icon indicating copy to clipboard operation
q copied to clipboard

Make sure that q is strict with regard to open-ended quoting

Open rainStone819 opened this issue 5 years ago • 4 comments

$ cat a.txt
"1
$ q  -W all 'select * from a.txt'
"1"

the output is not correct

rainStone819 avatar Jul 24 '20 12:07 rainStone819

Hi, sorry for the late reply,

This is related to the fact that the default input-quoting mode is minimal - If you set the input quoting to -w none, it will read the file properly.

Indeed, I would expect q to fail in the case you mention above, since there's no closing quote to the end of value, just an opening one. I'll check this out.

$ q -w none -W none 'select * from a.txt' -c 1
"1
$

# with -W all, the output will be wrapped with double quotes, and in addition, the internal double-quote in the value will doubled (according to the csv standard)
$ q -w none -W all 'select * from a.txt' -c 1
"""1"

harelba avatar Sep 11 '20 09:09 harelba

seems to be an issue in the csv module in python itself, which means that it would be a challenge to work around - https://bugs.python.org/issue30034

However, csv strict mode might help, i'll check it out and see if there's something that can be done (it might break other valid q usages).

harelba avatar Sep 11 '20 09:09 harelba

Hi, thank you for your reply,

Not using csv module can fix this bug, but it breaks other valid q usages. The changes are as follows and use -W none : image

rainStone819 avatar Sep 11 '20 10:09 rainStone819

Hi @rainStone819 the csv module provides a lot of very complicated functionality for reading CSVs, just replacing it with a .split() would break a lot of the current logical behaviour.

harelba avatar Sep 11 '20 13:09 harelba