CleverCSV icon indicating copy to clipboard operation
CleverCSV copied to clipboard

delimiter detection error

Open tooptoop4 opened this issue 2 years ago • 4 comments

For a pipe delimited file it is being detected as space delimited when i feed in around 1000 rows (however csv lib correctly detects pipe), if i only feed in 100 rows then clevercsv is also able to detect pipe

ie like below data but 55 columns wide "Private Company"|"Point Rd"

tooptoop4 avatar May 04 '22 01:05 tooptoop4

Hi @tooptoop4, thanks for reporting this issue. Are you able to share the file? I may be able to fix the problem

GjjvdBurg avatar May 04 '22 19:05 GjjvdBurg

unfortunately it contains private data

tooptoop4 avatar May 04 '22 21:05 tooptoop4

i have another case where 100 rows seems more accurate (it correctly picks , ) but 1200 rows picks \

tooptoop4 avatar May 06 '22 23:05 tooptoop4

Results can be a bit noisy if you don't read the entire file. Theoretically, it might be the case that the "key" to reading the file correctly is only the last row. Therefore, if time allows, it's generally recommended to detect the dialect using the entire file. If you have a case where detection fails on the entire file, please let me know

GjjvdBurg avatar May 12 '22 21:05 GjjvdBurg