fastcsv
fastcsv copied to clipboard
fastcsv breaks with StringIO containing new lines near empty fields near BUFSIZE boundaries
Hi, it seems like when there is a new line after a blank field near the BUFSIZE, passing in a StringIO object is broken, but a String seems to work. I'm specifically referring to the BUFSIZE at https://github.com/jpmckinney/fastcsv/blob/master/ext/fastcsv/fastcsv.rl#L177
using fastcsv 0.0.6 and ruby 2.7.6
minimal reproducible test case:
require 'fastcsv'
csv = ''
bufsize = 2 ** 14
rowsize = 2 ** 3
row = (',' * (rowsize - 1)) + "\n"
num_rows = bufsize / rowsize
num_rows.times do
csv << row
end
FastCSV.raw_parse(StringIO.new(csv)) do |row|
end
output:
% bundle exec ruby fastcsv_bug.rb
Traceback (most recent call last):
2: from fastcsv_bug.rb:13:in `<main>'
1: from /Users/thomas/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/fastcsv-0.0.6/lib/fastcsv.rb:8:in `raw_parse'
/Users/thomas/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/fastcsv-0.0.6/lib/fastcsv.rb:8:in `raw_parse': Unquoted fields do not allow \\r or \\n (line 2048). (FastCSV::MalformedCSVError)
I can reproduce the error. I haven't been able to determine why the error occurs (it's been a while since I wrote Ragel.)