rust-csv icon indicating copy to clipboard operation
rust-csv copied to clipboard

If headers are not provided then the first record is not trimmed properly

Open vladcebo opened this issue 4 years ago • 1 comments

When user indicates that no headers are available, then the trim doesn't work very well for the first row

#[derive(serde::Deserialize, Debug)]
struct TestStruct {
    col1: String,
    col2: String,
    col3: String,
}

fn test() {
    let mut csv = String::from("a1, b1, c1\n");
    csv.push_str("a2, b2, c2\n");
    csv.push_str("a3, b3, c3\n");


    let mut csv_reader = ReaderBuilder::new()
    .trim(Trim::All)
    .has_headers(false)
    .flexible(true)
    .from_reader(BufReader::new(csv.as_bytes()));

    // Read as byte records, that should improve the performance without a lot of reallocations
    let mut raw_record = csv::ByteRecord::new();
    let headers = csv::ByteRecord::from(vec!["col1", "col2", "col3"]);

    while csv_reader.read_byte_record(&mut raw_record).unwrap() {
        let record = raw_record.deserialize::<TestStruct>(Some(&headers));
        println!("{:?}", record);
    }
}

Note that the first row is not trimmed properly.

Ok(TestStruct { col1: "a1", col2: " b1", col3: " c1" })
Ok(TestStruct { col1: "a2", col2: "b2", col3: "c2" })
Ok(TestStruct { col1: "a3", col2: "b3", col3: "c3" })

With the fix we should trim the fields like this:

Ok(TestStruct { col1: "a1", col2: "b1", col3: "c1" })
Ok(TestStruct { col1: "a2", col2: "b2", col3: "c2" })
Ok(TestStruct { col1: "a3", col2: "b3", col3: "c3" })

vladcebo avatar Jul 23 '21 14:07 vladcebo