rust-csv icon indicating copy to clipboard operation
rust-csv copied to clipboard

Deserializing a set of fields as a vector

Open OmnipotentEntity opened this issue 7 years ago • 6 comments

I apologize if this is a somewhat basic question, but is there an easy way to direct rust-csv or serde from rust-csv to deserialize a set of fields as a Vec?

Example:

field1,field2,list
a,b,1,2,3,4,5,6,7
c,d,8,9,10,11,12,13,14

I'd like to deserialize this to a:

struct mydata {
    field1 : String,
    field2: String,
    list: Vec<u32>,
}

OmnipotentEntity avatar Sep 15 '18 02:09 OmnipotentEntity

@OmnipotentEntity Could you please describe what you actually tried, and include what happened and what you expected or wanted to happen? Without that information, it is very hard to help you. Moreover, have you read the docs for automatic deserialization, and if so, could you say what part of it was unclear so that it can be fixed?

As far as I can tell, your request "just works," but I'm not quite understanding where you ran into a problem. I'd like to understand that so that it can be fixed.

Here's an example:

extern crate csv;
extern crate serde;
#[macro_use]
extern crate serde_derive;

#[derive(Clone, Debug, Deserialize, Serialize)]
struct MyRecord {
    field1 : String,
    field2: String,
    list: Vec<u32>,
}

fn main() -> Result<(), Box<::std::error::Error>> {
    let data = "\
field1,field2,list
a,b,1,2,3,4,5,6,7
c,d,8,9,10,11,12,13,14
";

    let mut rdr = csv::ReaderBuilder::new()
        .flexible(true)
        .from_reader(data.as_bytes());
    for result in rdr.deserialize() {
        let record: MyRecord = result?;
        println!("{:?}", record);
    }
    Ok(())
}

BurntSushi avatar Sep 15 '18 11:09 BurntSushi

I do apologize, it does just work, but I had to tell the reader to ignore the header. I didn't see flexible documented, and that was my oversight. Thanks for following up.

OmnipotentEntity avatar Sep 16 '18 03:09 OmnipotentEntity

Great! I'm going to reopen this since it sounds like there is an opportunity to improve the docs here. Where would the best place be too mention the existence of flexible reading? Should we mention it in the docs on the deserialize method?

BurntSushi avatar Sep 16 '18 11:09 BurntSushi

I'm unsure about where the best place is. I eventually found it by searching the issues on this Github. Documentation on the deserialize method, would be a good place for it? Or perhaps in the "using with Serde" section of the overview?

OmnipotentEntity avatar Sep 17 '18 00:09 OmnipotentEntity

I just ran into this issue and spent at least an hour debugging until I found this thread. :(

Where would the best place be too mention the existence of flexible reading?

I would recommend the reader example to include additional comments. I believe, and you can correct me if I'm wrong, that you have to either specify .has_headers(false) or .flexible(true).

Alternatively, I think flexible is slightly wrong. It's overly aggressive.

parsing CSV data will return an error if a record is found with a number of fields different from the number of fields in a previous record.

However it is perfectly acceptable for non-header rows to contain >= fields as the header row IFF the last field is a nested record/Vec. (I'm not sure the correct terminology.) This would also allow non-flexible parse to ensure that a tail Vec field has the same number of values for all data rows. Which is desirable.

forrestthewoods avatar Apr 05 '20 08:04 forrestthewoods

I don't see a way to implement your desired flexible semantics. PRs to improve the docs are welcome.

BurntSushi avatar Apr 05 '20 11:04 BurntSushi