json icon indicating copy to clipboard operation
json copied to clipboard

Support for JSONL

Open aravinds03 opened this issue 1 year ago • 5 comments

Hi, Is there any support for jsonl format ? If yes, can you please document it with example code?

aravinds03 avatar Aug 29 '23 06:08 aravinds03

Does StreamDeserializer not work for this?

vidhanio avatar Sep 08 '23 15:09 vidhanio

For me it worked. I used it like this

let loaded: Vec<YourType> =
     serde_json::Deserializer::from_str(&std::fs::read_to_string(file)?)
                .into_iter::<YourType>()
                .collect::<std::result::Result<Vec<_>, _>>()?;

clotodex avatar Sep 10 '23 14:09 clotodex

How about serializing Vec<T> into JSONL ?

aravinds03 avatar Sep 13 '23 20:09 aravinds03

This was my naive approach (not working with gigantic data amounts though)

std::fs::write(
    file,
    your_vec
        .into_iter()
        .map(|e| serde_json::to_string(&e))
        .collect::<std::result::Result<Vec<_>, _>>()?
        .join("\n"),
)?;

clotodex avatar Sep 16 '23 14:09 clotodex

This was my naive approach (not working with gigantic data amounts though)

std::fs::write(
    file,
    your_vec
        .into_iter()
        .map(|e| serde_json::to_string(&e))
        .collect::<std::result::Result<Vec<_>, _>>()?
        .join("\n"),
)?;
let file = File::options()
    .write(true)
    .create(true)
    .open("your_file.json")?;

your_vec.into_iter().try_fold(file, |mut file, item| {
    serde_json::to_writer(&mut file, &item)?;
    writeln!(file)?;
    Ok(file)
})?

// or

for item in your_vec {
    serde_json::to_writer(&mut file, &item)?;
    writeln!(file)?;
}

vidhanio avatar Sep 17 '23 00:09 vidhanio