lancedb icon indicating copy to clipboard operation
lancedb copied to clipboard

Feature: Rust serde-based import / export

Open westonpace opened this issue 11 months ago • 1 comments

SDK

Rust

Description

Rust users are often unfamiliar with arrow and learning arrow can be a significant bottleneck to using lancedb. We could make it easier to import / export values by having the users create structs defining the record (similar to pydantic). In rust this is typically done with serde and there is a https://docs.rs/serde_arrow/latest/serde_arrow/ crate that we can investigate. The end result should be something like...

#[derive(Serialize, Deserialize, Debug)]
struct Location {
  x: f64,
  y: f64,
}

#[derive(Serialize, Deserialize, Debug)]
struct MyRecord {
  score: f32,
  caption: String,
  location: Location,
  clip: Vec<f64>,
}

...

tbl.add(data_source::from_structs(&[
  MyRecord { score: 32.0, caption: "my caption".to_string(), clip: vec![1.0, 2.0, 3.0], location: Location { x: 0.0, y: 0.0 } }
])).execute().await.unwrap();

westonpace avatar Mar 13 '24 11:03 westonpace

I can work towards this

HusainKapadia avatar Mar 15 '24 10:03 HusainKapadia