serde_arrow icon indicating copy to clipboard operation
serde_arrow copied to clipboard
verified publisher icon chmp

surprising performance

Open sthornington opened this issue 9 months ago • 6 comments

I am a little surprised at the performance of serde_arrow - a Vec<MyType> of only say 8 million records (~15 fields) is nevertheless taking 6+ seconds in to_record_batch -- is it possible that the generated code is not allocating the arrays with the total capacity to begin with or something ?

sthornington avatar Apr 07 '25 20:04 sthornington