fast-sqlite3-inserts icon indicating copy to clipboard operation
fast-sqlite3-inserts copied to clipboard

Try reusing memory

Open pickfire opened this issue 4 years ago • 8 comments

Fix #10

Unsafe is probably not nice but I am not sure how to satisfy borrow checker there. I tried using mem::swap and related functions but it still seemed like the borrow checker does not know that the reference is not longer valid when vec.clear().

29s -> 27s

pickfire avatar Jul 25 '21 12:07 pickfire

Is it possible to avoid unsafe in this?

avinassh avatar Jul 25 '21 12:07 avinassh

Is it possible to avoid unsafe in this?

I can't think of anything to help avoid these unsafe here, but I'm not a Rust expert, so...

darleybarreto avatar Jul 25 '21 13:07 darleybarreto

One thing I tried to do was using named_params!,

for batch in values.iter() {
    let (a,b,c) = batch;
    row_values.push(named_params!{":area": a, ":age": b, ":active": c});
}

but Params is not implemented for &[&[(&str, &dyn ToSql)]], which is the slice from the row_values.

darleybarreto avatar Jul 25 '21 13:07 darleybarreto

I switched to use https://docs.rs/rsor/0.1.2/rsor/ rsor library as suggested by @upsuper (he suggested a related library with runtime checks). std does not provide these tools and it only have compile-time cost.

pickfire avatar Jul 25 '21 13:07 pickfire

I ran it and I did not see much difference:

./bench.sh
Sun Aug  8 14:06:19 IST 2021 [RUST] basic_batched.rs (100_000_000) inserts

real	0m30.255s
user	0m27.745s
sys	0m2.247s

avinassh avatar Aug 08 '21 08:08 avinassh

What's the difference? 1s?

pickfire avatar Aug 09 '21 10:08 pickfire

@pickfire yup, seems so.

avinassh avatar Aug 14 '21 14:08 avinassh

Yeah, but it's consistent 1s and reduced memory allocation, I am not sure how good the effect will be on other parts.

pickfire avatar Aug 18 '21 02:08 pickfire