fast-sqlite3-inserts icon indicating copy to clipboard operation
fast-sqlite3-inserts copied to clipboard

Rust reuse memory

Open pickfire opened this issue 3 years ago • 6 comments

Rather than creating the Vec in the inner loop, you can create it in the outer loop using with_capacity with capacity of batch size so it will not allocate in inner loop, then just reuse the Vec.

Once the execute is used, can just Vec clear to clear the items without deallocating memory. I wonder how much faster will it make.

pickfire avatar Jul 19 '21 00:07 pickfire

I tried this one and it gives me 9% speed up on the basic_batched benchmark.

dovahcrow avatar Jul 19 '21 07:07 dovahcrow

damn, thats amazing. Would you like to send a PR @pickfire @dovahcrow

avinassh avatar Jul 25 '21 06:07 avinassh

I would like to see that too. I tried to remove those ::new() from the loops, but I couldn't find a way to make the borrow checker happy.

darleybarreto avatar Jul 25 '21 12:07 darleybarreto

I just tried it, I seemed to have faced issues with borrow checker as well. I just use transmute to increase their lifetime and it worked. But yeah, people is not happy seeing unsafe. &dyn ToSql make it hard for the borrow checker I believe.

pickfire avatar Jul 25 '21 12:07 pickfire

@pickfire we can have another benchmark code just for unsafe and I don't mind unsafe code. may be we call it as batched_unsafe?

avinassh avatar Jul 25 '21 12:07 avinassh

@avinassh Without unsafe there is another set of optimization in place, it already have reduced allocations with that patch.

pickfire avatar Jul 25 '21 13:07 pickfire