lemmeknow
lemmeknow copied to clipboard
proposal: use `SmallVec` instead of `Vec` for buffer
The idea is to use smallvec for buffer while extracting strings from file. We only consider the strings which are longer than 4 characters, so for other strings, which we are going to reject anyway, we can avoid heap allocation caused due to buffer vector. here.
let mut buffer: SmallVec<[u8; 4]> = smallvec![]; // TODO: change 4 to more optimal number
So, what to do?
- Change
VectoSmallVec - Experiment with different sizes ( at least 4 will be good imho, but we should try other using quasi-doubling strategy , i.e. 4, 8, 16, 32 )
- Benchmark the code to see if it actually improves performance.
- Choose the one with best performance. ( post the results here or while making PR )
It would be amazing if you can post benchmark of all sizes, then we can choose the most optimal.