lz4_flex icon indicating copy to clipboard operation
lz4_flex copied to clipboard

Use Vec::extend_from_within() in place of custom unsafe code

Open Shnatsel opened this issue 4 years ago • 3 comments

Vec::extend_from_within() is a newly stabilized function that allows appending a part of a vector to itself.

Looks like it could replace some uses of unsafe code in lz4_flex, e.g. https://github.com/PSeitz/lz4_flex/blob/053483689099420c8576a378d8bf4e1d1fe2964f/src/block/decompress.rs#L15-L16

If it turns out to be not as good as the unsafe variant (I see that you're specifically avoiding memcpy), perhaps it could still make the safe variant faster.

Shnatsel avatar Jun 18 '21 16:06 Shnatsel

I did some experiments after your comment on reddit last time. The unsafe version was faster, since there was still some overhead in comparison to the pointer approach.

In the meantime the block api changed and it uses slices now as parameters, so extend_from_slice is not applicable anymore. But there were some gains in the safe variant by using the slice copy methods.

PSeitz avatar Jun 20 '21 17:06 PSeitz

The slice-based approach as used right now is sadly unsound; I've opened #19 about that.

More generally, to make this sound you need to write either to a Vec without using set_len() or to &mut MaybeUninit to make things sound and fast. Writing to a slice would require initializing it first, which in case of something as fast as LZ4 would cut performance in half.

Shnatsel avatar Jun 20 '21 19:06 Shnatsel

Well, I guess writing to the Vec's capacity through raw pointers and only then calling set_len() would also work, provided you're careful about aliasing.

Shnatsel avatar Jun 20 '21 19:06 Shnatsel