exrs icon indicating copy to clipboard operation
exrs copied to clipboard

Switch to a `Read` API that does not initialize slices, once stable

Open Shnatsel opened this issue 2 years ago • 0 comments

What can be improved or is missing?

Right now initializing slices takes to be read into takes a non-trivial amount of time. Reading is not parallelized, so it becomes a bottleneck on lightly compressed images and/or on machines with a high core count.

Implementation Approach

In the general case we need Rust's Read trait to stop requiring slices to be initialized. There's an accepted RFC and a tracking issue for it: https://github.com/rust-lang/rust/issues/78485

Technically we could make a specialized fast path for reading from in-memory buffers that bypasses the Read wrapper and can call .extend_from_slice() in the Vec instead of going through the Read implementation, but it's probably not worth the complexity.

Shnatsel avatar Jan 03 '23 20:01 Shnatsel