sunder
sunder copied to clipboard
The default behavior of `std::read_all` is slow for large files
Discovered on commit 064e8ab19b64664729c1d39145225e9012bca267
. The function std::read_all_with_allocator
will buffer data 4096 bytes at a time and will call std::slice[[byte]]::resize_with_allocator
for each buffered read. The default std::global_allocator
(of type std::general_allocator
) behaves poorly when reallocations occur every 4096 bytes, leading to poor performance when multi-MB files are read with std::read_all
.
Changing the behavior of std::read_all_with_allocator
would be difficult, as a valid use case for std::read_all_with_allocator
could be to dynamically allocate a byte buffer exactly big enough to store the file contents, and then read the file with a std::linear_allocator
.