otp icon indicating copy to clipboard operation
otp copied to clipboard

Optimise prim_file in write mode

Open michalmuskala opened this issue 8 months ago • 2 comments

This adds two optimisations to prim_file.

For files in various write modes, it skips allocating the read buffer - this saves a lot on allocations when writing a lot of files.

For various write operations, if file is opened in just write mode, we don't need to call seek to reset position before writing, we can just write, since we couldn't have read from the file, there's nothing to synchronise. This saves one syscall per write.

For a simple file:write_file(~"/tmp/foo", ~"a", [raw]) call, this shows an improvement of about 50%, though the data is very noisy:

                ips        average  deviation         median         99th %
before       635.37        1.57 ms   ±249.91%        0.94 ms       11.76 ms
after        968.94        1.03 ms   ±156.32%        0.74 ms        6.70 ms

michalmuskala avatar May 12 '25 16:05 michalmuskala

CT Test Results

    3 files    142 suites   49m 54s ⏱️ 1 649 tests 1 592 ✅ 57 💤 0 ❌ 2 372 runs  2 295 ✅ 77 💤 0 ❌

Results for commit 78a61324.

:recycle: This comment has been updated with latest results.

To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass.

See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally.

Artifacts

// Erlang/OTP Github Action Bot

github-actions[bot] avatar May 12 '25 16:05 github-actions[bot]

Thanks for the PR, I'll include this in nightly tests after 28 is out :-)

jhogberg avatar May 19 '25 11:05 jhogberg

I pushed an update. It changes thing slightly more, avoiding using the r_buffer and r_ahead_size fields in general when the file is opened just for writing saving a bit more on allocations. It should also be handling correctly read/write or any combination of the modes. I checked file_SUITE and prim_file_SUITE

michalmuskala avatar Jul 25 '25 11:07 michalmuskala

gentle ping on this - is there anything else I need to do here?

michalmuskala avatar Oct 16 '25 08:10 michalmuskala

Nope, sorry for letting this linger for so long. I've re-added it to the nightly test runs and will merge tomorrow if all goes well.

jhogberg avatar Oct 16 '25 12:10 jhogberg

Merged, thanks again for the PR!

jhogberg avatar Oct 17 '25 06:10 jhogberg