axboe
one-core-peak.sh: Update this script to python3 and make it more user-friendly
The script one-core-peak.sh was added to assist fio users in obtaining single CPU core peak IOPS performance using the io_uring framework. This is a good discussion on the rational for creating this scipt here. However this script is written as a bash script and was never intended for large-scale consumption.
There has been some recent discussion to update this script to use python3 and make it more user-friendly. This issue captures that and allows for collaboration on this feature.
Couple of ideas on my side , I'd suggest coding in object-oriented python3
The tool could operate like
- detecting the env (os / kernel ver & config / iouring ver / hardware setup (storage / CPU / memory / ...)
- check env and provide hints if some parts are suboptimal
- detect fastest core to run the test (must include e/p core detection)
- define the appropriate flags regarding io_uring /kernel version / config
- run the test
- collect perf data and store an artifact containing the env to get reproducers
- we could archive interesting artifacts to have database-like of good setups
In addition we could also have a mode to rebuild io_uring and / or kernel with known good configs (compiler, configs etc..). This could help people reproducing the results. I started something to reproduce jens work but never published it .... We can also consider having a test suite to ensure the behavior of the tooling in various configs. I did some hardware mockup stuff at work I can probably reuse here.
@ErwanAliasr1 my colleague has already pushed a first attempt at this. I'll link his PR below.
@ErwanAliasr1 I think Martin's initial attempt addresses quite a few of your comments. I do also like the idea of recording the artifacts as much as possible for recreation purposes.
Rewriting the script in python as less value than reconsidering the whole task.
We'll reach the procedural limitations very fast.
I forgot to add we also need a mode to warmup the disk.
Le sam. 8 oct. 2022 à 23:43, Stephen Bates @.***> a écrit :
@ErwanAliasr1 https://github.com/ErwanAliasr1 I think Martin's initial attempt addresses quite a few of your comments. I do also like the idea of recording the artifacts as much as possible for recreation purposes.
— Reply to this email directly, view it on GitHub https://github.com/axboe/fio/issues/1476#issuecomment-1272401983, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCUIHMHMISX6EFXHKJDUMDWCHTH5ANCNFSM6AAAAAAQ7YAVJM . You are receiving this because you were mentioned.Message ID: @.***>
@ErwanAliasr1 the rewrite is just step 1. Let's get that reviewed and merged and then we can work on extended capabilities.
@ErwanAliasr1 Can you elaborate on you concern about reaching the procedural limitations very fast? The script calls t/io_uring in a new process
The code is very intricated, adding new features is very complicated.
If we want to add more features, tests and dumps, that will be z nightmzre to support.
I started a prototype to show my vision around this. I have the first features implemented including a tests and a dump mode.
Give a short and I will share it here.
Le mar. 11 oct. 2022, 20:11, iomartin @.***> a écrit :
@ErwanAliasr1 https://github.com/ErwanAliasr1 Can you elaborate on you concern about reaching the procedural limitations very fast? The script calls t/io_uring in a new process
— Reply to this email directly, view it on GitHub https://github.com/axboe/fio/issues/1476#issuecomment-1275088412, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCUIHNXGWQKKUQNJUM7NPTWCWUVBANCNFSM6AAAAAAQ7YAVJM . You are receiving this because you were mentioned.Message ID: @.***>
Give a short and I will share it here.
@ErwanAliasr1 thanks. If you have a prototype can you fork Jen's fio repo and push your code to a branch and point to that? That way we can comment and collaborate on it and possibly merge the best of yours and @iomartin into something we can to submit.