fio
fio copied to clipboard
axboe
Fio replay blktrace file failed when blktrace file is large than 3GB.
Fio replay blktrace file failed when blktrace file is large than 3GB. cmd: fio --name=replay --filename=/dev/md1 --read_iolog=md1.bin --ioengine=libaio --iodepth=1
When md1.bin is large than 3GB，replay io will make the system dump，then dump reason is out of memory. But it works fine when the read_iolog is 100MB.
Does anyone help me? Thank you!
@fwin : This might have been fixed by a6eaf6c9101cd3f2895376944e6425a43613dbf6 - do you still see this problem with the latest fio releases?
Looks like this is still a problem. With fio 2.13 a file with 100 million I/Os to replay uses up enough memory on a 4GByte system to cause an OOM. What's really needed is some sort of streaming replay mode...
Yes, that's exactly right, it should be made streaming somehow. We already have the infrastructure, in fact we stream the entries in at init time. We just stream all of them... Only complication here is that we need a helper to stream them in, we can't do it inline when we're running IO. At least not without potentially slowing down the replay.
This is semi-addressed by https://github.com/axboe/fio/commit/98e7161cda6a1f5b0cbd8979c261db8eb3a4ab72 but it won't work for blktrace files - only fio's own iolog format...
Indeed, the same kind of chunking could be done on blktrace replay. It should be trivial to do, since the input is an already sorted blktrace trace.
