zos icon indicating copy to clipboard operation
zos copied to clipboard

TC358. Actual speed is less than expected

Open VenyaPanou opened this issue 3 years ago • 4 comments

Actual speed : root@vm:~# fio fiorandomreadlatency.fio job1: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1 fio-3.16 Starting 1 process Jobs: 1 (f=1): [r(1)][100.0%][r=10.7MiB/s][r=2742 IOPS][eta 00m:00s] job1: (groupid=0, jobs=1): err= 0: pid=1227: Mon Jun 27 06:53:37 2022 read: IOPS=2732, BW=10.7MiB/s (11.2MB/s)(1281MiB/120001msec) slat (usec): min=11, max=600, avg=29.40, stdev=10.56 clat (usec): min=5, max=5490, avg=329.38, stdev=61.16 lat (usec): min=204, max=5565, avg=359.67, stdev=61.84 clat percentiles (usec): | 1.00th=[ 249], 5.00th=[ 269], 10.00th=[ 277], 20.00th=[ 285], | 30.00th=[ 293], 40.00th=[ 306], 50.00th=[ 314], 60.00th=[ 326], | 70.00th=[ 343], 80.00th=[ 371], 90.00th=[ 408], 95.00th=[ 441], | 99.00th=[ 494], 99.50th=[ 523], 99.90th=[ 619], 99.95th=[ 750], | 99.99th=[ 1598] bw ( KiB/s): min= 8432, max=12288, per=99.98%, avg=10926.26, stdev=628.41, samples=239 iops : min= 2108, max= 3072, avg=2731.53, stdev=157.12, samples=239 lat (usec) : 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%, 250=1.09% lat (usec) : 500=98.07%, 750=0.79%, 1000=0.02% lat (msec) : 2=0.03%, 4=0.01%, 10=0.01% cpu : usr=3.68%, sys=9.07%, ctx=335723, majf=0, minf=11 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=327870,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs): READ: bw=10.7MiB/s (11.2MB/s), 10.7MiB/s-10.7MiB/s (11.2MB/s-11.2MB/s), io=1281MiB (1343MB), run=120001-120001msec

VenyaPanou avatar Jun 27 '22 09:06 VenyaPanou

please attach the fio script

xmonader avatar Jun 27 '22 13:06 xmonader

could be related to https://github.com/threefoldtech/zos/issues/1467 @muhamadazmy

xmonader avatar Jun 27 '22 13:06 xmonader

@xmonader

The configuration

fiorandomreadlatency.fio (file name)
[global]
bs=4K
iodepth=1
direct=1
ioengine=libaio
group_reporting
time_based
runtime=120
numjobs=1
name=readlatency-test-job
rw=randread
filesize=10K

[job1]
filename=device name

And then we use it via

fio fiorandomreadlatency.fio

dmitri-alleo avatar Jun 27 '22 13:06 dmitri-alleo

Can you please provide information about your workloads:

  • Flist used
  • Node id, and network id.

It's known that when using containers flist performance of the rootfs of the container is degraded. Instead data disk needed to be added to the container (rootfs is only to serve your image files). or instead use a VM image.

muhamadazmy avatar Jul 08 '22 07:07 muhamadazmy