cbt icon indicating copy to clipboard operation
cbt copied to clipboard

Allow time within a test to override global time setting in YAML and allow multiple mixed read/write workloads within a YAML

Open lee-j-sanders opened this issue 10 months ago • 5 comments

This PR only changes the functionality of workloads in CBT.

This PR adds two pieces of functionality:

1) Workloads with randrw only let you specify 1 mixed workload within a YAML
   for a given block size,  If you wanted to do 64k 7030 and 64k 3070, you couldn't, as
   the same directory name was used for subsequent randrw workloads.
   Thus overwriting all the previous resuilts in the archive directory with
   the later tests. This commit adds the readwrite ratio into the directory
   name if the mode is randrw thus creating a unique directory name per test for
  each block size.

  The directory name for 100% read and 100% write tests are unaffected.

2) Prior to this PR, all workloads inherited "time" from the
   outer/global part of the YAML. This meant you couldn't set different
   time for each test within the "workloads". Typically for
   preconditioning you'd want to precondition for a set amount of time (
   600 seconds - 10minutes), then set the workload time to be 120 seconds (2 minutes).

By default, if you do not set a "time" within the workload, the time for that specific test will be inherited from the librbdfio "time" within the YAML.

Here is an example of usage:

 librbdfio:
    time: 10
    ramp: 10
    time_based: True
    norandommap: True
    vol_size: 1000
    use_existing_volumes: True
    procs_per_volume: [1]
    volumes_per_client: [8]
    osd_ra: [4096]
    cmd_path: '/usr/local/bin/fio'
    poolname: 'rbd_replicated'
    log_iops: True
    log_bw:  True
    log_lat: True
    fio_out_format: 'json'
    log_avg_msec: 100
    prefill:
      blocksize: '64k'
      numjobs: 1

    # Each block below uses its own local options during its execution
    workloads:
      precondition:
        jobname: 'precond1rw'
        mode: 'randwrite'
        op_size: 65536
        numjobs: [ 1 ]
        total_iodepth: [ 16 ]
        monitor: False # whether to run the monitors along the test
        time: 8
      seq32kwrite:
        jobname: 'seqwrite'
        mode: 'write'
        op_size: 32768
        numjobs: [ 1 ]
        total_iodepth: [ 1, 2, 3, 4, 5, 8, 16, 24, 32, 64, 128 ]
      16k7030:
        jobname: 'randmix'
        mode: 'randrw'
        op_size: 16384
        rwmixread: 70
        numjobs: [ 1 ]
        total_iodepth: [ 1, 2, 3, 4, 5, 8, 16, 24, 32, 64, 128 ]
        time: 5

precondition will use 8 seconds for the single test with queue depth 16. seq32kwrite will use the default of 10 seconds for all tests in total_iodepth[] specified in the librbdfio global section. 16k7030 will use 5 seconds for all tests in total_iodepth[].

lee-j-sanders avatar Feb 17 '25 16:02 lee-j-sanders