inference icon indicating copy to clipboard operation
inference copied to clipboard

Add automotive benchmarking setting

Open pgmpablo157321 opened this issue 10 months ago • 1 comments

  • Add server_constant_gen parameter to the test_settings
  • Add server_constant_gen parameter to the python API
  • Let server_constant_gen be able to be loaded from the config files (mlperf.conf and user.conf) using FromConfig
  • For the server scenario, the query scheduling distribution from poisson to constant when the parameter server_constant_gen is set to 1, i.e:
    • When server_constant_gen = 0 number of queries in a fixed time follows a poisson distribution (with mean qps) and time between queries follows an exponential distribution (with mean 1/qps).
    • When server_constant_gen = 1 time between queries is constant equal to 1/qps
  • Custom Sample Grouping
  • In order to run the a benchmark in a grouped setting:
    • Construct a QSL using ConstructGroupedQSL
      • Pass a vector that contains the group sizes in the order they appear in the dataset
    • Start running the test using StartTestWithGroupedQSL
    • Set the variable use_grouped_qsl in the mlperf.conf
    • Run your benchmark in the server scenario

pgmpablo157321 avatar Feb 26 '25 03:02 pgmpablo157321

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

github-actions[bot] avatar Feb 26 '25 03:02 github-actions[bot]