OpenROAD-flow-scripts icon indicating copy to clipboard operation
OpenROAD-flow-scripts copied to clipboard

Published 20 hours ago • verified publisher icon The-OpenROAD-Project

[Autotuner] WIP: Regression test 1

Open luarss opened this issue 5 months ago • 0 comments

This PR currently aims to implement regression for the default usage (python distributed.py --design <> --platform <> --config <> tune --samples <>). Future works can benchmark other algorithms

Explanation

  • The QoR benchmark are obtained by running 100 trials of each Ray version-platform-design combination. The value in the tests are the highest 1st quartile QoR for each combination.
  • Ray version: 2.7, 2.8, 2.9
  • Platform: asap7, sky130hd, ihpsg13g2
  • Design: gcd, aes, ibex
  • Criteria for passing each regression is that the QoR must be within ±25% of the baseline

Note

  • This regression should be activated only when AT/related files are touched.
  • When tools/Autotuner/requirements.txt is changed
  • When autotuner.json is changed -> test on that design
  • When distributed.py is changed.
  • Also track Makefile. Things that can potentially alter flow results

Tasks

  • [ ] Need to recalibrate with new changes

luarss avatar Sep 07 '24 12:09 luarss