Add `evaluate_oracle` method to `BenchmarkRunner`

[esantorella]

trafficstars

Summary: Docstring should make this self-explanatory.

In addition to the increased ease of development, enables reaping a lot of LOC in D61415525

Reviewed By: Balandat

Differential Revision: D61431980

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D61431980

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.28%. Comparing base (003983d) to head (e0841db).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2705   +/-   ##
=======================================
  Coverage   95.28%   95.28%           
=======================================
  Files         493      493           
  Lines       47899    47903    +4     
=======================================
+ Hits        45639    45643    +4     
  Misses       2260     2260           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D61431980

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D61431980

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D61431980

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D61431980

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D61431980

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D61431980

[facebook-github-bot]

This pull request has been merged in facebook/Ax@36194d3cb752c388cd838a59c1bde983f9a1189b.