facebook
Inference trace and Best Point Recommendation (BPR) bugfix
Summary: This diff addresses two issues in the computation of inference trace:
-
The generation strategy is copied inside run_optimization_with_orchestrator --> we retrieve the traces on an unused generation strategy --> get_best_point defaults to the best raw observation on ALL obserations
-
Relevant data not filtered in the fallback option for get_best_parameters_from_model_predictions_with_trial_index
Both of these individually lead to the inference trace being incorrect - the first to the best raw value of ALL trials, the second to the best predicted across ALL trials.
Changes:
- Moved copying of generation strategy
- Added argument use_model_only_if_good to force model-based BPR even if model fit is bad
- Clearer sequencing in get_best_parameters_from_model_predictions_with_trial_index
Differential Revision: D80019803
Codecov Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 96.07%. Comparing base (8c7cff6) to head (ef65cae).
Additional details and impacted files
@@ Coverage Diff @@
## main #4128 +/- ##
=======================================
Coverage 96.06% 96.07%
=======================================
Files 566 566
Lines 57208 57230 +22
=======================================
+ Hits 54955 54981 +26
+ Misses 2253 2249 -4
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
- :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
