Ax icon indicating copy to clipboard operation
Ax copied to clipboard

Fix progression normalization logic in early-stopping strategies

Open ltiao opened this issue 1 month ago • 2 comments

Summary: This diff fixes the progression normalization logic in early stopping strategies to use proper min-max normalization instead of dividing by the absolute maximum value.

The main changes include:

  • Corrected normalization logic: Replaced the previous implementation (dividing by abs().max()) with _maybe_normalize_map_key(), which properly performs min-max normalization: (x - min) / (max - min). This ensures progression values are correctly mapped to the [0, 1] range regardless of the original scale.
  • Updated documentation: Improved docstrings for the normalize_progressions parameter across BaseEarlyStoppingStrategy, BayesianEarlyStoppingStrategy, and PercentileEarlyStoppingStrategy to accurately describe the min-max normalization behavior.
  • Added dependency: Added //ax/adapter:data_utils to BUCK file to use the _maybe_normalize_map_key utility function.

The previous normalization approach of dividing by the max was incorrect because:

  1. It didn't guarantee a [0, 1] range when the minimum wasn't 0
  2. It was inconsistent with the documented behavior
  3. It can't handle negative progression values

The new approach uses standard min-max scaling which correctly handles arbitrary progression ranges and ensures the transformed values are in [0, 1].

Differential Revision: D86770422

ltiao avatar Nov 11 '25 20:11 ltiao

@ltiao has exported this pull request. If you are a Meta employee, you can view the originating Diff in D86770422.

meta-codesync[bot] avatar Nov 11 '25 20:11 meta-codesync[bot]

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 96.54%. Comparing base (b092b4d) to head (26cff6e). :warning: Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4525   +/-   ##
=======================================
  Coverage   96.53%   96.54%           
=======================================
  Files         561      561           
  Lines       58200    58248   +48     
=======================================
+ Hits        56186    56236   +50     
+ Misses       2014     2012    -2     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov-commenter avatar Nov 11 '25 20:11 codecov-commenter

This pull request has been merged in facebook/Ax@c57e6a8c981d23b1f9fbf369d19a4943c3227761.

meta-codesync[bot] avatar Dec 10 '25 00:12 meta-codesync[bot]