AutoMLExperiment_return_current_best_trial_when_ct_is_canceled_with_trial_completed_Async fails in CI

[ericstj]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=976255 Build error leg or test failing: Microsoft.ML.AutoML.Test.AutoMLExperimentTests.AutoMLExperiment_return_current_best_trial_when_ct_is_canceled_with_trial_completed_Async Pull request: https://github.com/dotnet/machinelearning/pull/7414

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "AutoMLExperiment_return_current_best_trial_when_ct_is_canceled_with_trial_completed_Async",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Error:

Starting test: Microsoft.ML.AutoML.Test.AutoMLExperimentTests.AutoMLExperiment_Taxi_Fare_CV_5_Test
    Microsoft.ML.AutoML.Test.AutoMLExperimentTests.AutoMLExperiment_return_current_best_trial_when_ct_is_canceled_with_trial_completed_Async [FAIL]
[31;1m[m[37m      Expected value to be less than or equal to 2500L, but found 3236L (difference of 736).
[m[30;1m      Stack Trace:
[m[37m           at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message)
[m[37m           at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message)
[m[37m           at FluentAssertions.Execution.DefaultAssertionStrategy.HandleFailure(String message)
[m[37m           at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
[m[37m           at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
[m[37m           at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args)
[m[37m           at FluentAssertions.Numeric.NumericAssertions`2.BeLessThanOrEqualTo(T expected, String because, Object[] becauseArgs)
[m[37m           at FluentAssertions.Numeric.NumericAssertions`2.BeLessOrEqualTo(T expected, String because, Object[] becauseArgs)
[m[37m        /__w/1/s/test/Microsoft.ML.AutoML.Tests/AutoMLExperimentTests.cs(146,0): at Microsoft.ML.AutoML.Test.AutoMLExperimentTests.AutoMLExperiment_return_current_best_trial_when_ct_is_canceled_with_trial_completed_Async()
[m[37m        --- End of stack trace from previous location ---

Known issue validation

Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=976255 Error message validated: [AutoMLExperiment_return_current_best_trial_when_ct_is_canceled_with_trial_completed_Async] Result validation: :white_check_mark: Known issue matched with the provided build. Validation performed at: 3/12/2025 1:13:43 AM UTC

Report

Build	Definition	Test	Pull Request
1206945	dotnet/machinelearning	Microsoft.ML.AutoML.Test.AutoFitTests.AutoFitRecommendationTest	dotnet/machinelearning#7556
1201754	dotnet/machinelearning	Microsoft.ML.AutoML.Tests.WorkItemExecution	dotnet/machinelearning#7548

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	2