WebApi
WebApi copied to clipboard
Some tests fail when run in full test suite (succeed individually)
When running all of the tests in Microsoft.Test.E2E.AspNet.OData in conjunction, some tests fail on the first try. (If run again individually or as a separate group, they then succeed.)
Assemblies affected
Observed on the latest main branch of OData WebApi (lib v7.5.17).
Reproduce steps
Run all tests in Microsoft.Test.E2E.AspNet.OData.
Expected result
All tests succeed.
Actual result
The following four tests fail in Microsoft.Test.E2E.AspNet.OData.ModelBoundQuerySettings.CombinedTest.CombinedTest:
- DefaultQuerySettings
- DefaultQuerySettings(url: "{0}/enablequery/Orders?$top=3", error: "top")
- DefaultQuerySettings(url: "{0}/modelboundapi/Orders?$top=4", error: "top")
- QuerySettingsOnPropertyNegative
- QuerySettingsOnPropertyNegative(entitySetUrl: "{0}/enablequery/Orders", url: "?$expand=Customers2($expand=Order($top=3))", error: "top")
- QuerySettingsOnPropertyNegative(entitySetUrl: "{0}/modelboundapi/Orders", url: "?$expand=Customers2($expand=Order($top=3))", error: "top")
Failure message (same for all failing tests): Assert.Equal() Failure Expected: BadRequest Actual: OK
Additional detail
If the entire E2E.AspNet.OData test suite is run a second time, these tests continue to fail. However, if the failed tests are run individually or in a smaller test set (such as ModelBoundQuerySettings.CombinedTest), they then succeed.
This could be impacted by potential interdependency between tests.
Repros on commit cda7e833b7e86453fdf41666d1d2e593a70e8176