dotnet
Disable the `Darc_CloneRepo` Maestro scenario test
Problem
This test runs for 9-10 minutes. It clones repos based on Version.Details.xml dependencies and verifies the right set of repos was cloned.
Additionally to this taking quite long, AzDO agents have slow network and cloning takes longer (even for the initial AzDO clone). For these cases, this test times out, like here: https://dev.azure.com/dnceng/internal/_build/results?buildId=2253986&view=logs&j=4d9f00d7-d155-5977-7b70-2e6b79720ff2&t=b6bdfb3b-13bb-5ba7-3230-bffa36e2e1b9&l=65
Potential action steps
- I am not sure this tests some logic that is actually run as Maestro doesn't clone repos in full.
- Even if we want coverage for this, it looks like most of the logic it tests is around parsing
Version.Details.xmland deducing dependencies which could be unit-tested. - I think this particular test is something we might consider improving/skipping. Turning it off would speed up the stage from 50 minutes to 40.
- We can also lower the overall 120 minute timeout on that stage to something like 75 minutes
Additional information
List of all tests and their duration:
| Test | Duration |
|---|---|
| Darc_AzDoFlow_Batched | 2 m 56 s |
| Darc_AzDoFlow_FeedFlow | 1 m 25 s |
| Darc_AzDoFlow_NonBatched | 3 m 53 s |
| Darc_AzDoFlow_NonBatched_AllChecksSuccessful | 1 m 26 s |
| ArcadeBuilds_EndToEnd | 9 s |
| ArcadeChannels_EndToEnd | 16 s |
| Darc_CloneRepo | 8 m 41 s |
| ArcadeChannels_DefaultChannels | 16 s |
| ArcadeDependencies_EndToEnd | 11 s |
| Darc_GitHubFlow_Batched | 2 m 29 s |
| Darc_GitHubFlow_NonBatched | 3 m 32 s |
| Darc_GitHubFlow_NonBatched_WithCoherency | 2 m 24 s |
| Darc_GitHubFlow_AutoMerge_GithubChecks_AllChecksSuccessful | 6 m 25 s |
| Darc_GitHubFlow_AutoMerge_GithubChecks_NoRequestedChanges | 6 m 24 s |
| Darc_GitHubFlow_AutoMerge_GithubChecks_Standard | 6 m 24 s |
| ArcadeRepoPolicies_EndToEnd | 45 s |
| ArcadeSdkUpdate | 1 m 48 s |
| Subscriptions_EndToEnd | 44 s |
@mmitche would you know whether this test is testing logic that we actually care about? Is there some flow that would clone whole repos like that?
It was added here: https://github.com/dotnet/arcade-services/pull/1396
I don't think so. It doesn't appear to be actually exercising much of Maestro/Darc's codepaths. Get rid of it.
