azure-sdk-for-net icon indicating copy to clipboard operation
azure-sdk-for-net copied to clipboard

Split test dependencies across test jobs.

Open sima-zhu opened this issue 3 years ago • 10 comments

Contributing to the Azure SDK

Resolve https://github.com/Azure/azure-sdk-tools/issues/2951 Here is the core testing pipeline: https://dev.azure.com/azure-sdk/public/_build/results?buildId=1735434&view=logs&j=76992fd1-2312-5fae-7c45-7baba86b87ae&t=76992fd1-2312-5fae-7c45-7baba86b87ae Here is the storage testing pipeline: https://dev.azure.com/azure-sdk/public/_build/results?buildId=1762139&view=results

@weshaggard Live tests run the target service only and the pipeline is not ready for the other tests. There are some test resources auth issues. https://dev.azure.com/azure-sdk/internal/_build/results?buildId=1769260&view=logs&j=9b84a2c2-369f-5cfc-6757-a840b36086a8&t=3d63f680-b278-5188-126f-c4fc35c087e2

If we want to test dependencies in live tests as well, we'd better to have a follow up PR.

Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.

For specific information about pull request etiquette and best practices, see this section.

sima-zhu avatar Jul 18 '22 17:07 sima-zhu

This pull request is protected by Check Enforcer. For more information about how to run a pipeline against this pull request, see this.

check-enforcer[bot] avatar Jul 18 '22 18:07 check-enforcer[bot]

As we discussed offline, let's think about and plan for follow-on optimizations for when we expand this sharding out beyond NET core tests. Otherwise I think things are 99% in a great state here, so no reason to block the current functionality on approval. This is awesome!

benbp avatar Aug 05 '22 20:08 benbp

@sima-zhu can you please add a link to the issue you are addressing with this PR? Also I think it will be helpful to add some documentation in a MD file so we understand what this is for and how someone might use it.

@JoshLove-msft @christothes @m-nash for your context this is the work to split up the core tests across multiple jobs to help parallelize running all the dependent tests. @ArthurMa1978 @m-nash we should use this same approach for the resource manager core pipeline to help it run more efficient.

weshaggard avatar Aug 05 '22 20:08 weshaggard

/azp run net - core - tests

weshaggard avatar Aug 05 '22 20:08 weshaggard

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines[bot] avatar Aug 05 '22 20:08 azure-pipelines[bot]

@sima-zhu lets also make sure the live test pipelines can benefit from this update.

weshaggard avatar Aug 05 '22 20:08 weshaggard

/azp run net - core - tests

sima-zhu avatar Aug 05 '22 21:08 sima-zhu

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines[bot] avatar Aug 05 '22 21:08 azure-pipelines[bot]

/azp run net - core - tests

sima-zhu avatar Aug 05 '22 21:08 sima-zhu

Azure Pipelines failed to run 1 pipeline(s).

azure-pipelines[bot] avatar Aug 05 '22 21:08 azure-pipelines[bot]

LGTM

m-nash avatar Aug 11 '22 18:08 m-nash

/check-enforcer override

sima-zhu avatar Aug 15 '22 16:08 sima-zhu

/check-enforcer override

sima-zhu avatar Aug 15 '22 17:08 sima-zhu

/check-enforcer reset

sima-zhu avatar Aug 15 '22 17:08 sima-zhu

The msbuild restore failed once we enable to the split.

image https://dev.azure.com/azure-sdk/public/_build/results?buildId=1781872&view=logs&j=163c4b53-87cb-5052-8574-b52a51504003&t=ab588958-e161-539b-389a-ed0959fe3e4c

sima-zhu avatar Aug 15 '22 17:08 sima-zhu