Tooling-Windows-Submodule icon indicating copy to clipboard operation
Tooling-Windows-Submodule copied to clipboard

Add ability to create secondary test projects

Open Sergio0694 opened this issue 2 years ago • 4 comments

Describe the bug

In https://github.com/CommunityToolkit/Labs-Windows/pull/438, I need a second test project referencing the source generator (not the main library), so I can write tests for it as well. This test project should just target .NET (eg. .NET 6 or whatever). Right now there isn't really a way to setup secondary test projects within an experiment. I mean you can create a test project, but it wouldn't run in the CI.

Sergio0694 avatar Jun 01 '23 15:06 Sergio0694

Test discovery is done by the props files here:

https://github.com/CommunityToolkit/Tooling-Windows-Submodule/blob/35c03dff2df6a1423860710e6633faf65cf3e788/ProjectHeads/AllComponents/Tests.Head.AllComponents.props#L5-L9

Which are run as part of the parent repos workflow from the parent test app head:

https://github.com/CommunityToolkit/Windows/blob/744e6aba53329b3f04980237ca4d19eae3fbd5c9/.github/workflows/build.yml#L138-L140

So, theoretically as long as you have another shared test project, you could include those tests, but they'll still get run under UWP or WinUI3 contexts.

We'd really just need a convention for naming or something to pull other independent test projects and have something in the build workflow which can find and run those.

michael-hawker avatar Jun 12 '23 20:06 michael-hawker

Can we not just have projects ending on "Tests"? So you could also have normal .NET test projects, not shared ones.

Sergio0694 avatar Jun 12 '23 20:06 Sergio0694

Can we not just have projects ending on "Tests"? So you could also have normal .NET test projects, not shared ones.

I mean you can add any project you want as it'll be picked up by the slngen command:

https://github.com/CommunityToolkit/Tooling-Windows-Submodule/blob/35c03dff2df6a1423860710e6633faf65cf3e788/ProjectHeads/GenerateSingleSampleHeads.ps1#L85

So you could add just a regular .NET test project in the test folder called whatever and it'll be there when you open the solution for that component.

Though we have a slightly different pattern for the all solution:

https://github.com/CommunityToolkit/Tooling-Windows-Submodule/blob/35c03dff2df6a1423860710e6633faf65cf3e788/GenerateAllSolution.ps1#L65-L68

The main thing is that since the build script is basically handled outside the infrastructure here if it's not part of a project file, then we'd need to somehow figure out how to run the tests as part of the build or add a manual extra step to the build.yml in the end repository for that specific component.

michael-hawker avatar Jun 12 '23 21:06 michael-hawker

Test discovery is done by the props files here:

https://github.com/CommunityToolkit/Tooling-Windows-Submodule/blob/35c03dff2df6a1423860710e6633faf65cf3e788/ProjectHeads/AllComponents/Tests.Head.AllComponents.props#L5-L9

Which are run as part of the parent repos workflow from the parent test app head:

https://github.com/CommunityToolkit/Windows/blob/744e6aba53329b3f04980237ca4d19eae3fbd5c9/.github/workflows/build.yml#L138-L140

So, theoretically as long as you have another shared test project, you could include those tests, but they'll still get run under UWP or WinUI3 contexts.

We'd really just need a convention for naming or something to pull other independent test projects and have something in the build workflow which can find and run those.

We should be able to add support for .csproj alongside existing support for .shproj, enabling the MultiTarget system for test projects.

This will give us the same build features afforded to Sample and Source projects, including generating correctly targeted test heads.

We need a .NET Core test head for the netstandard MultiTarget, since the .shproj test projects currently in use are hardcoded to multitargeting UWP and WinAppSdk via their respective test heads.

Additional notes

  • This will also give us more flexibility when implementing https://github.com/CommunityToolkit/Tooling-Windows-Submodule/issues/67
  • Future: Usage of the MultiTarget system in tests allows per-component opting into or out of test execution on certain platforms. Same flexibility we get with source / sample projects.

Arlodotexe avatar Jun 12 '23 22:06 Arlodotexe