MSTest.Sdk/4.0.1 is packable by default and creates a NuGet package.

[nightowl286]

Describe the bug

After updating a test project from MSTest.Sdk/3.6.4 to MSTest.Sdk/4.0.1 the test project now produces a NuGet package when running dotnet pack on the solution, this used to not be the case.

Steps To Reproduce

1. Create a test project with MSTest.Sdk/4.0.1.
2. Run dotnet pack on the solution.

Expected behavior

No NuGet package is created for the test project, as this is the old (and desirable) behaviour.

Actual behavior

The test project will generate a NuGet package.

Additional context

Manually setting <IsPackable>false</IsPackable> in the test project worked, so it seems that this is a difference in behaviour between 3.6.4 and 4.0.1, but I don't see how it could be a desirable change, therefore I'm assuming it's something that was missed and should be counted as a bug.

[Youssef1313]

Which .NET SDK version are you using currently? This was fixed previously in https://github.com/NuGet/NuGet.Client/pull/6458

[nightowl286]

I noticed this happening while running a Github Action, where I use the actions/setup-dotnet@v5 action to install the SDK, and that seems to have tried installing 9.0.306 (but it was already installed) according to the log.

The issue that you linked seems to suggest that there might be a difference between VSTest and MTP, I'm not sure exactly how that would affect things in my scenario, but this is how my test project looked before I set the IsPackable to false manually:

<Project Sdk="MSTest.Sdk/4.0.1">

	<!-- Project -->
	<PropertyGroup Label="project">
		<TargetFrameworks>net9.0</TargetFrameworks>
		<LangVersion>13</LangVersion>
		<Nullable>enable</Nullable>

		<AssemblyName>OwlDomain.Testing.Base.Tests</AssemblyName>
		<RootNamespace>OwlDomain.Testing.Base.Tests</RootNamespace>

		<UseVSTest>true</UseVSTest>
	</PropertyGroup>

	<!-- Project references -->
	<ItemGroup Label="projects">
		<ProjectReference Include="..\Testing.Base\Testing.Base.csproj" />
	</ItemGroup>

</Project>

The project looked the exact same when I was using MSTest.Sdk/3.6.4, except that it was targeting net8.0, and the actions/setup-dotnet@v5 action tried to install 8.0.20 and 9.0.305, so I'm not sure which one it actually ended up using (the log does not seem to mention it), but I would assume it was using 9.0.305 since that was the latest one.

[Youssef1313]

@nightowl286 Sorry for getting back to you late.

If you set <UseVSTest>true</UseVSTest>, I don't see why would IsPackable not be set to false by default. Can you provide binlogs before and after the MSTest.Sdk update?

[nightowl286]

I'm not able to provide the binlogs (I can't seem to be able to inspect them beforehand, neither on my linux machine nor on the web inspector as it doesn't seem to support the latest version of the binlog format).

However I have created a minimal repro test case available at nightowl286/IsPackableTest, which just contains two C# projects.

<Project Sdk="MSTest.Sdk/4.0.1">
	<PropertyGroup>
		<TargetFrameworks>net9.0</TargetFrameworks>
		<UseVSTest>true</UseVSTest>
	</PropertyGroup>
</Project>

<Project Sdk="MSTest.Sdk/3.6.4">
	<PropertyGroup>
		<TargetFrameworks>net9.0</TargetFrameworks>
		<UseVSTest>true</UseVSTest>
	</PropertyGroup>
</Project>

Running dotnet pack locally (with the SDK version 9.0.107) on each project individually makes the one targeting MSTest.Sdk/4.0.1 produce a NuGet package in the bin folder, but the one targeting MSTest.Sdk/3.6.4 does not.

[Youssef1313]

I'm not able to reproduce.

@nohwnd Can you try on your side to see if you can repro please?

[nohwnd]

I can see the problem I don't know the cause yet, and have to move to something else.

To me it seems that before sdk 4, microsoft.net.test sdk was always included, so IsTestProject was set to true, and this target then set the IsPackable to false.

In new one the target is not imported and so the IsPackable remains true.

I can only see the problem when I build with net 9.0.306 sdk, I don't see it when I build with net10 sdk (I am also on older version so I cannot build net9 target with that, and targetting net10 did not have the issue for me).

binlogs attached, I renamed the directory to avoid noise when looking for IsPackable.

IsPTest.zip

[nohwnd]

this is the working case (look at project id 57)

in non working case we don't have that target, and so isPackable is true:

[Youssef1313]

I'm not sure why I wasn't able to repro, but given the binlogs, this is a VSTest regression. The packaging is wrong.

buildMultiTargeting is intended to be for "outer" builds and must not have TFM-specific folders, otherwise it's not considered.

Pack is reading IsPackable on the outer build.

[nohwnd]

so this one was used, and now it is not used anymore

17.13 versus 18.0

interesting that I cannot repro with net10 sdk though

[Youssef1313]

@nohwnd With net10, the NuGet does an additional check for IsTestingPlatformApplication. Probably the read that NuGet does is done early before we set back IsTestingPlatformApplication to false.

[nightowl286]

I am also on older version so I cannot build net9 target with that, and targetting net10 did not have the issue for me

The problem also occurs if you set the TFM to net8 and use either the .NET 8 or 9 SDK to run dotnet pack (8.0.117 and 9.0.107 in my case), I checked by using a global.json file to set the SDK version.

.NET 7 shouldn't matter since MSTest.SDk/4.0.1 only says it supports .NET 8.0+ on the NuGet readme.