sdk
sdk copied to clipboard
dotnet
Update the sdk target to net10.0
There are a bunch of template test failure error messages as follows:
warning MSB4011: "/Users/runner/work/1/s/artifacts/.nuget/packages/microsoft.dotnet.arcade.sdk/10.0.0-beta.24504.4/tools/Tests.targets" cannot be imported again. It was already imported at "/Users/runner/work/1/s/artifacts/.nuget/packages/microsoft.dotnet.arcade.sdk/10.0.0-beta.24504.4/tools/Imports.targets (12,3)". This is most likely a build authoring error. This subsequent import will be ignored. [/Users/runner/work/1/s/artifacts/tmp/Release/dotnet-new.IntegrationTests/Restore_Basic/20241008062742474/Target/Output/MyProject.csproj]\n/Users/runner/work/1/s/artifacts/.nuget/packages/microsoft.dotnet.arcade.sdk/10.0.0-beta.24504.4/tools/GenerateChecksums.targets(4,71): error MSB4022: The result "" of evaluating the value "$(ArcadeSdkBuildTasksAssembly)" of the "AssemblyFile" attribute in element <UsingTask> is not valid. [/Users/runner/work/1/s/artifacts/tmp/Release/dotnet-new.IntegrationTests/Restore_Basic/20241008062742474/Target/Output/MyProject.csproj]\nStdErr:\nRestore failed.\nPost action failed.\nManual instructions: Run 'dotnet restore'\n
@dotnet/templating-engine-maintainers @dotnet/arcade-contrib Could you please take a look at this issue?
warning MSB4011: "/Users/runner/work/1/s/artifacts/.nuget/packages/microsoft.dotnet.arcade.sdk/10.0.0-beta.24504.4/tools/Tests.targets" cannot be imported again. It was already imported at "/Users/runner/work/1/s/artifacts/.nuget/packages/microsoft.dotnet.arcade.sdk/10.0.0-beta.24504.4/tools/Imports.targets (12,3)". This is most likely a build authoring error. This subsequent import will be ignored. [/Users/runner/work/1/s/artifacts/tmp/Release/dotnet-new.IntegrationTests/Restore_Basic/20241008062742474/Target/Output/MyProject.csproj]\n/Users/runner/work/1/s/artifacts/.nuget/packages/microsoft.dotnet.arcade.sdk/10.0.0-beta.24504.4/tools/GenerateChecksums.targets(4,71): error MSB4022: The result "" of evaluating the value "$(ArcadeSdkBuildTasksAssembly)" of the "AssemblyFile" attribute in element is not valid. [/Users/runner/work/1/s/artifacts/tmp/Release/dotnet-new.IntegrationTests/Restore_Basic/20241008062742474/Target/Output/MyProject.csproj]\nStdErr:\nRestore failed.\nPost action failed.\nManual instructions: Run 'dotnet restore'\n
@mmitche do you recall these errors from https://github.com/dotnet/arcade/issues/12740
net10 templates are blocked
@kasperk81 I think the Arcade issue is a red herring. When the tests are being run, the .NET SDK imports the Directory.Build.targets from the root of the SDK repo, which then imports the arcade targets. I am pretty sure this is not intended for template tests as they aren't really testing the simple create+restore flow. In a helix run, I don't think this ends up happening because the SDK's repo content isn't on the machine, so that's good.
There is a Directory.Build.props file that is set up as part of the test run in the test temp dir. I think there needs to be a corresponding Directory.Build.targets alongside it to avoid accidentally importing Directory.Build..targets from the SDK repo.
/cc @nagilson and @marcpopMSFT
/Users/runner/work/1/s/test/UnitTests.proj(109,5): error MSB3030: Could not copy the file "/Users/runner/work/1/s/artifacts/tmp/Release/NuGet.config" because it was not found. ##[error]test/UnitTests.proj(109,5): error MSB3030: (NETCORE_ENGINEERING_TELEMETRY=Build) Could not copy the file "/Users/runner/work/1/s/artifacts/tmp/Release/NuGet.config" because it was not found.
these are persistent. when is nuget.config gets copied in artifacts?
vmr is https://github.com/dotnet/arcade/pull/15153#issuecomment-2405616601 (need arcade bump, applies to all prs not just this one)
@kasperk81 It's due to this change:
https://github.com/dotnet/sdk/pull/43015/files#diff-37aabd56f54205e41e1a83387bfefad4ff62b60b0ec360e3beb1160d1c635722R20
That causes the targets in src/Tests/Microsoft.NET.TestFramework/SetupTestRoot.targets to not run. I think the conditionals in that file look pretty suspect. I'm guessing they were working around some race condition.
dotnet-format uses NetCurrent right now. I think it should be using SdkTargetFramework
Looks like we'll have to update some dotnet new baselines as the web templates have already updated to support net10 (though a few things look off there). Additionally, we need to update the global.json for source build. I have the changes locally but I figured I'd wait a bit and see if we get any results from the sdk tests
i think new object[] { ToolsetInfo.NextTargetFramework }, lines should be just removed. at no point we can actually use it. not during the whole year or at this time of the year when we are moving from one version to the next
It's possible it may have been better to retarget the templates to net10 before targeting the SDK at net10 per some of the failures. I think that just means more tests we'll have to disable temporarily until the templates support it.
@dotnet/illink Macos failure in Microsoft.NET.Publish.Tests.GivenThatWeWantToPublishAnAotApp.NativeAot_hw_runs_with_no_warnings_when_PublishAot_is_enabled(targetFramework: \"net10.0\")
The command output contained a result it should not have contained: warning
...
Generating native code
ld: warning: __LD,__compact_unwind entries for _RhpNewFast overlap at offset 0x32
ld: warning: __LD,__compact_unwind entries for _RhpNewObject overlap at offset 0x6B
...
A bunch of warnings when generating the native code which the test flags.
@dotnet/product-construction error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/ms/BuildXL/_packaging/BuildXL/nuget/v3/index.json. in the VMR legs.
From helix logs:
The command output contained a result it should not have contained: warning
File Name: /tmp/helix/working/BB8C09A0/p/d/dotnet
Arguments: msbuild /t:Publish /private/tmp/helix/working/BB8C09A0/w/A1EB088D/e/testExecutionDirectory/NativeAot_hw_---788461B1_1/HellowWorldNativeAotApp/HellowWorldNativeAotApp.csproj /restore /p:UseCurrentRuntimeIdentifier=true /p:SelfContained=true /p:CheckEolTargetFramework=false
Exit Code: 0
StdOut:
MSBuild version 17.13.0-preview-24504-04+c4d51a11b for .NET
Determining projects to restore...
Restored /private/tmp/helix/working/BB8C09A0/w/A1EB088D/e/testExecutionDirectory/NativeAot_hw_---788461B1_1/HellowWorldNativeAotApp/HellowWorldNativeAotApp.csproj (in 382 ms).
/tmp/helix/working/BB8C09A0/p/d/sdk/10.0.100-ci/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(326,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/private/tmp/helix/working/BB8C09A0/w/A1EB088D/e/testExecutionDirectory/NativeAot_hw_---788461B1_1/HellowWorldNativeAotApp/HellowWorldNativeAotApp.csproj]
HellowWorldNativeAotApp -> /private/tmp/helix/working/BB8C09A0/w/A1EB088D/e/testExecutionDirectory/NativeAot_hw_---788461B1_1/HellowWorldNativeAotApp/bin/Debug/net10.0/osx-x64/HellowWorldNativeAotApp.dll
Generating native code
ld: warning: __LD,__compact_unwind entries for _RhpNewFast overlap at offset 0x32
ld: warning: __LD,__compact_unwind entries for _RhpNewObject overlap at offset 0x6B
ld: warning: __LD,__compact_unwind entries for _RhNewString overlap at offset 0x61
ld: warning: __LD,__compact_unwind entries for _RhpNewArray overlap at offset 0x6C
ld: warning: __LD,__compact_unwind entries for _RhpNewArrayRare overlap at offset 0x6F
ld: warning: __LD,__compact_unwind entries for _RhpThrowHwEx overlap at offset 0x6B
ld: warning: __LD,__compact_unwind entries for _RhpThrowEx overlap at offset 0x9F
ld: warning: __LD,__compact_unwind entries for _RhpRethrow overlap at offset 0x65
ld: warning: __LD,__compact_unwind entries for _RhpCallCatchFunclet overlap at offset 0xC0
ld: warning: __LD,__compact_unwind entries for _RhpCallFinallyFunclet overlap at offset 0xAB
ld: warning: __LD,__compact_unwind entries for _RhpCallFilterFunclet overlap at offset 0x32
ld: warning: __LD,__compact_unwind entries for _RhpCallPropagateExceptionCallback overlap at offset 0xCA
ld: warning: __LD,__compact_unwind entries for _RhpWaitForGC overlap at offset 0x96
ld: warning: __LD,__compact_unwind entries for _RhpGcPollRare overlap at offset 0x43
ld: warning: __LD,__compact_unwind entries for _RhpStackProbe overlap at offset 0x1F
ld: warning: __LD,__compact_unwind entries for _RhpPInvoke overlap at offset 0x33
ld: warning: __LD,__compact_unwind entries for _RhpUniversalTransition overlap at offset 0xDE
ld: warning: __LD,__compact_unwind entries for RhpUniversalTransition_DebugStepTailCall overlap at offset 0xDE
HellowWorldNativeAotApp -> /private/tmp/helix/working/BB8C09A0/w/A1EB088D/e/testExecutionDirectory/NativeAot_hw---788461B1_1/HellowWorldNativeAotApp/bin/Debug/net10.0/osx-x64/publish/
@filipnavara these warnings are showing up on osx-x64 13. Do they look familiar?
@filipnavara these warnings are showing up on osx-x64 13. Do they look familiar?
Unfortunately no, they don't. It's the native assembly code and it's coming from this macro: https://github.com/dotnet/runtime/blob/6d23f64952d0f242c4dcc7cbb230a619be930309/src/coreclr/nativeaot/Runtime/unix/unixasmmacrosamd64.inc#L19-L27. Weirdly we don't emit the same block on arm64, where it would equally make sense to ensure that LLVM or LD64 doesn't try to convert the DWARF unwinding codes into the compact unwinding code (which is buggy for non-standard unwind sequences).
I haven't done osx-x64 builds for a while. Do we know what Xcode version is on the machine? (should be the clang version in the log I guess)
Do we know what Xcode version is on the machine? (should be the clang version in the log I guess)
I think the reason why it is failing in this PR and not main branch is due to some changes between 9.0 rc2 and 10.0 alpha1. Which means both runtime and SDK repos are now on LKG testing plan, so even the unintentional breaking changes can make it to SDK release unchecked. The integration tests will only spot the issues when global.json / SDK is updated in either runtime or SDK repos. cc @dotnet/ilc-contrib
@dotnet/sdk-container-builds-maintainers there are still a few container tests failing. Most are from the rate limiting so rerunning would probably be sufficient. Not sure about Microsoft.NET.Build.Containers.IntegrationTests.EndToEndTests.EndToEnd_NoAPI_ProjectType(projectType: \"webapi\", addPackageReference: True) Error: No such container: c0c2cdf623077d9341b7ed228db6f2cadef3bbf216370e935be732ba47129e2e
We can likely disregard these __unwind warnings: https://github.com/dotnet/sdk/compare/main...am11:sdk:ignore-ld-warning, unless someone is particularly interested in investigating it further and going down the rabbit hole.
Justification:
- The
osx-x64platform is gradually becoming less relevant: https://github.com/dotnet/runtime/issues/108570 - The older version of
clangonosx-x64is even less of a priority. - These warnings (as far as we can tell) don't have any observable impact on functionality, exception handling or debugging.
@filipnavara, @MichalStrehovsky what do you guys think?
This PR and not main branch is due to some changes between 9.0 rc2 and 10.0 alpha1.
What is the change between 9.0 rc2 and 10.0 alpha1 that introduced this? It would be useful to connect the dots. The best fix would be to revert the offending change if possible.
Which means both runtime and SDK repos are now on LKG testing plan
I am not sure what you mean. The LKG changes in dotnet/runtime only apply to the toolset used for building the compilers. The tests are all live bits. I think we have not seen this in dotnet/runtime due to different XCode versions used for testing or not checking for XCode warnings when running the tests.
I think we haven’t seen this in
dotnet/runtimedue to different Xcode versions used for testing or not checking for Xcode warnings when running tests.
Yup, since several dotnet/runtime to dotnet/sdk codeflow PRs were merged between rc2 and this PR, it shows the failing test isn’t using ILCompiler from packages being updated, but rather the version bundled with SDK specified in global.json. As you can see that this PR is only updating global.json related to the failing test. We skipped over RTM/GA (this PR jumps from rc2 to alpha1), so these tests haven’t run with RTM. I think this is unintentional and a concerning gap.
What’s the change between 9.0 rc2 and 10.0 alpha1?
Locally, with Xcode 16, I only get one warning (which will also fail this sensitive test—test expects zero warnings on stdout and stderr):
ld: warning: -ld_classic is deprecated and will be removed in a future release
I’m downloading Xcode 13 to see if I can reproduce this warning locally.
Since toolchain versions aren’t printed in Helix runs, I think runtime uses AppleClang 14, and SDK uses v13 on osx.13. We can align helix queues between runtime and SDK: OSX.1200.Amd64.Open for PRs and OSX.1200.Amd64 for official builds:
https://github.com/dotnet/sdk/blob/1c88a979b7109d83e5317b0e374c8d13ef2b3b68/.vsts-pr.yml#L65
the failing test isn’t using ILCompiler from packages being updated, but rather the version bundled with SDK specified in global.json.
I don't think that's correct. The tests are using ILCompiler package 10.0.0-alpha.1.24507.22. This version is specified in https://github.com/dotnet/sdk/blob/main/eng/Versions.props.
I think that we are seeing this failure in this PR since this PR is enabling the tests for .NET 10 TFM.
You can run runfo get-helix-payload -j f20fa4a6-963c-4538-80f5-dd163343cca5 -w 1ed3f53c-bf12-4be1-8b68-1ab87b11983b -o c:\helix_payload to get the exact bits that are downloaded to the test machine to tell what's going on.
The tests are using ILCompiler package 10.0.0-alpha.1.24507.22. This version is specified in main/eng/Versions.props.
This failing test https://github.com/dotnet/sdk/blob/1c88a979b7109d83e5317b0e374c8d13ef2b3b68/test/Microsoft.NET.Publish.Tests/GivenThatWeWantToPublishAnAotApp.cs#L24-L78 is using SDK bundled version. I think the example you are referring to would look like https://github.com/dotnet/sdk/blob/1c88a979b7109d83e5317b0e374c8d13ef2b3b68/test/Microsoft.NET.Publish.Tests/GivenThatWeWantToPublishAnAotApp.cs#L267 but the failing test is not using PackageReference.
Yes, the test is using the 10.0.0-alpha.1.24507.22 ILCompiler version embedded into the 10.0.100-ci live-built SDK.
It has:
GetKnownILCompilerPackVersion(testAsset, targetFramework, out string expectedVersion);
CheckIlcVersions(testAsset, targetFramework, rid, expectedVersion);
it is explicitly checking for known ILCompiler pack which my understanding is the bundled version.
Passing on main, failing in PR that is modifying global.json. Other tests in the same file are not affected.
@jkoritzinsky We have picked up more failures in this PR that seems to be caused by https://github.com/dotnet/runtime/pull/107889 . Could you please take a look?
From https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-sdk-refs-pull-43015-merge-5284031d0a3746c894/Microsoft.NET.Sdk.Web.Tests.dll/3/console.03fc7d48.log?skoid=8eda00af-b5ec-4be9-b69b-0919a2338892:
[m[37m Generating native code
[m[37m /usr/bin/ld.bfd: /usr/bin/ld.bfd: DWARF error: invalid or unhandled FORM value: 0x23
[m[37m /datadisks/disk1/work/997E08B4/w/A9900972/e/testExecutionDirectory/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/10.0.0-alpha.1.24523.5/sdk/libeventpipe-enabled.a(unity_0_cxx.cxx.o): in function `ds_dump_protocol_helper_handle_ipc_message(_DiagnosticsIpcMessage*, _DiagnosticsIpcStream*)':
[m[37m unity_0_cxx.cxx:(.text._Z42ds_dump_protocol_helper_handle_ipc_messageP22_DiagnosticsIpcMessageP21_DiagnosticsIpcStream+0x2ed): undefined reference to `minipal_get_length_utf16_to_utf8'
[m[37m /usr/bin/ld.bfd: unity_0_cxx.cxx:(.text._Z42ds_dump_protocol_helper_handle_ipc_messageP22_DiagnosticsIpcMessageP21_DiagnosticsIpcStream+0x325): undefined reference to `minipal_convert_utf16_to_utf8'
[m[37m /usr/bin/ld.bfd: unity_0_cxx.cxx:(.text._Z42ds_dump_protocol_helper_handle_ipc_messageP22_DiagnosticsIpcMessageP21_DiagnosticsIpcStream+0x47e): undefined reference to `minipal_get_length_utf8_to_utf16'
[m[37m /usr/bin/ld.bfd: unity_0_cxx.cxx:(.text._Z42ds_dump_protocol_helper_handle_ipc_messageP22_DiagnosticsIpcMessageP21_DiagnosticsIpcStream+0x4c5): undefined reference to `minipal_convert_utf8_to_utf16'
...
Looks like we now have a circular dependency between the runtime lib and the eventpipe lib by putting the minipal in the runtime lib. I think the easiest solution here (other than adding the flags to do a second linker pass, which I think we want to avoid), is to ship the minipal as a separate static lib in the package for NativeAOT. I'll work on getting that working.
it is explicitly checking for known ILCompiler pack which my understanding is the bundled version.
It is the bundled version in the live build of the SDK. It is independent on the version in global.json.
What is the change between 9.0 rc2 and 10.0 alpha1 that introduced this?
Here is what I found: These warnings show up when dotnet/runtime compiled using Apple clang 15+ is used with classic linker (either Apple clang 14 or clang 15+ with -ld_classic) for PublishAot. The warnings got introduced as side-effect of https://github.com/dotnet/runtime/pull/107871 that updated Apple clang version used to build dotnet/runtime main branch from 14 to 15.
It is the bundled version in the live build of the SDK. It is independent on the version in global.json.
Right, https://github.com/dotnet/runtime/pull/107871 and https://github.com/dotnet/runtime/pull/107889 are already part of the daily build releases: https://github.com/dotnet/sdk/blob/main/documentation/package-table.md (10.0.x) and they did not fail coherency updates PR. We can already build projects with <TargetFramework>net10.0. If these were bugs, they weren't caught during the coherency update.
If these were bugs, they weren't caught during the coherency update.
Right, that's expected given how things are setup in this repo. There are no publish tests running against .NET 10 TFM in dotnet/sdk main. Ingesting .NET 10 runtime packages into sdk have not automatically enabled all tests to run against .NET 10 TFM. Changes in this PR (ToolsetInfo.CurrentTargetFramework) are enabling the publish tests to run against .NET 10 TFM and that's why we are seeing the failures.
We can likely disregard these __unwind warnings: https://github.com/dotnet/sdk/compare/main...am11:sdk:ignore-ld-warning
@am11 Thank you for proposing a fix. I have merged your change into this PR and added some explanatory comments.