BenchmarkDotNet
BenchmarkDotNet copied to clipboard
dotnet
Cannot build NativeAOT 8.0 on Arch Linux
I wrote a simple benchmark, with NativeAOT enabled:
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Running;
BenchmarkRunner.Run<MyBench>();
[SimpleJob(RuntimeMoniker.NativeAot80)]
public class MyBench
{
[Benchmark]
public void Bench() { }
}
But when I run dotnet run -c Release, it fails with:
...
// Build Error: Standard output:
Standard error:
MSBuild version 17.8.5+b5265ef37 for .NET
$HOME/.nuget/packages/microsoft.net.illink.tasks/8.0.3/build/Microsoft.NET.ILLink.targets(210,5): warning : Property 'TrimmerDefaultAction' is deprecated in .NET 7 and higher and will be ignored. Use TrimMode instead. [<PROJECT_DIR>/bin/Release/net8.0/136c2e4d-c592-4318-baf9-67ab64a593fc/BenchmarkDotNet.Autogenerated.csproj]
Generating native code
EXEC : error : Unrecognized instruction set avx-512f [<PROJECT_DIR>/bin/Release/net8.0/136c2e4d-c592-4318-baf9-67ab64a593fc/BenchmarkDotNet.Autogenerated.csproj]
System.CommandLine.CommandLineException: Unrecognized instruction set avx-512f
at System.CommandLine.Helpers.ConfigureInstructionSetSupport(String, Int32, Boolean, TargetArchitecture, TargetOS, String, String, Logger, Boolean) + 0x9d5
at ILCompiler.Program.Run() + 0x59b
at ILCompiler.ILCompilerRootCommand.<>c__DisplayClass227_0.<.ctor>b__0(ParseResult result) + 0x310
$HOME/.nuget/packages/microsoft.dotnet.ilcompiler/8.0.3/build/Microsoft.NETCore.Native.targets(308,5): error MSB3073: The command ""$HOME/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/8.0.3/tools/ilc" @"obj/Release/net8.0/linux-x64/native/136c2e4d-c592-4318-baf9-67ab64a593fc.ilc.rsp"" exited with code 1. [<PROJECT_DIR>/bin/Release/net8.0/136c2e4d-c592-4318-baf9-67ab64a593fc/BenchmarkDotNet.Autogenerated.csproj]
// BenchmarkDotNet has failed to build the auto-generated boilerplate code.
// It can be found in <PROJECT_DIR>/bin/Release/net8.0/136c2e4d-c592-4318-baf9-67ab64a593fc
// Please follow the troubleshooting guide: https://benchmarkdotnet.org/articles/guides/troubleshooting.html
...
My dotnet --info output:
.NET SDK:
Version: 8.0.103
Commit: 6a90b4b4bc
Workload version: 8.0.100-manifests.e99a2be4
Runtime Environment:
OS Name: arch
OS Version:
OS Platform: Linux
RID: arch-x64
Base Path: /usr/share/dotnet/sdk/8.0.103/
.NET workloads installed:
Workload version: 8.0.100-manifests.e99a2be4
There are no installed workloads to display.
Host:
Version: 8.0.3
Architecture: x64
Commit: 9f4b1f5d66
.NET SDKs installed:
8.0.103 [/usr/share/dotnet/sdk]
.NET runtimes installed:
Microsoft.AspNetCore.App 8.0.3 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 8.0.3 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
Other architectures found:
None
Environment variables:
Not set
global.json file:
Not found
Learn more:
https://aka.ms/dotnet/info
Download .NET:
https://aka.ms/dotnet/download
It was installed from Arch Linux repository.
And my .csproj file:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="BenchmarkDotNet" Version="0.13.12" />
</ItemGroup>
</Project>
avx512f is supported on my CPU:
LANG=C lscpu | grep 512
Flags: ... invpcid avx512f avx512dq rdseed adx smap avx512ifma ...
If I remove all avx-512* items from <PROJECT_DIR>/bin/Release/net8.0/<UUID>/BenchmarkDotNet.Autogenerated.csproj, it just compiles.
Hello @Jisu-Woniu
Could you please go to <PROJECT_DIR>/bin/Release/net8.0/136c2e4d-c592-4318-baf9-67ab64a593fc/BenchmarkDotNet.Autogenerated.csproj and share with us what value was used for the <IlcInstructionSet>$this</IlcInstructionSet> element?
It should be just native and in theory should just work (https://github.com/dotnet/BenchmarkDotNet/pull/2464). I just need to check if it's BenchmarkDotNet or NativeAOT bug.
I build it again and this is the content of bin/Release/net8.0/f58553a8-0c73-499a-98cc-5ad358f5fe53/BenchmarkDotNet.Autogenerated.csproj:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<ImportDirectoryBuildProps>false</ImportDirectoryBuildProps>
<ImportDirectoryBuildTargets>false</ImportDirectoryBuildTargets>
<OutputType>Exe</OutputType>
<TargetFramework>net8.0</TargetFramework>
<RuntimeIdentifier>linux-x64</RuntimeIdentifier>
<RuntimeFrameworkVersion></RuntimeFrameworkVersion>
<AssemblyName>f58553a8-0c73-499a-98cc-5ad358f5fe53</AssemblyName>
<AssemblyTitle>f58553a8-0c73-499a-98cc-5ad358f5fe53</AssemblyTitle>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<PlatformTarget>x64</PlatformTarget>
<TreatWarningsAsErrors>False</TreatWarningsAsErrors>
<DebugSymbols>false</DebugSymbols>
<UseSharedCompilation>false</UseSharedCompilation>
<Deterministic>true</Deterministic>
<RunAnalyzers>false</RunAnalyzers>
<PublishAot Condition=" '$(TargetFramework)' != 'net6.0' ">true</PublishAot>
<IlcOptimizationPreference>Speed</IlcOptimizationPreference>
<TrimMode>link</TrimMode><TrimmerDefaultAction>link</TrimmerDefaultAction>
<IlcGenerateCompleteTypeMetadata>True</IlcGenerateCompleteTypeMetadata>
<IlcGenerateStackTraceData>True</IlcGenerateStackTraceData>
<EnsureNETCoreAppRuntime>false</EnsureNETCoreAppRuntime> <!-- workaround for 'This runtime may not be supported by.NET Core.' error -->
<ErrorOnDuplicatePublishOutputFiles>false</ErrorOnDuplicatePublishOutputFiles> <!-- workaround for 'Found multiple publish output files with the same relative path.' error -->
<ValidateExecutableReferencesMatchSelfContained>false</ValidateExecutableReferencesMatchSelfContained>
<IlcInstructionSet>base,sse,sse2,sse3,ssse3,sse4.1,sse4.2,avx,avx2,avx-512f,avx-512bw,avx-512cd,avx-512dq,avx-512vbmi,aes,bmi,bmi2,fma,lzcnt,pclmul,popcnt</IlcInstructionSet>
</PropertyGroup>
<PropertyGroup>
<ServerGarbageCollection>false</ServerGarbageCollection>
<ConcurrentGarbageCollection>true</ConcurrentGarbageCollection>
</PropertyGroup>
<ItemGroup>
<Compile Include="f58553a8-0c73-499a-98cc-5ad358f5fe53.notcs" Exclude="bin\**;obj\**;**\*.xproj;packages\**" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="<PROJECT_DIR>/Bench.csproj" />
</ItemGroup>
<ItemGroup>
<RdXmlFile Include="bdn_generated.rd.xml" />
</ItemGroup>
</Project>
Maybe because the Instruction set should be avx512** instead of avx-512**?
I could set the IlcInstructionSet field to native using the following code:
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Running;
using BenchmarkDotNet.Toolchains.NativeAot;
ManualConfig config = DefaultConfig.Instance.AddJob(
Job.Default.WithToolchain(
NativeAotToolchain.CreateBuilder().UseNuGet().IlcInstructionSet("native").ToToolchain()
)
);
BenchmarkRunner.Run<MyBench>(config);
// Use custom runtime instead of the built-in one:
// [SimpleJob(RuntimeMoniker.NativeAot80)]
public class MyBench
{
[Benchmark]
public void Bench() { }
}
Generated .csproj file:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<ImportDirectoryBuildProps>false</ImportDirectoryBuildProps>
<ImportDirectoryBuildTargets>false</ImportDirectoryBuildTargets>
<OutputType>Exe</OutputType>
<TargetFramework>net8.0</TargetFramework>
<RuntimeIdentifier>linux-x64</RuntimeIdentifier>
<RuntimeFrameworkVersion></RuntimeFrameworkVersion>
<AssemblyName>6bb876b0-44dd-4195-9707-766e8b87018a</AssemblyName>
<AssemblyTitle>6bb876b0-44dd-4195-9707-766e8b87018a</AssemblyTitle>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<PlatformTarget>x64</PlatformTarget>
<TreatWarningsAsErrors>False</TreatWarningsAsErrors>
<DebugSymbols>false</DebugSymbols>
<UseSharedCompilation>false</UseSharedCompilation>
<Deterministic>true</Deterministic>
<RunAnalyzers>false</RunAnalyzers>
<PublishAot Condition=" '$(TargetFramework)' != 'net6.0' ">true</PublishAot>
<IlcOptimizationPreference>Speed</IlcOptimizationPreference>
<TrimMode>link</TrimMode><TrimmerDefaultAction>link</TrimmerDefaultAction>
<IlcGenerateCompleteTypeMetadata>True</IlcGenerateCompleteTypeMetadata>
<IlcGenerateStackTraceData>True</IlcGenerateStackTraceData>
<EnsureNETCoreAppRuntime>false</EnsureNETCoreAppRuntime> <!-- workaround for 'This runtime may not be supported by.NET Core.' error -->
<ErrorOnDuplicatePublishOutputFiles>false</ErrorOnDuplicatePublishOutputFiles> <!-- workaround for 'Found multiple publish output files with the same relative path.' error -->
<ValidateExecutableReferencesMatchSelfContained>false</ValidateExecutableReferencesMatchSelfContained>
<IlcInstructionSet>native</IlcInstructionSet>
</PropertyGroup>
<PropertyGroup>
<ServerGarbageCollection>false</ServerGarbageCollection>
<ConcurrentGarbageCollection>true</ConcurrentGarbageCollection>
</PropertyGroup>
<ItemGroup>
<Compile Include="6bb876b0-44dd-4195-9707-766e8b87018a.notcs" Exclude="bin\**;obj\**;**\*.xproj;packages\**" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="<PROJECT_DIR>/Bench.csproj" />
</ItemGroup>
<ItemGroup>
<RdXmlFile Include="bdn_generated.rd.xml" />
</ItemGroup>
</Project>
It can compile and produce correct result.
It can compile and produce correct result.
Big thanks for letting me know!
There must be some kind of bug here:
https://github.com/dotnet/BenchmarkDotNet/blob/4ab69be430b74469359d70d4557d2ed039e661ce/src/BenchmarkDotNet/Toolchains/NativeAot/Generator.cs#L231-L234
I'll try to fix it after the weekend, I am glad you are unblocked for now.
And maybe these should be replaced with avx512** too.
https://github.com/dotnet/BenchmarkDotNet/blob/4ab69be430b74469359d70d4557d2ed039e661ce/src%2FBenchmarkDotNet%2FToolchains%2FNativeAot%2FGenerator.cs#L251-L255
I am seeing this issue on Windows as well, does not seem to be Linux-specific.
Same issue on my Windows PC after switching from a Intel Core I7 to an AMD Ryzen 5.
Same issue on my Windows PC after switching from a Intel Core I7 to an AMD Ryzen 5.
@eNeRGy164 could you please attach the debugger and see what is happening here:
https://github.com/dotnet/BenchmarkDotNet/blob/4ab69be430b74469359d70d4557d2ed039e661ce/src/BenchmarkDotNet/Toolchains/NativeAot/Generator.cs#L231-L234
@adamsitnik I have the same Problem on a minimal Project. Actually the GettingStarted Example.
This is whats happening in the GetCurrentProcessInstructionSets-method.
I did flag the Class with [SimpleJob(RuntimeMoniker.NativeAot80)]
Platform: X64 TargetFrameworkMoniker: "net8.0" runtimeMoniker: Net80
If I manually set runtimeMoiniker to NativeAot80 the test seams to work.
My guess is the way
if (!ConfigParser.TryParse(TargetFrameworkMoniker, out RuntimeMoniker runtimeMoniker))
{
throw new NotSupportedException($"Invalid TFM: '{TargetFrameworkMoniker}'");
}
is written runtimeMoniker can never get a value other than what TargetFrameworkMoniker is givien. Since for NativeAot80 TargetFrameworkMoniker is net8.0 runtimeMoniker can only be Net80 and that is wrong since we want NativeAot80.
Also I do find the line runtimeMoniker >= RuntimeMoniker.NativeAot80 dangerous
Is it realy given that ever entry in the enum now and in the funture that comes after NativeAot80 supports native?
What happens if there is some similar requirement that needs some other sorting that invalidates the current sorting of the enum and is incompatible to the current statement?
