llvm-project icon indicating copy to clipboard operation
llvm-project copied to clipboard

[AArch64] Add getHostCPUFeatures to query for enabled features in cpu…

Open neildhickey opened this issue 1 year ago • 2 comments

…info

This pull request adds code to call getHostCPUInfo into the AArch64 Target Parser to query the cpuinfo for the device in the case where we are compiling with -mcpu=native

neildhickey avatar Jul 04 '24 15:07 neildhickey

@llvm/pr-subscribers-clang-driver @llvm/pr-subscribers-backend-aarch64

@llvm/pr-subscribers-clang

Author: None (neildhickey)

Changes

…info

This pull request adds code to call getHostCPUInfo into the AArch64 Target Parser to query the cpuinfo for the device in the case where we are compiling with -mcpu=native


Full diff: https://github.com/llvm/llvm-project/pull/97749.diff

1 Files Affected:

  • (modified) clang/lib/Driver/ToolChains/Arch/AArch64.cpp (+17)
diff --git a/clang/lib/Driver/ToolChains/Arch/AArch64.cpp b/clang/lib/Driver/ToolChains/Arch/AArch64.cpp
index ec248b80251ea3..2862c297622fa9 100644
--- a/clang/lib/Driver/ToolChains/Arch/AArch64.cpp
+++ b/clang/lib/Driver/ToolChains/Arch/AArch64.cpp
@@ -445,4 +445,21 @@ void aarch64::getAArch64TargetFeatures(const Driver &D,
 
   if (Args.getLastArg(options::OPT_mno_bti_at_return_twice))
     Features.push_back("+no-bti-at-return-twice");
+
+  // Parse AArch64 CPU Features
+  const Arg *CPUArg = Args.getLastArg(options::OPT_mcpu_EQ);
+  StringRef CPUName;
+
+  if (CPUArg) {
+    CPUName = CPUArg->getValue();
+    if (CPUName == "native") {
+      llvm::StringMap<bool> HostFeatures;
+      if (llvm::sys::getHostCPUFeatures(HostFeatures)) {
+        for (auto &F : HostFeatures) {
+          Features.push_back(
+            Args.MakeArgString((F.second ? "+" : "-") + F.first()));
+        }
+      }
+    }
+  }
 }

llvmbot avatar Jul 04 '24 15:07 llvmbot

:white_check_mark: With the latest revision this PR passed the C/C++ code formatter.

github-actions[bot] avatar Jul 04 '24 15:07 github-actions[bot]

Hi this sounds like a good idea to me. Note that the implementation of getHostCPUFeatures isn't amazing for AArch64 at the moment, there was an attempt to fix it up in #95694 (but thaat has gone a bit quiet). One point we noticed is that it could end up turning "aes+sha2" into "crypto" and "crypto" back into "sha2+aes+sha3+sm4", as it uses the old meaning of "crypto"

davemgreen avatar Jul 05 '24 07:07 davemgreen

Thanks for the review comments @madhur13490 and @davemgreen I plan to improve getHostCPUFeatures for AArach64 as well, this is just the first phase to add the function call.

neildhickey avatar Jul 05 '24 07:07 neildhickey

Looks ok to me, X86 and ARM already do this.

One point we noticed is that it could end up turning "aes+sha2" into "crypto" and "crypto" back into "sha2+aes+sha3+sm4", as it uses the old meaning of "crypto"

Then this needs to be fixed before this PR can go in, we don't want to be making binaries folks can't run. @davemgreen do we have a way that @neildhickey can reproduce that easily?

DavidSpickett avatar Jul 05 '24 07:07 DavidSpickett

Looks ok to me, X86 and ARM already do this.

One point we noticed is that it could end up turning "aes+sha2" into "crypto" and "crypto" back into "sha2+aes+sha3+sm4", as it uses the old meaning of "crypto"

Then this needs to be fixed before this PR can go in, we don't want to be making binaries folks can't run. @davemgreen do we have a way that @neildhickey can reproduce that easily?

Perhaps this is the part of code causing the issue

#if defined(__aarch64__)
  // If we have all crypto bits we can add the feature
  if (crypto == (CAP_AES | CAP_PMULL | CAP_SHA1 | CAP_SHA2))
    Features["crypto"] = true;
#endif

If so I can change this to remove the addition of crypto and just assert the features found in the cpuinfo file

neildhickey avatar Jul 24 '24 07:07 neildhickey

It is that bit of code, yeah. I don't know of a way to reproduce this without logging into different machines with different sets of options and trying it.

If we had a way to test/mock that various /proc/cpuinfo files gave us the correct results, that would be helpful in giving us more confidence it was working as intended.

davemgreen avatar Jul 24 '24 08:07 davemgreen

Sorry for the delays in response. I addressed the comments and rebased the branch

ElvinaYakubova avatar Oct 28 '24 16:10 ElvinaYakubova

LLVM Buildbot has detected a new failure on builder llvm-clang-x86_64-gcc-ubuntu running on sie-linux-worker3 while building clang,llvm at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/174/builds/7513

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'Clang :: Driver/aarch64-mcpu-native.c' FAILED ********************
Exit Code: 2

Command Output (stderr):
--
RUN: at line 2: export LLVM_CPUINFO=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/Inputs/cpunative/neoverse-v2
+ export LLVM_CPUINFO=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/Inputs/cpunative/neoverse-v2
+ LLVM_CPUINFO=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/Inputs/cpunative/neoverse-v2
RUN: at line 3: /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang --target=aarch64 --print-enabled-extensions -mcpu=native | /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/FileCheck --strict-whitespace --check-prefix=CHECK-FEAT-NV2 --implicit-check-not=FEAT_ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/aarch64-mcpu-native.c
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/FileCheck --strict-whitespace --check-prefix=CHECK-FEAT-NV2 --implicit-check-not=FEAT_ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/aarch64-mcpu-native.c
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang --target=aarch64 --print-enabled-extensions -mcpu=native
clang version 20.0.0git (https://github.com/llvm/llvm-project.git d732c0b13c55259177f2936516b6087d634078e0)
Target: aarch64
Thread model: posix
InstalledDir: /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin
Build config: +assertions
clang: error: unsupported argument 'native' to option '-mcpu='
FileCheck error: '<stdin>' is empty.
FileCheck command line:  /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/FileCheck --strict-whitespace --check-prefix=CHECK-FEAT-NV2 --implicit-check-not=FEAT_ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/aarch64-mcpu-native.c

--

********************


llvm-ci avatar Oct 29 '24 13:10 llvm-ci

LLVM Buildbot has detected a new failure on builder llvm-clang-aarch64-darwin running on doug-worker-4 while building clang,llvm at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/190/builds/8413

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'Clang :: Driver/aarch64-mcpu-native.c' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
RUN: at line 2: export LLVM_CPUINFO=/Users/buildbot/buildbot-root/aarch64-darwin/llvm-project/clang/test/Driver/Inputs/cpunative/neoverse-v2
+ export LLVM_CPUINFO=/Users/buildbot/buildbot-root/aarch64-darwin/llvm-project/clang/test/Driver/Inputs/cpunative/neoverse-v2
+ LLVM_CPUINFO=/Users/buildbot/buildbot-root/aarch64-darwin/llvm-project/clang/test/Driver/Inputs/cpunative/neoverse-v2
RUN: at line 3: /Users/buildbot/buildbot-root/aarch64-darwin/build/bin/clang --target=aarch64 --print-enabled-extensions -mcpu=native | /Users/buildbot/buildbot-root/aarch64-darwin/build/bin/FileCheck --strict-whitespace --check-prefix=CHECK-FEAT-NV2 --implicit-check-not=FEAT_ /Users/buildbot/buildbot-root/aarch64-darwin/llvm-project/clang/test/Driver/aarch64-mcpu-native.c
+ /Users/buildbot/buildbot-root/aarch64-darwin/build/bin/clang --target=aarch64 --print-enabled-extensions -mcpu=native
+ /Users/buildbot/buildbot-root/aarch64-darwin/build/bin/FileCheck --strict-whitespace --check-prefix=CHECK-FEAT-NV2 --implicit-check-not=FEAT_ /Users/buildbot/buildbot-root/aarch64-darwin/llvm-project/clang/test/Driver/aarch64-mcpu-native.c
clang version 20.0.0git (https://github.com/llvm/llvm-project.git d732c0b13c55259177f2936516b6087d634078e0)
Target: aarch64
Thread model: posix
InstalledDir: /Users/buildbot/buildbot-root/aarch64-darwin/build/bin
Build config: +assertions
[1m/Users/buildbot/buildbot-root/aarch64-darwin/llvm-project/clang/test/Driver/aarch64-mcpu-native.c:11:23: [0m[0;1;31merror: [0m[1mCHECK-FEAT-NV2: expected string not found in input
[0m// CHECK-FEAT-NV2:    FEAT_BF16                                              Enable BFloat16 Extension
[0;1;32m                      ^
[0m[1m<stdin>:6:93: [0m[0;1;30mnote: [0m[1mscanning from here
[0m    FEAT_AdvSIMD                                           Enable Advanced SIMD instructions
[0;1;32m                                                                                            ^
[0m[1m<stdin>:16:5: [0m[0;1;30mnote: [0m[1mpossible intended match here
[0m    FEAT_FP16                                              Enable half-precision floating-point data processing
[0;1;32m    ^
[0m
Input file: <stdin>
Check file: /Users/buildbot/buildbot-root/aarch64-darwin/llvm-project/clang/test/Driver/aarch64-mcpu-native.c

-dump-input=help explains the following input dump.

Input was:
<<<<<<
[1m[0m[0;1;30m            1: [0m[1m[0;1;46m[0mExtensions enabled for the given AArch64 target[0;1;46m [0m
[0;1;32mcheck:5        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[0m[0;1;32mnot:imp1'0     X
[0m[0;1;32mnot:imp1'1                                                    X
[0m[0;1;30m            2: [0m[1m[0;1;46m[0m [0m
[0;1;32mempty:6        ^
[0m[0;1;32mnot:imp1'2     X
[0m[0;1;30m            3: [0m[1m[0;1;46m    [0mArchitecture Feature(s)                                Description[0;1;46m [0m
[0;1;32mcheck:7            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[0m[0;1;32mnot:imp1'2     ~~~~
[0m[0;1;32mnot:imp1'3                                                                           X
[0m[0;1;30m            4: [0m[1m[0;1;46m    [0mFEAT_AES, FEAT_PMULL                                   Enable AES support[0;1;46m [0m
[0;1;32mcheck:8            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[0m[0;1;32mnot:imp1'3     ~~~~
[0m[0;1;32mnot:imp1'4                                                                                  X
[0m[0;1;30m            5: [0m[1m[0;1;46m    [0mFEAT_AMUv1                                             Enable Armv8.4-A Activity Monitors extension[0;1;46m [0m
[0;1;32mcheck:9            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...

llvm-ci avatar Oct 29 '24 13:10 llvm-ci

LLVM Buildbot has detected a new failure on builder llvm-x86_64-debian-dylib running on gribozavr4 while building clang,llvm at step 6 "test-build-unified-tree-check-clang".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/60/builds/11290

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-clang) failure: test (failure)
******************** TEST 'Clang :: Driver/aarch64-mcpu-native.c' FAILED ********************
Exit Code: 2

Command Output (stderr):
--
RUN: at line 2: export LLVM_CPUINFO=/b/1/llvm-x86_64-debian-dylib/llvm-project/clang/test/Driver/Inputs/cpunative/neoverse-v2
+ export LLVM_CPUINFO=/b/1/llvm-x86_64-debian-dylib/llvm-project/clang/test/Driver/Inputs/cpunative/neoverse-v2
+ LLVM_CPUINFO=/b/1/llvm-x86_64-debian-dylib/llvm-project/clang/test/Driver/Inputs/cpunative/neoverse-v2
RUN: at line 3: /b/1/llvm-x86_64-debian-dylib/build/bin/clang --target=aarch64 --print-enabled-extensions -mcpu=native | /b/1/llvm-x86_64-debian-dylib/build/bin/FileCheck --strict-whitespace --check-prefix=CHECK-FEAT-NV2 --implicit-check-not=FEAT_ /b/1/llvm-x86_64-debian-dylib/llvm-project/clang/test/Driver/aarch64-mcpu-native.c
+ /b/1/llvm-x86_64-debian-dylib/build/bin/FileCheck --strict-whitespace --check-prefix=CHECK-FEAT-NV2 --implicit-check-not=FEAT_ /b/1/llvm-x86_64-debian-dylib/llvm-project/clang/test/Driver/aarch64-mcpu-native.c
+ /b/1/llvm-x86_64-debian-dylib/build/bin/clang --target=aarch64 --print-enabled-extensions -mcpu=native
clang version 20.0.0git (https://github.com/llvm/llvm-project.git d732c0b13c55259177f2936516b6087d634078e0)
Target: aarch64
Thread model: posix
InstalledDir: /b/1/llvm-x86_64-debian-dylib/build/bin
Build config: +assertions
clang: error: unsupported argument 'native' to option '-mcpu='
FileCheck error: '<stdin>' is empty.
FileCheck command line:  /b/1/llvm-x86_64-debian-dylib/build/bin/FileCheck --strict-whitespace --check-prefix=CHECK-FEAT-NV2 --implicit-check-not=FEAT_ /b/1/llvm-x86_64-debian-dylib/llvm-project/clang/test/Driver/aarch64-mcpu-native.c

--

********************


llvm-ci avatar Oct 29 '24 13:10 llvm-ci

LLVM Buildbot has detected a new failure on builder clang-x86_64-debian-fast running on gribozavr4 while building clang,llvm at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/56/builds/10880

Here is the relevant piece of the build log for the reference
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'Clang :: Driver/aarch64-mcpu-native.c' FAILED ********************
Exit Code: 2

Command Output (stderr):
--
RUN: at line 2: export LLVM_CPUINFO=/b/1/clang-x86_64-debian-fast/llvm.src/clang/test/Driver/Inputs/cpunative/neoverse-v2
+ export LLVM_CPUINFO=/b/1/clang-x86_64-debian-fast/llvm.src/clang/test/Driver/Inputs/cpunative/neoverse-v2
+ LLVM_CPUINFO=/b/1/clang-x86_64-debian-fast/llvm.src/clang/test/Driver/Inputs/cpunative/neoverse-v2
RUN: at line 3: /b/1/clang-x86_64-debian-fast/llvm.obj/bin/clang --target=aarch64 --print-enabled-extensions -mcpu=native | /b/1/clang-x86_64-debian-fast/llvm.obj/bin/FileCheck --strict-whitespace --check-prefix=CHECK-FEAT-NV2 --implicit-check-not=FEAT_ /b/1/clang-x86_64-debian-fast/llvm.src/clang/test/Driver/aarch64-mcpu-native.c
+ /b/1/clang-x86_64-debian-fast/llvm.obj/bin/clang --target=aarch64 --print-enabled-extensions -mcpu=native
+ /b/1/clang-x86_64-debian-fast/llvm.obj/bin/FileCheck --strict-whitespace --check-prefix=CHECK-FEAT-NV2 --implicit-check-not=FEAT_ /b/1/clang-x86_64-debian-fast/llvm.src/clang/test/Driver/aarch64-mcpu-native.c
clang version 20.0.0git (https://github.com/llvm/llvm-project.git d732c0b13c55259177f2936516b6087d634078e0)
Target: aarch64
Thread model: posix
InstalledDir: /b/1/clang-x86_64-debian-fast/llvm.obj/bin
Build config: +assertions
clang: error: unsupported argument 'native' to option '-mcpu='
FileCheck error: '<stdin>' is empty.
FileCheck command line:  /b/1/clang-x86_64-debian-fast/llvm.obj/bin/FileCheck --strict-whitespace --check-prefix=CHECK-FEAT-NV2 --implicit-check-not=FEAT_ /b/1/clang-x86_64-debian-fast/llvm.src/clang/test/Driver/aarch64-mcpu-native.c

--

********************


llvm-ci avatar Oct 29 '24 13:10 llvm-ci

LLVM Buildbot has detected a new failure on builder premerge-monolithic-linux running on premerge-linux-1 while building clang,llvm at step 7 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/153/builds/13027

Here is the relevant piece of the build log for the reference
Step 7 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'Clang :: Driver/aarch64-mcpu-native.c' FAILED ********************
Exit Code: 2

Command Output (stderr):
--
RUN: at line 2: export LLVM_CPUINFO=/build/buildbot/premerge-monolithic-linux/llvm-project/clang/test/Driver/Inputs/cpunative/neoverse-v2
+ export LLVM_CPUINFO=/build/buildbot/premerge-monolithic-linux/llvm-project/clang/test/Driver/Inputs/cpunative/neoverse-v2
+ LLVM_CPUINFO=/build/buildbot/premerge-monolithic-linux/llvm-project/clang/test/Driver/Inputs/cpunative/neoverse-v2
RUN: at line 3: /build/buildbot/premerge-monolithic-linux/build/bin/clang --target=aarch64 --print-enabled-extensions -mcpu=native | /build/buildbot/premerge-monolithic-linux/build/bin/FileCheck --strict-whitespace --check-prefix=CHECK-FEAT-NV2 --implicit-check-not=FEAT_ /build/buildbot/premerge-monolithic-linux/llvm-project/clang/test/Driver/aarch64-mcpu-native.c
+ /build/buildbot/premerge-monolithic-linux/build/bin/clang --target=aarch64 --print-enabled-extensions -mcpu=native
+ /build/buildbot/premerge-monolithic-linux/build/bin/FileCheck --strict-whitespace --check-prefix=CHECK-FEAT-NV2 --implicit-check-not=FEAT_ /build/buildbot/premerge-monolithic-linux/llvm-project/clang/test/Driver/aarch64-mcpu-native.c
clang version 20.0.0git (https://github.com/llvm/llvm-project.git d732c0b13c55259177f2936516b6087d634078e0)
Target: aarch64
Thread model: posix
InstalledDir: /build/buildbot/premerge-monolithic-linux/build/bin
Build config: +assertions
clang: error: unsupported argument 'native' to option '-mcpu='
FileCheck error: '<stdin>' is empty.
FileCheck command line:  /build/buildbot/premerge-monolithic-linux/build/bin/FileCheck --strict-whitespace --check-prefix=CHECK-FEAT-NV2 --implicit-check-not=FEAT_ /build/buildbot/premerge-monolithic-linux/llvm-project/clang/test/Driver/aarch64-mcpu-native.c

--

********************


llvm-ci avatar Oct 29 '24 14:10 llvm-ci