Update CMakeLists.txt to link new header files for NVTX v3 for CUDA 1…

[DevinCharles]

User description

…2.9+

Description

Adding a switch in CMakeLists.txt to locate and link the appropriate NVTX headers for CUDA 12.9+, while maintaining the previous compatibility with <= CUDA 12.8.

Type of change

Please delete options that are not relevant.

• [x] Bug fix (non-breaking change which fixes an issue)

Scope

• [x] This PR comprises a set of related changes with a common goal

How Has This Been Tested?

Builds with both nvhpc 25.3 and 25.5 (CUDA 12.8 and 12.9) with --debug and --gpu flags enabled. Solve 1x randomly chosen 2D example problem per build.

Test Configuration:

• Personal Laptop (Older Dell P5540 w/ Quadro P2000, Rocky Linux 8.10 - 4.18.0-553.el8_10.x86_64)
• nvhpc 25.3 and nvhpc 25.5

Checklist

• [x] I have added comments for the new code
• [ ] I added Doxygen docstrings to the new code
• [ ] I have made corresponding changes to the documentation (docs/)
• [ ] I have added regression tests to the test suite so that people can verify in the future that the feature is behaving as expected
• [ ] I have added example cases in examples/ that demonstrate my new feature performing as expected. They run to completion and demonstrate "interesting physics"
• [ ] I ran ./mfc.sh format before committing my code
• [ ] New and existing tests pass locally with my changes, including with GPU capability enabled (both NVIDIA hardware with NVHPC compilers and AMD hardware with CRAY compilers) and disabled
• [x] This PR does not introduce any repeated code (it follows the DRY principle)
• [ x] I cannot think of a way to condense this code and reduce any introduced additional line count

---

PR Type

Bug fix

---

Description

• Update CMakeLists.txt to support NVTX v3 for CUDA 12.9+

• Add conditional linking for different CUDA versions

• Maintain backward compatibility with CUDA <= 12.8

• Include nvtx3 component and cudalib linker option

---

Diagram Walkthrough

flowchart LR
  A["CUDA Toolkit Detection"] --> B["Check CUDA Version"]
  B --> C["CUDA <= 12.8"]
  B --> D["CUDA >= 12.9"]
  C --> E["Link CUDA::nvToolsExt"]
  D --> F["Link CUDA::nvtx3"]
  E --> G["Add cudalib=nvtx option"]
  F --> G

File Walkthrough

	Relevant files
Bug fix	CMakeLists.txt Update NVTX linking for CUDA version compatibility              CMakeLists.txt Add conditional NVTX linking based on CUDA version Request nvtx3 component in CUDAToolkit package Add cudalib=nvtx linker option for all cases Maintain compatibility with older CUDA versions +14/-3

---

[qodo-code-review[bot]]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Logic Error The endif() statement on line 540 appears to close the wrong conditional block. It should close the NVHPC/PGI compiler check that starts on line 527, but there's an additional endif() on line 541 that suggests incorrect nesting of conditional blocks. endif() Missing Fallback The code assumes that either CUDA::nvToolsExt or CUDA::nvtx3 will be available, but there's no error handling if neither target exists. This could lead to silent failures or build issues. if (TARGET CUDA::nvToolsExt) # CUDA <= 12.8 target_link_libraries(${a_target} PRIVATE CUDA::nvToolsExt) else() # CUDA >= 12.9 target_link_libraries(${a_target} PRIVATE CUDA::nvtx3) endif()

[qodo-code-review[bot]]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Handle missing nvtx3 component gracefully The nvtx3 component may not be available in older CUDA versions, causing build failures. Consider making the nvtx3 component optional and falling back to the standard CUDAToolkit package if it's not found. CMakeLists.txt [530-537] -# Ask specifically for the header-only NVTX v3 interface -find_package(CUDAToolkit REQUIRED COMPONENTS nvtx3) +# Try to find NVTX v3 interface first, fallback to standard toolkit +find_package(CUDAToolkit REQUIRED COMPONENTS nvtx3 QUIET) +if (NOT CUDAToolkit_FOUND) + find_package(CUDAToolkit REQUIRED) +endif() if (TARGET CUDA::nvToolsExt) # CUDA <= 12.8 target_link_libraries(${a_target} PRIVATE CUDA::nvToolsExt) else() # CUDA >= 12.9 target_link_libraries(${a_target} PRIVATE CUDA::nvtx3) endif() Suggestion importance[1-10]: 8 __ Why: The suggestion correctly identifies that requiring the nvtx3 component in find_package could cause build failures with older CUDA versions, and the proposed fallback mechanism improves the build script's robustness and backward compatibility.	Medium
[ ] More <!-- /improve --more_suggestions=true -->

[codecov[bot]]

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 44.06%. Comparing base (565178a) to head (09b0f54). :warning: Report is 338 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #949   +/-   ##
=======================================
  Coverage   44.06%   44.06%           
=======================================
  Files          68       68           
  Lines       18220    18220           
  Branches     2292     2292           
=======================================
  Hits         8029     8029           
  Misses       8821     8821           
  Partials     1370     1370           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sbryngelson]

works on runners but i'm seeing some weird behavior with nvhpc/25.5 on Phoenix (which is not used on the runners right now, 24.5 is). will try some things out myself before merging.

[DevinCharles]

Let me know if there's anything I can check on my side. I'll add 24.5 to the list to check. Do the runners run all tests? Is there a specific test I can check our can you add the errors here if you think they're relevant to other systems?

[sbryngelson]

Well 25.5 fails for me on Phoenix w/ your PR but that computer is currently running v24.5 when the runners go - so the current PR has v24.5 working fine. I can change this PR so it loads the latest modules (I should probably just do that) and then we will see what breaks/doesn't break. When I tried doing it myself on Phoenix it just broke in a strange way so I'll try again later.

[sbryngelson]

Hmm I cannot get this working with nvhpc v25.5 on my end (using your/our changes). Can you check out the latest master branch and see if you can compile? If you don't mind, can you give me details of your compiler stack?

[DevinCharles]

Sorry @sbryngelson , just seeing your comments. I'll try building with the latest master branch. I'll follow up here with my script to create my build environment from a clean Rocky 8.10 install.

[sbryngelson]

No worries @DevinCharles but I'm quite curious for your results.

We've been having trouble (even with your cmakelists file) w/ nvhpc 25.X.

If your build works, can you list your full environment (that is relevant), like

• nvfortran --version
• mpif90 --version or mpif90 --show-me
• python --version
• the commands you ran to build MFC... presumably you should be running with ./mfc.sh build --gpu <...>.

[DevinCharles]

$ nvfortran --version
nvfortran 25.5-0 64-bit target on x86-64 Linux -tp haswell

$ mpif90 --version
nvfortran 25.5-0 64-bit target on x86-64 Linux -tp haswell

$ python3 --version
Python 3.12.11

Running build with:

$ ./mfc.sh clean && ./mfc.sh build --mpi --gpu --debug

[DevinCharles]

This is the script I used to bootstrap my environment on a clean build of Rocky 8.10... To be fair, this was written after running all these commands, sorting through my history log, so it might not be exact. I can try building from a clean 8.10 install on a VM.

#!/usr/bin/bash

# Rocky 8.10 Installs with Python 3.6, MFC needs >=3.9
python_version=3.12

## Install NVIDIA Drivers ##
# (https://docs.rockylinux.org/desktop/display/installing_nvidia_gpu_drivers/)

# Utilities and Dependencies
sudo dnf install epel-release -y
sudo dnf groupinstall "Development Tools" -y
sudo dnf install kernel-devel -y
sudo dnf install dkms -y
sudo dnf install openmpi openmpi-devel -y
sudo dnf install cmake -y

# Add Repo Matching RHEL Version
platform_id=$(awk -F':' '/^PLATFORM_ID=/ {gsub(/"/,""); print "rh"$2}' /etc/os-release)
sudo dnf config-manager --add-repo http://developer.download.nvidia.com/compute/cuda/repos/$platform_id/$(uname -i)/cuda-$platform_id.repo

# Install Kernel Headers
sudo dnf install kernel-headers-$(uname -r) kernel-devel-$(uname -r) tar bzip2 make automake gcc gcc-c++ pciutils elfutils-libelf-devel libglvnd-opengl libglvnd-glx libglvnd-devel acpid pkgconf dkms -y

# Install Latest Nvidia Driver
sudo dnf module install nvidia-driver:latest-dkms -y

# For Compute Only (headless system)
#sudo dnf install nvidia-driver-cuda kmod-nvidia-latest-dkms -y

# Disable Nouveau
sudo grubby --args="nouveau.modeset=0 rd.driver.blacklist=nouveau" --update-kernel=ALL

## Install Lmod ##
sudo dnf config-manager --set-enabled powertools
sudo dnf update
sudo dnf install Lmod

## Install Latest NVHPC Compiler ##
sudo dnf config-manager --add-repo https://developer.download.nvidia.com/hpc-sdk/rhel/nvhpc.repo
sudo dnf install -y nvhpc

# Add in NVHPC Module Files
module use /opt/nvidia/hpc_sdk/modulefiles/
module load nvhpc/

## Install Updated Python ##
sudo dnf install python$python_version
sudo alternatives --set python3 /usr/bin/python$python_version

## Multiple NVHPC Versions
#sudo dnf install -y nvhpc-25.3
#module load nvhpc/25.3

[sbryngelson]

very thorough, thanks! does this also work 25.5? You don't have to build from the ground-up.