Wrap compiler with NVCC

[mikebentley15]

Even if the test framework does not directly support CUDA anymore, there are code bases that at least need to be compiled with the nvcc compiler, if they are used within the tests, or even if the included code has CUDA kernels that are not used in the tests. There is some real need to be able to compile with the nvcc compiler as a wrapper to the compiler(s) under test.

When using nvcc, you specify the underlying compiler (e.g. clang-5.0) using -ccbin=clang-5.0. Some of the compiler flags are interpreted directly by nvcc, for example, the include flags. But other compiler flags are passed to the underlying compiler, such as -pthread and -Wall and -Wextra. The flags that are passed to the underlying compiler are put in a comma-separated list in the following form: -Xcompiler=-pthread,-Wall,-Wextra. You can specify the -Xcompiler flag more than once.

When using nvcc during linking, you can pass linker flags to the underlying linker with -Xlinker in a similar fashion. However, even when getting the linker flags passed in, it is possible for nvcc to still fail to link with many mysterious errors. There is no need to link using nvcc since there are no NVIDIA-specific steps needed during linking. The nvcc magic has already been performed when creating the object files.

When wrapping a compiler with nvcc, we would use the -ccbin flag to specify what we are wrapping. Then some flags need to be wrapped with the -Xcompiler flag, but it may not be easy to determine which ones. This is the trickiest part of this feature request. What's worse is this has to be done in the Makefile since the user is allowed to specify arbitrary compiler flags within the custom.mk configuration file.

There are a few possible approaches to deal with this. Here are three I can think of:

1. Have a hard-coded list of flags that nvcc supports and wrap everything else. This may be simple to implement, but has the problem of dealing with multiple versions of nvcc and maintaining the list of flags as newer versions come out
2. Assume everything is a flag for nvcc, and dynamically determine which ones are not handled by it by parsing the error messages from nvcc when the flag is not recognized.
3. Split up the variables from CC_REQUIRED into multiple variables. Something like CUDA_REQUIRED, CC_REQUIRED, and INCLUDES. This would make it much easier to implement the wrapping, but it is a possible breaking compatibility change. But maybe it wouldn't be since the CC_REQUIRED from other usages wouldn't actually break things if they put include paths there.

So far, I'm leaning toward option number 3 since it will be easy to implement and I think it can be done in a non-breaking way so that it is simply an added feature.

The next aspect of this issue implementation is to indicate when you want to wrap a compiler with nvcc. This should be done in the flit-config.toml. But this probably should be on a compiler-by-compiler basis, which would mean that we would need to implement issue #120 before we can complete this.

[mikebentley15]

One thing to note that I do not intend to check is that each version of nvcc only supports specific versions of other compilers and will refuse to work with unsupported compiler versions. This seems reasonable. I found it nearly impossible to get the list of supported compilers by nvcc version. It is easy to find the list of supported compilers for the latest version, but for various reasons, certain groups will be tied into older versions of nvcc for some time. For this reason, I think it is reasonable to leave it to the user to make sure they are using a supported compiler version for the nvcc that they use for wrapping. In other words, it is not maintainable to try and manage this for the user.

[mikebentley15]

One possible nicity that we could do is to dynamically call nvcc with the compiler under question with simple flags and parse the error output for something saying that the compiler version is unsupported. This would simply be for

1. early failure of compilation - to give feedback sooner
2. better error messages specifically describing what the problem is

At the time of printing the error, it would be convenient if we could list the set of supported compiler versions for the type of compiler used, but this is the difficult to maintain portion, so will not be done.