apex icon indicating copy to clipboard operation
apex copied to clipboard

g++/c++ compiler incompatibility

Open wpm opened this issue 4 years ago • 7 comments

I am working with an application that uses Pytorch and Apex. The application requires that Apex be compiled with the --cpp_ext and --cuda_ext flags. I set up my environment like so.

conda create -n layoutlm
conda activate layoutlm
conda install -c creditx gcc-7
conda install pytorch cudatoolkit=10.1 -c pytorch
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

I see the following warning when Apex compiles.

                                  !! WARNING !!

    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    Your compiler (c++) is not compatible with the compiler Pytorch was
    built with for this platform, which is g++ on linux. Please
    use g++ to to compile your extension. Alternatively, you may
    compile PyTorch from source using c++, and then you can also use
    c++ to compile your extension.

    See https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md for help
    with compiling PyTorch from source.
    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Looking at the build logs it does appear that sometimes g++ is being used and other times c++.

The application I'm building ultimately fails with a mysterious error, so I'm trying to see if this compiler incompatibility is the cause.

How do I "use g++ to compile [my] extension"? I don't see a Makefile in Apex, and it doesn't look like it's possible to specify a compiler using the options in setup.py.

  • Pytorch 1.6
  • Apex installed from latest source
  • CUDA 10.1
  • CentOS Linux release 7.6.1810 (Core)

wpm avatar Oct 02 '20 18:10 wpm

I'm having the same problem. Any progress?

calee88 avatar Dec 11 '20 12:12 calee88

I just removed every c++ and make one link to the g++. Something like this:

which c++
mv c++ c--
ln -s g++ c++

After that I can install the apex. It is really painful to install anything on a CentOS.

calee88 avatar Dec 11 '20 12:12 calee88

I just set the environment variable CXX=g++. Then it work

Updates: Using environment variables is a more reasonable way, otherwise c++ will not be used correctly if using the way @calee88 proposed. There are three ways follows to choose according to your situation:

  • Add CXX=g++ in front of the execution command like CXX=g++ pip install xxx
  • execute export CXX=g++ first. This will increment an environment variable for the current session rather than for an individual command
  • Add a line of code export CXX=g++ to the corresponding shell resource file. Since most people choose bash, you need to execute a command like echo "export CXX=g++" >> ~/.bashrc. This will be fixed to add this environment variable before you enter the bash environment.

I personally recommend the first one, because it won't affect other commands. The second is a good choice if you need this environment variable multiple times. The third is not recommended unless you always want to use g++ by default.

EeyoreLee avatar Nov 12 '21 08:11 EeyoreLee

I just set the environment variable CXX=g++. Then it work

Than you! This work for me!

ChambinLee avatar Nov 13 '21 12:11 ChambinLee

I just set the environment variable CXX=g++. Then it work

how to set the environment variable?

jbh1998 avatar Nov 30 '21 07:11 jbh1998

I just set the environment variable CXX=g++. Then it work

how to set the environment variable?

Hello, you can try this: export CXX=g++ it works for me.

ChambinLee avatar Nov 30 '21 11:11 ChambinLee

yes that works. Thank you

Cucaracha18 avatar Feb 08 '22 14:02 Cucaracha18