Use out-of-source build and install extensions in Python source tree / Use in-source build and clean up without removing cythonized extensions

[ashwinvis]

Motivation for using CMake

I am developing this python package called Fluidsim which use some Cython extensions as wrappers for FFTW libraries. Therefore while building these extensions (say for eg: fftw2dmpiccy) it becomes an issue when the following are not detected:

• CC: The machine uses a custom executable for the MPI C compiler. ie. it could be anything of these: cc/gcc/mpicc/icc. Also sometimes if one uses an anaconda Python distribution it might provide its own MPI compiler.
• libmpi.so : The MPI library is in a non-standard location
• libfftw3.so and libfftw3_mpi.so : The FFTW3 library is in a non-standard location.

Of course I could handle this by installing like this:

CC="cc"   \
CFLAGS="-I/path/to/include"  \
LDFLAGS="-L/path/to/lib"    \
LDSHARED="cc -shared -pthread" \
python setup.py develop

Then this becomes a labourious process of finding include paths and library paths, especially when you have multiple MPI libraries installed.

I was looking for something more generic and automatic, a one click solution.

I tried to use CMake, by following this example. It partially solves this issue with some help from FindMPI.cmake, FindFFTW3.cmake and FindFFTW3MPI.cmake modules (the latter two found in Github).

TLDR: The issue is..

I understand that CMake works best when the extensions are built in a separate build directory. and this requires copying all python source files into it (an out of source build). This is a clutter of files and deploying the project in development mode (python setup.py develop) is a nightmare. Running make clean would remove all the built extensions as well.

Possible solutions: (Not sure how to do it)

• Ideally, I would prefer an in source build. This would help to deploy the python project in development mode. Can CMake clean up after the build operation, without removing the built *.so files themselves? Also I would need CMake not to replace any existing Makefile.
• Can I use an out of source build and instruct Cmake to install the *.c and *.so files back into the source tree.