Build NiftyPET>=2

[paskino]

Building NiftyPET>=2 will allow dropping python2 support, see https://github.com/UCL/STIR/issues/795, https://github.com/SyneRBI/SIRF-SuperBuild/issues/484

NiftyPET binaries could be installed via pip, or by integrating it in the SuperBuild.

Current issues for integration in SuperBuild:

• [ ] CMake required 3.18

[paskino]

Interestingly enough NiftyPET doesn't seem to build on my Ubuntu18.04 with python3:

[  4%] Building CUDA object nipet/CMakeFiles/mmr_auxe.dir/src/aux_module.cu.o
cd /home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/build/nipet && /usr/local/cuda/bin/nvcc  -Dmmr_auxe_EXPORTS -I/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src -I/usr/include/python3.6m -I/home/ofn77899/.local/lib/python3.6/site-packages/numpy/core/include -I/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/include -O3 -DNDEBUG --generate-code=arch=compute_30,code=[compute_30,sm_30] -Xcompiler=-fPIC -std=c++14 -x cu -c /home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu -o CMakeFiles/mmr_auxe.dir/src/aux_module.cu.o
/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(148): error: identifier "NPY_ARRAY_INOUT_ARRAY2" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(222): error: identifier "PyArray_DiscardWritebackIfCopy" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(302): error: identifier "PyArray_ResolveWritebackIfCopy" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(368): error: identifier "NPY_ARRAY_INOUT_ARRAY2" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(374): error: identifier "PyArray_DiscardWritebackIfCopy" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(395): error: identifier "PyArray_ResolveWritebackIfCopy" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(457): error: identifier "NPY_ARRAY_INOUT_ARRAY2" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(463): error: identifier "PyArray_DiscardWritebackIfCopy" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(483): error: identifier "PyArray_ResolveWritebackIfCopy" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(566): error: identifier "NPY_ARRAY_INOUT_ARRAY2" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(573): error: identifier "PyArray_DiscardWritebackIfCopy" is undefined

/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu(600): error: identifier "PyArray_ResolveWritebackIfCopy" is undefined

However, on another machine with CentOS and conda python it works. I checked the imports and they seem to be in order.

[KrisThielemans]

Create a NiftyPET issue? Or possibly @casperdcl knows immediatelly...

[casperdcl]

Your python3 also needs to have numpy installed (if you pip install "nipet>=2" this would be done for you)

[paskino]

@casperdcl I have numpy installed! That's installed with pip to the local user.

[casperdcl]

Yes but I presume you're using some sort of custom CMake wrapper so you need to find_package() first... have a look at https://github.com/NiftyPET/NIPET/blob/master/niftypet/CMakeLists.txt

[casperdcl]

Wait hang on... your output shows you're picking up numpy. My guess is it's just really old. Try pip install --upgrade --user numpy

[paskino]

Not that I don't want to do it, but this is what I've got currently: a pip installed local version of numpy 1.19.5

numpy.version
<module 'numpy.version' from '/home/ofn77899/.local/lib/python3.6/site-packages/numpy/version.py'>
>>> numpy.version.version
'1.19.5'

[paskino]

Just to make this clear @casperdcl I'm not building NiftyPET as part of SIRF-SuperBuild yet, but rather just the package itself using the instructions on the NiftyPET repo.

In the process I found out that the CUDA compiler must already be in the ${PATH} if CUDA is defined as one of the project's LANGUAGES.

Also,

python3 -m pip install --upgrade --user numpy
Requirement already satisfied: numpy in ./.local/lib/python3.6/site-packages (1.19.5)

[paskino]

I downgraded numpy to 1.18.

I added a message(WARNING to the NiftyPET CMakeLists.txt to see that Python is found and includes are there, but it still fails in the same way.

Do you manage to build on ubuntu 18.04 with stock python3?

 cmake ../niftypet/ -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- The CUDA compiler identification is NVIDIA 10.1.168
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Found Python3: /usr/bin/python3.6 (found version "3.6.9") found components: Interpreter Development NumPy Development.Module Development.Embed
CMake Warning at CMakeLists.txt:11 (message):
  Found /usr/include/python3.6m
  /home/ofn77899/.local/lib/python3.6/site-packages/numpy/core/include


-- Found CUDAToolkit: /usr/local/cuda/include (found version "10.1.168")
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- CUDA architectures: 30
-- Build type: Release
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/build

[casperdcl]

Curious.

• NumPy: >=1.14 should work. Just tested on python=3.6.12 numpy=1.19.5
• CUDA: you should be able to set any of CUDA_ROOT, CUDAToolkit_ROOT, or CUDACXX env vars instead of adding to PATH but whatever

I also don't understand your problem. You say that you've already pip installed nipet (so it's already built) and you're just using find_package & link libs functionality, yet you post a build error.

[paskino]

I haven't pip installed nipet, I've cloned the repo and started building as you suggest in the readme:

# ... or build & install directly with cmake
mkdir build && cd build
cmake ../niftypet && cmake --build . && cmake --install . --prefix /my/install/dir

For me it fails at --build stage.

Maybe this will help

[  4%] Building CUDA object nipet/CMakeFiles/mmr_auxe.dir/src/aux_module.cu.o
cd /home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/build/nipet && /usr/local/cuda/bin/nvcc  -Dmmr_auxe_EXPORTS -I/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src -I/usr/include/python3.6m -I/home/ofn77899/.local/lib/python3.6/site-packages/numpy/core/include -I/home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/include -O3 -DNDEBUG --generate-code=arch=compute_30,code=[compute_30,sm_30] -Xcompiler=-fPIC -std=c++14 -x cu -c /home/ofn77899/devel/buildNiftyPET/sources/NiftyPET/niftypet/nipet/src/aux_module.cu -o CMakeFiles/mmr_auxe.dir/src/aux_module.cu.o

[casperdcl]

ah right.

Well this is what I've done on a clean Ubuntu machine (18.04 and 20.04):

sudo snap install --classic cmake
wget https://raw.githubusercontent.com/pypa/get-pip/master/get-pip.py
python3 get-pip.py --user
python3 -m pip install --user 'numpy==1.19.5'  # no need to specify version. just testing...
git clone https://github.com/NiftyPET/NIPET nipet
mkdir nipet/build
cd nipet/build
cmake ../niftypet && cmake --build .

If that doesn't work for you, then please do open an issue at https://github.com/NiftyPET/NIPET

[paskino]

So,

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.5 LTS
Release:        18.04
Codename:       bionic

I ran

sudo apt update
sudo apt upgrade
sudo apt install python3-dev
sudo apt install nvidia-cuda-toolkit
sudo snap install --classic cmake
wget https://raw.githubusercontent.com/pypa/get-pip/master/get-pip.py
python3 get-pip.py --user
python3 -m pip install --user 'numpy==1.19.5'  # no need to specify version. just testing...
git clone https://github.com/NiftyPET/NIPET nipet
mkdir nipet/build
cd nipet/build
cmake ../niftypet && cmake --build .

Error

[  4%] Building CUDA object nipet/CMakeFiles/mmr_auxe.dir/src/aux_module.cu.o
cd /home/ofn77899/nipet/build/nipet && /usr/bin/nvcc  -Dmmr_auxe_EXPORTS -I/home/ofn77899/nipet/niftypet/nipet/src -I/usr/include/python3.6m -I/home/ofn77899/.local/lib/python3.6/site-packages/numpy/core/include -I/home/ofn77899/nipet/niftypet/nipet/include -O3 -DNDEBUG --generate-code=arch=compute_30,code=[compute_30,sm_30] -Xcompiler=-fPIC -std=c++14 -x cu -c /home/ofn77899/nipet/niftypet/nipet/src/aux_module.cu -o CMakeFiles/mmr_auxe.dir/src/aux_module.cu.o
/home/ofn77899/.local/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h(84): error: expected a "}"

/home/ofn77899/.local/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h(89): warning: parsing restarts here after previous syntax error

/home/ofn77899/.local/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h(450): error: identifier "NPY_NTYPES_ABI_COMPATIBLE" is undefined

2 errors detected in the compilation of "/tmp/tmpxft_00005c59_00000000-6_aux_module.cpp1.ii".

I then tried to install CUDA Toolkit from NVidia, and got 10.1. With it it works, both with conda and stock python.

Maybe 9.1, which is installed by Ubuntu is not sufficient for NiftyPET?

[paskino]

There must be more than this going on. On my old machine it still fails with the old message, even after upgrading to 10.1 from NVIDIA, rather than from ubuntu.

[KrisThielemans]

We will not do this. NiftyPET has dropped C++ support, so the STIR interface is no longer functional with newer NiftyPET versions.

Contributions for direct support for NiftyPET in CIL/SIRF welcome.