elastix icon indicating copy to clipboard operation
elastix copied to clipboard

Published 20 hours ago • verified publisher icon SuperElastix

Building with OpenCL

Open mavanb opened this issue 6 years ago • 17 comments

Hi guys,

I am trying to build Elastix with OpenCL support, but I am not able to get it work (building without OpenCL works fine).

I trying on ubuntu 18.04 with:

  • Nvidia GeForce RTX 2080 Ti
  • gcc 7.3.0
  • cmake 3.10.2

I installed OpenCL using CUDA toolkit 10.0 (with nvidia driver 410.79).

I compiled ITK 4.13 with: -BUILD_SHARED_LIBS=OFF -Module_ITKReview=ON -ITK_WRAP_*=OFF

I am trying to compile Elastix 4.9.0 with: -CMAKE_BUILD_TYPE=Release -ELASTIX_BUILD_EXECUTABLE=OFF -ELASTIX_USE_OPENCL=ON -OPENCL_C_VERSION_1_2=ON -OPENCL_INCLUDE_DIRS=/usr/local/cuda/include -OPENCL_LIBRARIES=/usr/local/cuda/lib64/libOpenCL.so -USE_OpenCLFixedGenericPyramid=ON -USE_OpenCLMovingGenericPyramid=ON -USE_OpenCLResampler=ON -OPENCL_USENVIDIA_SDK=ON

I get the following error:

[ 29%] Building CXX object Testing/CMakeFiles/itkOpenCLStringUtilsTest.dir/itkTestOutputWindow.cxx.o
In file included from /home/mart/Elastix/elastix/Common/OpenCL/ITKimprovements/itkOpenCLKernel.h:22:0,
                 from /home/mart/Elastix/elastix/Testing/itkOpenCLKernelTest.cxx:18:
/home/mart/Elastix/elastix/Common/OpenCL/ITKimprovements/itkOpenCLGlobal.h:29:29: warning: ‘template<class> class std::auto_ptr’ is deprecated [-Wdeprecated-declarations]
 static inline typename std::auto_ptr< TObjectType >::element_type * OpenCLGetPtrHelper(
                             ^~~~~~~~
In file included from /usr/include/c++/7/memory:80:0,
                 from /home/mart/Elastix/elastix/Common/OpenCL/ITKimprovements/itkOpenCLGlobal.h:21,
                 from /home/mart/Elastix/elastix/Common/OpenCL/ITKimprovements/itkOpenCLKernel.h:22,
                 from /home/mart/Elastix/elastix/Testing/itkOpenCLKernelTest.cxx:18:
/usr/include/c++/7/bits/unique_ptr.h:51:28: note: declared here
   template<typename> class auto_ptr;
                            ^~~~~~~~

[ 35%] Building CXX object Testing/CMakeFiles/itkBSplineInterpolationWeightFunctionTest.dir/itkCommandLineArgumentParser.cxx.o
In file included from /home/mart/Elastix/elastix/Common/OpenCL/Factories/itkGPUSupportedImages.h:22:0,
                 from /home/mart/Elastix/elastix/Common/OpenCL/Factories/itkGPUObjectFactoryBase.h:21,
                 from /home/mart/Elastix/elastix/Common/OpenCL/Factories/itkGPUImageFactory.h:21,
                 from /home/mart/Elastix/elastix/Common/OpenCL/Filters/itkGPUTransformCopier.hxx:42,
                 from /home/mart/Elastix/elastix/Common/OpenCL/Filters/itkGPUTransformCopier.h:240,
                 from /home/mart/Elastix/elastix/Testing/itkGPUResampleImageFilterTest.cxx:24:
/home/mart/Elastix/elastix/Common/TypeList.h: In member function ‘void typelist::Visit<TTypeList>::operator()(Predicate&)’:
/home/mart/Elastix/elastix/Common/TypeList.h:494:45: error: expected primary-expression before ‘>’ token
     visitor.CLANG_TEMPLATE operator()< Head >( );
                                             ^

[ 43%] Built target itkOpenCLSimpleTest
In file included from /home/mart/Elastix/elastix/Common/OpenCL/Factories/itkGPUSupportedImages.h:22:0,
                 from /home/mart/Elastix/elastix/Common/OpenCL/Factories/itkGPUObjectFactoryBase.h:21,
                 from /home/mart/Elastix/elastix/Common/OpenCL/Factories/itkGPUImageFactory.h:21,
                 from /home/mart/Elastix/elastix/Common/OpenCL/Filters/itkGPUTransformCopier.hxx:42,
                 from /home/mart/Elastix/elastix/Common/OpenCL/Filters/itkGPUTransformCopier.h:240,
                 from /home/mart/Elastix/elastix/Testing/itkGPUResampleImageFilterTest.cxx:24:
/home/mart/Elastix/elastix/Common/TypeList.h: In instantiation of ‘void typelist::DualVisitDimension<TLeftTypeList, TRightTypeList, Dimension>::operator()(Visitor&) const [with Visitor = itk::GPUBSplineDecompositionImageFilterFactory2<typelist::TypeList<short int, typelist::NullType>, typelist::TypeList<short int, typelist::NullType>, OCLImageDims>; TLeftTypeList = typelist::TypeList<short int, typelist::NullType>; TRightTypeList = typelist::TypeList<short int, typelist::NullType>; unsigned int Dimension = 1]’:
/home/mart/Elastix/elastix/Common/OpenCL/Factories/itkGPUBSplineDecompositionImageFilterFactory.hxx:53:10:   required from ‘void itk::GPUBSplineDecompositionImageFilterFactory2<TTypeListIn, TTypeListOut, NDimensions>::Register1D() [with TTypeListIn = typelist::TypeList<short int, typelist::NullType>; TTypeListOut = typelist::TypeList<short int, typelist::NullType>; NDimensions = OCLImageDims]’
/home/mart/Elastix/elastix/Testing/itkGPUResampleImageFilterTest.cxx:1202:1:   required from here
/home/mart/Elastix/elastix/Common/TypeList.h:744:63: error: return-statement with a value, in function returning 'void' [-fpermissive]
     return impl.CLANG_TEMPLATE operator()< Visitor >( visitor );
                                                               ^
/home/mart/Elastix/elastix/Testing/itkGPUResampleImageFilterTest.cxx:1202:1:   required from here
/home/mart/Elastix/elastix/Common/TypeList.h:744:63: error: return-statement with a value, in function returning 'void' [-fpermissive]
/home/mart/Elastix/elastix/Common/TypeList.h: In instantiation of ‘void typelist::DualVisitDimension<TLeftTypeList, TRightTypeList, Dimension>::operator()(Visitor&) const [with Visitor = itk::GPUResampleImageFilterFactory2<typelist::TypeList<short int, typelist::NullType>, typelist::TypeList<short int, typelist::NullType>, OCLImageDims>; TLeftTypeList = typelist::TypeList<short int, typelist::NullType>; TRightTypeList = typelist::TypeList<short int, typelist::NullType>; unsigned int Dimension = 3]’:
/home/mart/Elastix/elastix/Common/OpenCL/Factories/itkGPUResampleImageFilterFactory.hxx:77:10:   required from ‘void itk::GPUResampleImageFilterFactory2<TTypeListIn, TTypeListOut, NDimensions>::Register3D() [with TTypeListIn = typelist::TypeList<short int, typelist::NullType>; TTypeListOut = typelist::TypeList<short int, typelist::NullType>; NDimensions = OCLImageDims]’
/home/mart/Elastix/elastix/Testing/itkGPUResampleImageFilterTest.cxx:1202:1:   required from here
/home/mart/Elastix/elastix/Common/TypeList.h:744:63: error: return-statement with a value, in function returning 'void' [-fpermissive]
[ 43%] Linking CXX executable ../bin/itkThinPlateSplineTransformPerformanceTest
[ 43%] Built target elxTransformParametersCompare
[ 43%] Built target itkThinPlateSplineTransformPerformanceTest
[ 43%] Linking CXX executable ../bin/itkBSplineTransformPointPerformanceTest
[ 43%] Linking CXX executable ../bin/elxInvertTransform
[ 43%] Built target itkBSplineTransformPointPerformanceTest
Testing/CMakeFiles/itkGPUResampleImageFilterTest.dir/build.make:62: recipe for target 'Testing/CMakeFiles/itkGPUResampleImageFilterTest.dir/itkGPUResampleImageFilterTest.cxx.o' failed
make[2]: *** [Testing/CMakeFiles/itkGPUResampleImageFilterTest.dir/itkGPUResampleImageFilterTest.cxx.o] Error 1
CMakeFiles/Makefile2:6377: recipe for target 'Testing/CMakeFiles/itkGPUResampleImageFilterTest.dir/all' failed
make[1]: *** [Testing/CMakeFiles/itkGPUResampleImageFilterTest.dir/all] Error 2
[ 44%] Linking CXX executable ../bin/itkAdvancedLinearInterpolatorTest
[ 44%] Built target elxInvertTransform
[ 45%] Linking CXX executable ../bin/itkAdvancedBSplineDeformableTransformTest
[ 45%] Built target itkAdvancedBSplineDeformableTransformTest
[ 45%] Built target itkAdvancedLinearInterpolatorTest
[ 45%] Linking CXX executable ../bin/itkAdvancedRecursiveBSplineTransformTest
[ 45%] Built target itkAdvancedRecursiveBSplineTransformTest
[ 45%] Linking CXX executable ../bin/itkCompareCompositeTransformsTest
[ 45%] Built target itkCompareCompositeTransformsTest
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

What am I doing wrong?

Thanks for any help!

mavanb avatar Jan 28 '19 16:01 mavanb

@dpshamonin do you have any inputs?

kaspermarstal avatar Jan 29 '19 10:01 kaspermarstal

@N-Dekker Would you have time to have a look later?

mstaring avatar Feb 12 '19 08:02 mstaring

I believe I have the same issue:

Testing/CMakeFiles/itkGPUResampleImageFilterTest.dir/build.make:62: recipe for target 'Testing/CMakeFiles/itkGPUResampleImageFilterTest.dir/itkGPUResampleImageFilterTest.cxx.o' failed
make[2]: *** [Testing/CMakeFiles/itkGPUResampleImageFilterTest.dir/itkGPUResampleImageFilterTest.cxx.o] Error 1
CMakeFiles/Makefile2:6067: recipe for target 'Testing/CMakeFiles/itkGPUResampleImageFilterTest.dir/all' failed
make[1]: *** [Testing/CMakeFiles/itkGPUResamplTesting/CMakeFiles/itkGPUResampleImageFilterTest.dir/build.make:62: recipe for target 'Testing/CMakeFiles/itkGPUResampleImageFilterTest.dir/itkGPUResampleImageFilterTest.cxx.o' failed
make[2]: *** [Testing/CMakeFiles/itkGPUResampleImageFilterTest.dir/itkGPUResampleImageFilterTest.cxx.o] Error 1
CMakeFiles/Makefile2:6067: recipe for target 'Testing/CMakeFiles/itkGPUResampleImageFilterTest.dir/all' failed
make[1]: *** [Testing/CMakeFiles/itkGPUResampleImageFilterTest.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2
eImageFilterTest.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

Ubuntu 18.04 cmake version 3.10.2 gcc/g++ 7.3.0 OpenCL 1.2 CUDA 9.1.84 GeForce GTX 1080 Ti (Driver 390.77) ITK 4.13 (ITK_USE_GPU = OFF)

-USE_OpenCLFixedGenericPyramid=OFF -USE_OpenCLMovingGenericPyramid=OFF -USE_OpenCLResampler=OFF

(I had to disable the 3 above as I would always have issues with pyramyids).

Interestingly building would fail at a different part if I downgraded to gcc/g++ 4.8. Perhaps I should try a different cmake version...

@mavanb did you manage to find a fix?

vzickus avatar Mar 06 '19 14:03 vzickus

@kaspermarstal reported today that OpenCL build was working for him under Ubuntu 18.10 and GCC 7.3.0, CMake 3.10.2 with ITK5 development branch. Fixed with 07bb702e930f0500af61b8d9271b6f8e21b11016

dpshamonin avatar Mar 06 '19 14:03 dpshamonin

@vzickus Can you rebuild with make (i.e. not make -j4so that the build stops when the error is encountered) and post the error message if 07bb702 is not working for you?

kaspermarstal avatar Mar 06 '19 14:03 kaspermarstal

@dpshamonin I will try to upgrade to ITK5 devel if https://github.com/SuperElastix/elastix/commit/07bb702e930f0500af61b8d9271b6f8e21b11016 does not help (It seems my Elastix development branch was not up to date despite git telling me it was.)

vzickus avatar Mar 06 '19 17:03 vzickus

@dpshamonin @kaspermarstal OK, haven't tested Elastix itself yet, but can confirm I was able to build and install for my previously stated configuration, but also for:

-USE_OpenCLFixedGenericPyramid=ON -USE_OpenCLMovingGenericPyramid=ON -USE_OpenCLResampler=ON

Thank you for your help!

vzickus avatar Mar 06 '19 17:03 vzickus

@dpshamonin @kaspermarstal OK, haven't tested Elastix itself yet, but can confirm I was able to build and install for my previously stated configuration, but also for:

-USE_OpenCLFixedGenericPyramid=ON -USE_OpenCLMovingGenericPyramid=ON -USE_OpenCLResampler=ON

Thank you for your help!

Hi, vzickus, have you already tested your elastix? I can compile and install openCL-elastix without errors reported. However, when I use elastix, it reports that all openCL-related components are not installed.

sxl1174 avatar Mar 24 '19 04:03 sxl1174

@sxl1174 I can compile and install, however, I cannot actually use it, see #70

vzickus avatar Mar 25 '19 11:03 vzickus

Thank you for your reply,

@sxl1174 I can compile and install, however, I cannot actually use it, see #70

It's sad to hear about this. Still thank you though.

sxl1174 avatar Mar 25 '19 15:03 sxl1174

@sxl1174 Did you try testing if OpenCL works in general on your machine?

vzickus avatar Mar 25 '19 15:03 vzickus

I was thinking that perhaps there is something wrong with my GPU drivers, so I tested it with pyopencl, and it was working fine. However, if I try to use elastix, I get the out of resources error, which reverts elastix back to using CPU.

vzickus avatar Mar 25 '19 16:03 vzickus

@sxl1174 Did you try testing if OpenCL works in general on your machine?

No. Haven't. Does OpenCL work on your machine? I mean your issue is, although OpenCL work on your machine, you still can't use elastix?

sxl1174 avatar Mar 25 '19 16:03 sxl1174

I was thinking that perhaps there is something wrong with my GPU drivers, so I tested it with pyopencl, and it was working fine. However, if I try to use elastix, I get the out of resources error, which reverts elastix back to using CPU.

Thank you. I will test my OpenCL and tell you the results. I really appreciate your reply.

sxl1174 avatar Mar 25 '19 16:03 sxl1174

Dear @vzickus @sxl1174 ,

I was wondering if you found out what caused these issues of maybe have a workaround? I've got the same issues...

Regards,

Hans

1989HD avatar Feb 17 '21 13:02 1989HD

Sorry Hans, I haven't solved this issue. Best, Shan

On Wed, Feb 17, 2021 at 8:21 AM 1989HD [email protected] wrote:

Dear @vzickus https://github.com/vzickus @sxl1174 https://github.com/sxl1174 ,

I was wondering if you found out what caused these issues of maybe have a workaround? I've got the same issues...

Regards,

Hans

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SuperElastix/elastix/issues/107#issuecomment-780551252, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALRVRH7NZMZL6RRJTT2KZ5TS7O7ELANCNFSM4GSZDPJQ .

-- Shan Ling | Biophotonics imaging laboratory | Department of Biomedical Engineering | Case Western Reserve University | Wickenden Building, 10900 Euclid Ave. | Cleveland, OH 44106 | E-mail: [email protected] | Telephone: (216) 633-7585

sxl1174 avatar Feb 17 '21 14:02 sxl1174

It looks that @vzickus managed to build Elastix with OpenCL. Are you (@mavanb, @sxl1174) still getting errors when building with the latest main branch? We have merged PRs #734 and #741 that fix the OpenCL pyramids and resampler (see issue #70).

ntatsisk avatar Oct 12 '22 11:10 ntatsisk