HYPRE cuda enabled, ex1 failed.

[skywalker610]

I installed the hypre in a official cuda 11 docker container, the cuda function is ok since I have tested the PETSc ksp solver with cuda in the same container.

I want to combine PETSc and hypre together, and plan to utilize the boomeramg preconditioner of hypre inside of PETSc, first I need to make sure that the cuda function is ok, so I run the ex1 under example file.

The whole process is :

hypre/src# ./configure --prefix=/opt/hypre/ --enable-shared --with-cuda=1 --with-cuda-home=/usr/local/cuda --enable-unified-memory --disable-fortran --with-gpu-arch="80" --with-debug hypre/src# make install hypre/src/examples# make ex1 hypre/src/examples# mpirun -np 2 ./ex1

However, I got an error as shown in the following:

[memory.c, 108] hypre_assert failed: 0 ex1: memory.c:108: void hypre_UnifiedMemset(void*, HYPRE_Int, size_t): Assertion `0' failed. CUDA ERROR (code = 700, an illegal memory access was encountered) at memory.c:108 [63d3d253d1d6:38937] *** Process received signal *** [63d3d253d1d6:38937] Signal: Aborted (6) [63d3d253d1d6:38937] Signal code: (-6) [63d3d253d1d6:38937] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x46210)[0x7fb2c335a210] [63d3d253d1d6:38937] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7fb2c335a18b] [63d3d253d1d6:38937] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7fb2c3339859] [63d3d253d1d6:38937] [ 3] /lib/x86_64-linux-gnu/libc.so.6(+0x25729)[0x7fb2c3339729] [63d3d253d1d6:38937] [ 4] /lib/x86_64-linux-gnu/libc.so.6(+0x36f36)[0x7fb2c334af36] [63d3d253d1d6:38937] [ 5] /opt/hypre/lib/libHYPRE-2.24.0.so(+0x51cda3)[0x7fb2c3b73da3] [63d3d253d1d6:38937] [ 6] /opt/hypre/lib/libHYPRE-2.24.0.so(+0x51d2a7)[0x7fb2c3b742a7] [63d3d253d1d6:38937] [ 7] /opt/hypre/lib/libHYPRE-2.24.0.so(+0x51d470)[0x7fb2c3b74470] [63d3d253d1d6:38937] [ 8] /opt/hypre/lib/libHYPRE-2.24.0.so(hypre_CAlloc+0x39)[0x7fb2c3b75210] [63d3d253d1d6:38937] [ 9] /opt/hypre/lib/libHYPRE-2.24.0.so(hypre_InitializeCommunication+0xd94)[0x7fb2c3b35544] [63d3d253d1d6:38937] [10] /opt/hypre/lib/libHYPRE-2.24.0.so(hypre_StructMatrixAssemble+0x5d6)[0x7fb2c3b3c676] [63d3d253d1d6:38937] [11] ./ex1(+0x2e7b)[0x557dd7477e7b] [63d3d253d1d6:38937] [12] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fb2c333b0b3] [63d3d253d1d6:38937] [13] ./ex1(+0x24ee)[0x557dd74774ee] [63d3d253d1d6:38937] *** End of error message ***

The content of line 108 in src/utilities/memory.c is:

#if defined(HYPRE_USING_CUDA) HYPRE_CUDA_CALL( cudaMemset(ptr, value, num) );

[liruipeng]

@skywalker610 Can you try

make use_cuda=1

? It works for me on V100 with CUDA 10 Thanks!

[lahwaacz]

Does this mean that when Hypre is built with CUDA, any code using Hypre without CUDA will fail?

The Makefile for Hypre examples should be adjusted to detect if Hypre was built with CUDA or not. The use_cuda=1 Makefile option is not user-friendly.