top-level continuation 'name' got free def

[richardmembarth]

We get this assertion when compiling the traversal for GPU (nvvm & amdgpu). When commenting out unused code in the intrinsics_amdgpu.impala (amdgcn_atomic_* lines 81 - 93), the code compiles for nvvm and amdgpu as GPU_BACKEND.

Branches:

• thorin -> llvm_40
• impala -> llvm_40
• runtime -> llvm_40 @ 6904becdc7d7d0f34805c7b3685ab5265d6cc58e
• traversal -> amdgpu

[leissa]

sorry. I can't reproduce:

$ make VERBOSE=1
/usr/bin/cmake -H/home/roland/anydsl_no_llvm/traversal -B/home/roland/anydsl_no_llvm/traversal/build --check-build-system CMakeFiles/Makefile.cmake 0
Re-run cmake file: Makefile older than: ../CMakeLists.txt
-- Selected GPU backend: nvvm
-- Found llvm-as: /usr/bin/llvm-as
-- Note: llvm-as version needs to match the required LLVM bitcode version from NVVM.
-- Python found: 3.6.1
-- Configuring done
-- Generating done
-- Build files have been written to: /home/roland/anydsl_no_llvm/traversal/build
/usr/bin/cmake -E cmake_progress_start /home/roland/anydsl_no_llvm/traversal/build/CMakeFiles /home/roland/anydsl_no_llvm/traversal/build/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory '/home/roland/anydsl_no_llvm/traversal/build'
make -f src/CMakeFiles/gen_shadow.dir/build.make src/CMakeFiles/gen_shadow.dir/depend
make[2]: Entering directory '/home/roland/anydsl_no_llvm/traversal/build'
cd /home/roland/anydsl_no_llvm/traversal/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/roland/anydsl_no_llvm/traversal /home/roland/anydsl_no_llvm/traversal/src /home/roland/anydsl_no_llvm/traversal/build /home/roland/anydsl_no_llvm/traversal/build/src /home/roland/anydsl_no_llvm/traversal/build/src/CMakeFiles/gen_shadow.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/home/roland/anydsl_no_llvm/traversal/build'
make -f src/CMakeFiles/gen_shadow.dir/build.make src/CMakeFiles/gen_shadow.dir/build
make[2]: Entering directory '/home/roland/anydsl_no_llvm/traversal/build'
make[2]: Nothing to be done for 'src/CMakeFiles/gen_shadow.dir/build'.
make[2]: Leaving directory '/home/roland/anydsl_no_llvm/traversal/build'
[  5%] Built target gen_shadow
make -f src/CMakeFiles/gen_random.dir/build.make src/CMakeFiles/gen_random.dir/depend
make[2]: Entering directory '/home/roland/anydsl_no_llvm/traversal/build'
cd /home/roland/anydsl_no_llvm/traversal/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/roland/anydsl_no_llvm/traversal /home/roland/anydsl_no_llvm/traversal/src /home/roland/anydsl_no_llvm/traversal/build /home/roland/anydsl_no_llvm/traversal/build/src /home/roland/anydsl_no_llvm/traversal/build/src/CMakeFiles/gen_random.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/home/roland/anydsl_no_llvm/traversal/build'
make -f src/CMakeFiles/gen_random.dir/build.make src/CMakeFiles/gen_random.dir/build
make[2]: Entering directory '/home/roland/anydsl_no_llvm/traversal/build'
make[2]: Nothing to be done for 'src/CMakeFiles/gen_random.dir/build'.
make[2]: Leaving directory '/home/roland/anydsl_no_llvm/traversal/build'
[ 11%] Built target gen_random
make -f src/CMakeFiles/fbufdiff.dir/build.make src/CMakeFiles/fbufdiff.dir/depend
make[2]: Entering directory '/home/roland/anydsl_no_llvm/traversal/build'
cd /home/roland/anydsl_no_llvm/traversal/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/roland/anydsl_no_llvm/traversal /home/roland/anydsl_no_llvm/traversal/src /home/roland/anydsl_no_llvm/traversal/build /home/roland/anydsl_no_llvm/traversal/build/src /home/roland/anydsl_no_llvm/traversal/build/src/CMakeFiles/fbufdiff.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/home/roland/anydsl_no_llvm/traversal/build'
make -f src/CMakeFiles/fbufdiff.dir/build.make src/CMakeFiles/fbufdiff.dir/build
make[2]: Entering directory '/home/roland/anydsl_no_llvm/traversal/build'
make[2]: Nothing to be done for 'src/CMakeFiles/fbufdiff.dir/build'.
make[2]: Leaving directory '/home/roland/anydsl_no_llvm/traversal/build'
[ 16%] Built target fbufdiff
make -f src/CMakeFiles/fbuf2png.dir/build.make src/CMakeFiles/fbuf2png.dir/depend
make[2]: Entering directory '/home/roland/anydsl_no_llvm/traversal/build'
cd /home/roland/anydsl_no_llvm/traversal/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/roland/anydsl_no_llvm/traversal /home/roland/anydsl_no_llvm/traversal/src /home/roland/anydsl_no_llvm/traversal/build /home/roland/anydsl_no_llvm/traversal/build/src /home/roland/anydsl_no_llvm/traversal/build/src/CMakeFiles/fbuf2png.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/home/roland/anydsl_no_llvm/traversal/build'
make -f src/CMakeFiles/fbuf2png.dir/build.make src/CMakeFiles/fbuf2png.dir/build
make[2]: Entering directory '/home/roland/anydsl_no_llvm/traversal/build'
make[2]: Nothing to be done for 'src/CMakeFiles/fbuf2png.dir/build'.
make[2]: Leaving directory '/home/roland/anydsl_no_llvm/traversal/build'
[ 22%] Built target fbuf2png
make -f src/CMakeFiles/traversal_gpu-interface.dir/build.make src/CMakeFiles/traversal_gpu-interface.dir/depend
make[2]: Entering directory '/home/roland/anydsl_no_llvm/traversal/build'
cd /home/roland/anydsl_no_llvm/traversal/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/roland/anydsl_no_llvm/traversal /home/roland/anydsl_no_llvm/traversal/src /home/roland/anydsl_no_llvm/traversal/build /home/roland/anydsl_no_llvm/traversal/build/src /home/roland/anydsl_no_llvm/traversal/build/src/CMakeFiles/traversal_gpu-interface.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/home/roland/anydsl_no_llvm/traversal/build'
make -f src/CMakeFiles/traversal_gpu-interface.dir/build.make src/CMakeFiles/traversal_gpu-interface.dir/build
make[2]: Entering directory '/home/roland/anydsl_no_llvm/traversal/build'
[ 25%] Generating frontend/traversal_gpu.h
cd /home/roland/anydsl_no_llvm/traversal/src && /home/roland/anydsl_no_llvm/impala/build/bin/impala /home/roland/anydsl_no_llvm/runtime/src/runtime.impala /home/roland/anydsl_no_llvm/runtime/platforms/intrinsics.impala /home/roland/anydsl_no_llvm/runtime/platforms/intrinsics_cpu.impala /home/roland/anydsl_no_llvm/runtime/platforms/intrinsics_cuda.impala /home/roland/anydsl_no_llvm/runtime/platforms/intrinsics_nvvm.impala /home/roland/anydsl_no_llvm/runtime/platforms/intrinsics_opencl.impala /home/roland/anydsl_no_llvm/runtime/platforms/intrinsics_amdgpu.impala /home/roland/anydsl_no_llvm/runtime/platforms/intrinsics_thorin.impala /home/roland/anydsl_no_llvm/traversal/src/mappings/intrinsics_nvvm.impala /home/roland/anydsl_no_llvm/traversal/src/mappings/mapping_gpu.impala /home/roland/anydsl_no_llvm/traversal/src/common/debug.impala /home/roland/anydsl_no_llvm/traversal/src/common/float.impala /home/roland/anydsl_no_llvm/traversal/src/common/intersection.impala /home/roland/anydsl_no_llvm/traversal/src/common/stack.impala /home/roland/anydsl_no_llvm/traversal/src/common/range.impala /home/roland/anydsl_no_llvm/traversal/src/common/vector.impala /home/roland/anydsl_no_llvm/traversal/src/common/transform.impala /home/roland/anydsl_no_llvm/traversal/src/common/transparency.impala /home/roland/anydsl_no_llvm/traversal/src/common/instancing.impala /home/roland/anydsl_no_llvm/traversal/src/common/traversal.impala -emit-c-interface -o frontend/traversal_gpu
make[2]: Leaving directory '/home/roland/anydsl_no_llvm/traversal/build'
[ 25%] Built target traversal_gpu-interface
make -f src/CMakeFiles/traversal_gpu.dir/build.make src/CMakeFiles/traversal_gpu.dir/depend
make[2]: Entering directory '/home/roland/anydsl_no_llvm/traversal/build'
[ 27%] Generating traversal_gpu.ll
cd /home/roland/anydsl_no_llvm/traversal/build/src && /home/roland/anydsl_no_llvm/impala/build/bin/impala /home/roland/anydsl_no_llvm/runtime/src/runtime.impala /home/roland/anydsl_no_llvm/runtime/platforms/intrinsics.impala /home/roland/anydsl_no_llvm/runtime/platforms/intrinsics_cpu.impala /home/roland/anydsl_no_llvm/runtime/platforms/intrinsics_cuda.impala /home/roland/anydsl_no_llvm/runtime/platforms/intrinsics_nvvm.impala /home/roland/anydsl_no_llvm/runtime/platforms/intrinsics_opencl.impala /home/roland/anydsl_no_llvm/runtime/platforms/intrinsics_amdgpu.impala /home/roland/anydsl_no_llvm/runtime/platforms/intrinsics_thorin.impala /home/roland/anydsl_no_llvm/traversal/src/mappings/intrinsics_nvvm.impala /home/roland/anydsl_no_llvm/traversal/src/mappings/mapping_gpu.impala /home/roland/anydsl_no_llvm/traversal/src/common/debug.impala /home/roland/anydsl_no_llvm/traversal/src/common/float.impala /home/roland/anydsl_no_llvm/traversal/src/common/intersection.impala /home/roland/anydsl_no_llvm/traversal/src/common/stack.impala /home/roland/anydsl_no_llvm/traversal/src/common/range.impala /home/roland/anydsl_no_llvm/traversal/src/common/vector.impala /home/roland/anydsl_no_llvm/traversal/src/common/transform.impala /home/roland/anydsl_no_llvm/traversal/src/common/transparency.impala /home/roland/anydsl_no_llvm/traversal/src/common/instancing.impala /home/roland/anydsl_no_llvm/traversal/src/common/traversal.impala -emit-llvm -g --log-level info -o traversal_gpu
warning: built without LLVM support - I don't emit an LLVM file
cd /home/roland/anydsl_no_llvm/traversal/build/src && /usr/bin/python /home/roland/anydsl_no_llvm/runtime/post-patcher.py /home/roland/anydsl_no_llvm/traversal/build/src/traversal_gpu
cd /home/roland/anydsl_no_llvm/traversal/build/src && /usr/bin/cmake -D_irfile=/home/roland/anydsl_no_llvm/traversal/build/src/traversal_gpu.nvvm -DLLVM_AS_BIN=/usr/bin/llvm-as -P /home/roland/anydsl_no_llvm/runtime/cmake/check_llvmir.cmake
cd /home/roland/anydsl_no_llvm/traversal/build/src && /usr/bin/cmake -D_basename=traversal_gpu -P /home/roland/anydsl_no_llvm/runtime/cmake/check_amdgpuir.cmake
[ 30%] Generating traversal_gpu.o
cd /home/roland/anydsl_no_llvm/traversal/build/src && /usr/bin/clang -g -mavx2 -mavx -mfma -ffast-math -c -o /home/roland/anydsl_no_llvm/traversal/build/src/traversal_gpu.o /home/roland/anydsl_no_llvm/traversal/build/src/traversal_gpu.ll
clang-4.0: error: no such file or directory: '/home/roland/anydsl_no_llvm/traversal/build/src/traversal_gpu.ll'
clang-4.0: error: no input files
make[2]: *** [src/CMakeFiles/traversal_gpu.dir/build.make:62: src/traversal_gpu.o] Error 1
make[2]: Leaving directory '/home/roland/anydsl_no_llvm/traversal/build'
make[1]: *** [CMakeFiles/Makefile2:368: src/CMakeFiles/traversal_gpu.dir/all] Error 2
make[1]: Leaving directory '/home/roland/anydsl_no_llvm/traversal/build'
make: *** [Makefile:84: all] Error 2

I'm using LLVM=false. Maybe the error occurs later on?

[richardmembarth]

The hash was on the wrong branch, I've updated my previous post.

[pgrit]

I have the same (or similar) issue on master:

With everything in Debug mode (thorin, impala, runtime, and LLVM), the assertion is triggered when compiling the CPU version of the traversal. The GPU version works (compiling and running).

The behaviour in Release mode (thorin, impala, runtime, and LLVM) is different: Both versions compile, but the GPU version crashes during execution.

To trigger this bug, use the master branch used at the time of this writing, which is:

• Impala: 39ef64533899df27dbb4c0d672a86591627d4023
• Thorin: 4bf40a233d315114fba72351df1ebc0bb2b00ff3
• Runtime: b0d9c50e743d0b53116f01bbcd64d9f1988e60c4
• Traversal: b02903d0933efba52b54400f3e6948f5ef0e5c87

[pgrit]

The issue goes away when using the lower2cff_pe branch of thorin, and compiling thorin in Debug mode. When compiling thorin in Release mode (on that branch), and impala in Debug mode, the following assertion is triggered for both the CPU and the GPU variant:

impala: /home/grittmann/anydsl/thorin/src/thorin/continuation.h:122: const thorin::Param* thorin::Continuation::param(size_t) const: Assertion `i < num_params()' failed.

[madmann91]

This assertion is a known issue. Impala and Thorin have to be built both in Release mode or both in Debug mode.

[pgrit]

With both Thorin and Impala in Release, Impala seg-faults when compiling the traversal for old gpus, i.e. specifically when replacing the nvvm_ldg4_i32 in line 93 of mapping_gpu.impala. Replacing any combination of the nvvm_ldg4_f32 intrinsics works.