AMGX icon indicating copy to clipboard operation
AMGX copied to clipboard

An issue from Fraunhofer-Chalmers Centre: Unit tests

Open bnase opened this issue 7 years ago • 11 comments

Which unit tests are relevant and up to date? We have extensively test on different GPUs, CUDA 8 vs 9.0 and quite a few keep failing under Linux. Besides the missing matrices, which we have filled in from either Matrix Market or proper use of generate_poisson, there are some tests that seem basic and keep failing. Find attached the logs of unit tests on an array of GPUs and configurations.

gtx970_9.0.log gtx970.log gtx1080ti.log gtx660ti.log

All unit tests were compiled with CUDA 8.0, except gtx_970_9.0.log. The tests performed on 660ti were done after we have changed some (possibly too strict? check #8) hardcoded CUDA_ARCH>=3.5 to CUDA_ARCH>=3.0.

We will be more than happy to fix them, as long as we have a list of the tests that are supposed to work, because some seem very outdated or poorly maintained.

bnase avatar Dec 07 '17 11:12 bnase

I will review those unit tests. Thanks for report.

marsaev avatar Dec 15 '17 12:12 marsaev

Hi,

I am experiencing numerous unit test failures similarly to what reported above, plus the test runner reports information about detected memory leaks, and my screen goes black after few minutes the tests are running. I need to switch off and restart the computer to resume operation.

My configuration is as follows:

  • Windows 10 64bit
  • Visual Studio 2015
  • CMake 3.4.2
  • CUDA 9.1
  • Titan V
  • Driver 288.59 (I will upgrade to 390.65 and repeat the tests)

After launching amgx_tests_launcher.exe I get these errors on screen:

D:\NES_Libs\AmgX-bin\Release>amgx_tests_launcher.exe > tests_output.txt !!! detected some memory leaks in the code: trying to free non-empty temporary device pool !!! ptr: 000000072500A000 size: 4096 !!! detected some memory leaks in the code: trying to free non-empty temporary device pool !!! ptr: 000000072280A000 size: 4096 !!! detected some memory leaks in the code: trying to free non-empty temporary device pool !!! ptr: 000000072240A000 size: 4096 !!! detected some memory leaks in the code: trying to free non-empty temporary device pool !!! ptr: 000000072280A000 size: 4096 !!! detected some memory leaks in the code: trying to free non-empty temporary device pool !!! ptr: 000000072240A000 size: 4096 !!! detected some memory leaks in the code: trying to free non-empty temporary device pool !!! ptr: 000000072280A000 size: 4096


Numerous unit tests fail as per output attached below.

AmgX is potentially the best option for linear sparse solving on NVIDIA GPUs, thanks for releasing it publicly! It would be great if these issues can be smoothed out.

-------- ATTACHED TESTS OUTPUT -----------------

Spawning new worker [ 1/ 183] AggregatesCoarseGeneratorTest dDDI : [PASSED] [ 2/ 183] AggregatesCoarseGeneratorTest dDFI : [PASSED] [ 3/ 183] AggregatesCoarseGeneratorTest dFFI : [PASSED] [ 4/ 183] AggregatesCoarseningFactor dDDI : [PASSED] [ 5/ 183] AggregatesCoarseningFactor dDFI : [PASSED] [ 6/ 183] AggregatesCoarseningFactor dFFI : [PASSED] [ 7/ 183] AggregatesDeterminism dDDI : [PASSED] [ 8/ 183] AggregatesDeterminism dDFI : [PASSED] [ 9/ 183] AggregatesDeterminism dFFI : [PASSED] [ 10/ 183] AggregatesDiagonalOutside dDDI : [FAILED] Spawning new worker Launching test AggregatesDiagonalOutside dDDI in the new process for the second time [ 1/ 174] AggregatesDiagonalOutside dDDI : [FAILED] [ 2/ 174] AggregatesDiagonalOutside dDFI : [FAILED] Spawning new worker Launching test AggregatesDiagonalOutside dDFI in the new process for the second time [ 1/ 173] AggregatesDiagonalOutside dDFI : [FAILED] [ 2/ 173] AggregatesDiagonalOutside dFFI : [FAILED] Spawning new worker Launching test AggregatesDiagonalOutside dFFI in the new process for the second time [ 1/ 172] AggregatesDiagonalOutside dFFI : [FAILED] [ 2/ 172] AmgLevelsReuse dDDI : Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

AMGX version 2.0.0.130-opensource Built on Jan 11 2018, 15:28:36 Compiled with CUDA Runtime 9.1, using CUDA driver 9.1 Cannot read file as JSON object, trying as AMGX config Cannot read file as JSON object, trying as AMGX config Total Time: 0.0439724 setup: 0.03917 s solve: 0.00480237 s solve(per iteration): 0.00240118 s Total Time: 0.0383374 setup: 0.0338131 s solve: 0.00452429 s solve(per iteration): 0.00226214 s AMGX version 2.0.0.130-opensource Built on Jan 11 2018, 15:28:36 Compiled with CUDA Runtime 9.1, using CUDA driver 9.1 Cannot read file as JSON object, trying as AMGX config Cannot read file as JSON object, trying as AMGX config Total Time: 0.0406164 setup: 0.0364431 s solve: 0.00417322 s solve(per iteration): 0.00208661 s Total Time: 0.0388261 setup: 0.034298 s solve: 0.00452806 s solve(per iteration): 0.00226403 s AMGX version 2.0.0.130-opensource Built on Jan 11 2018, 15:28:36 Compiled with CUDA Runtime 9.1, using CUDA driver 9.1 Cannot read file as JSON object, trying as AMGX config Cannot read file as JSON object, trying as AMGX config Total Time: 0.0446308 setup: 0.0354959 s solve: 0.00913485 s solve(per iteration): 0.00456742 s Total Time: 0.0376344 setup: 0.0328809 s solve: 0.0047535 s solve(per iteration): 0.00237675 s AMGX version 2.0.0.130-opensource Built on Jan 11 2018, 15:28:36 Compiled with CUDA Runtime 9.1, using CUDA driver 9.1 Cannot read file as JSON object, trying as AMGX config Cannot read file as JSON object, trying as AMGX config Total Time: 0.040341 setup: 0.0357601 s solve: 0.00458093 s solve(per iteration): 0.00229046 s Total Time: 0.0406462 setup: 0.0308963 s solve: 0.00974989 s solve(per iteration): 0.00487494 s AMGX version 2.0.0.130-opensource Built on Jan 11 2018, 15:28:36 Compiled with CUDA Runtime 9.1, using CUDA driver 9.1 Cannot read file as JSON object, trying as AMGX config Cannot read file as JSON object, trying as AMGX config Total Time: 0.0367517 setup: 0.0325652 s solve: 0.0041865 s solve(per iteration): 0.00209325 s Total Time: 0.0271621 setup: 0.0227314 s solve: 0.00443078 s solve(per iteration): 0.00221539 s AMGX version 2.0.0.130-opensource Built on Jan 11 2018, 15:28:36 Compiled with CUDA Runtime 9.1, using CUDA driver 9.1 Cannot read file as JSON object, trying as AMGX config Cannot read file as JSON object, trying as AMGX config Total Time: 0.0362045 setup: 0.0320297 s solve: 0.00417475 s solve(per iteration): 0.00208738 s Total Time: 0.0229544 setup: 0.0189357 s solve: 0.00401869 s solve(per iteration): 0.00200934 s AMGX version 2.0.0.130-opensource Built on Jan 11 2018, 15:28:36 Compiled with CUDA Runtime 9.1, using CUDA driver 9.1 Cannot read file as JSON object, trying as AMGX config Cannot read file as JSON object, trying as AMGX config Total Time: 0.0357202 setup: 0.0315331 s solve: 0.00418717 s solve(per iteration): 0.00209358 s Total Time: 0.0207047 setup: 0.0166849 s solve: 0.00401987 s solve(per iteration): 0.00200994 s AMGX version 2.0.0.130-opensource Built on Jan 11 2018, 15:28:36 Compiled with CUDA Runtime 9.1, using CUDA driver 9.1 Cannot read file as JSON object, trying as AMGX config Cannot read file as JSON object, trying as AMGX config Total Time: 0.0367149 setup: 0.0325745 s solve: 0.00414042 s solve(per iteration): 0.00207021 s Total Time: 0.0201124 setup: 0.0157552 s solve: 0.00435718 s solve(per iteration): 0.00217859 s AMGX version 2.0.0.130-opensource Built on Jan 11 2018, 15:28:36 Compiled with CUDA Runtime 9.1, using CUDA driver 9.1 Cannot read file as JSON object, trying as AMGX config Cannot read file as JSON object, trying as AMGX config Total Time: 0.0385637 setup: 0.0329216 s solve: 0.00564211 s solve(per iteration): 0.00282106 s Total Time: 0.0179537 setup: 0.0139469 s solv[PASSED] [ 3/ 172] BlockConversionTest hDDI : [FAILED] e: 0.00400672 s solve(per iteration): 0.00200336 s AMGX version 2.0.0.130-opensource Built on Jan 11 2018, 15:28:36 Compiled with CUDA Runtime 9.1, using CUDA driver 9.1 Cannot read file as JSON object, trying as AMGX config Cannot read file as JSON object, trying as AMGX config Total Time: 0.0669452 setup: 0.0621353 s solve: 0.00480992 s solve(per iteration): 0.00240496 s Total Time: 0.0152904 setup: 0.0113075 s solve: 0.00398291 s solve(per iteration): 0.00199146 s AMGX version 2.0.0.130-opensource Built on Jan 11 2018, 15:28:36 Compiled with CUDA Runtime 9.1, using CUDA driver 9.1 Spawning new worker Launching test BlockConversionTest hDDI in the new process for the second time [ 1/ 170] BlockConversionTest hDDI : [FAILED] [ 2/ 170] BlockConversionTest hDFI : [FAILED] Spawning new worker Launching test BlockConversionTest hDFI in the new process for the second time [ 1/ 169] BlockConversionTest hDFI : [FAILED] [ 2/ 169] BlockConversionTest hFFI : [FAILED] Spawning new worker Launching test BlockConversionTest hFFI in the new process for the second time [ 1/ 168] BlockConversionTest hFFI : [FAILED] [ 2/ 168] CAPIFailure dDDI : [FAILED] Error opening file 'nonexisting_file.cfg' Error opening file 'nonexisting_file.cfg' Error parsing parameter file: nonexisting_file.cfg Error opening file 'nonexisting_file.cfg' Error opening file 'nonexisting_file.cfg' Error parsing parameter file: nonexisting_file.cfg Error opening file 'nonexisting_file.cfg' Error opening file 'nonexisting_file.cfg' Error parsing parameter file: nonexisting_file.cfg Cannot read file as JSON object, trying as AMGX config Converting config string to current config version Parsing configuration string: determinism_flag=1 ; Error parsing parameter string: Variable 'determinism_flag' not registered

Error parsing parameter string: Error parsing parameter string: determinism_flag=1 ;

Spawning new worker Launching test CAPIFailure dDDI in the new process for the second time [ 1/ 167] CAPIFailure dDDI : [FAILED] [ 2/ 167] CAPIVersionCheck dDDI : [PASSED] [ 3/ 167] ClassicalStrengthAffinityTest dDDI : [PASSED] [ 4/ 167] ClassicalStrengthAffinityTest dDFI : [PASSED] [ 5/ 167] ClassicalStrengthAffinityTest dFFI : [PASSED] [ 6/ 167] ClassicalStrengthTest hDDI : [PASSED] [ 7/ 167] ClassicalStrengthTest hDFI : [PASSED] [ 8/ 167] ClassicalStrengthTest hFFI : [PASSED] [ 9/ 167] ClassicalStrengthTest dDDI : [PASSED] [ 10/ 167] ClassicalStrengthTest dDFI : [PASSED] [ 11/ 167] ClassicalStrengthTest dFFI : [PASSED] [ 12/ 167] ConfigStringParsing hDDI : [PASSED] [ 13/ 167] CsrMultiplyTests_Poisson27_100_100 dDDI : [PASSED] [ 14/ 167] CsrMultiplyTests_Poisson27_100_100 dFFI : [PASSED] [ 15/ 167] CsrMultiplyTests_Poisson27_10_10 dDDI : [PASSED] [ 16/ 167] CsrMultiplyTests_Poisson27_10_10 dFFI : [PASSED] [ 17/ 167] CsrMultiplyTests_Poisson5_100_100 dDDI : [PASSED] [ 18/ 167] CsrMultiplyTests_Poisson5_100_100 dFFI : [PASSED] [ 19/ 167] CsrMultiplyTests_Poisson5_10_10 dDDI : [PASSED] [ 20/ 167] CsrMultiplyTests_Poisson5_10_10 dFFI : [PASSED] [ 21/ 167] CsrMultiplyTests_Poisson7_100_100 dDDI : [PASSED] [ 22/ 167] CsrMultiplyTests_Poisson7_100_100 dFFI : [PASSED] [ 23/ 167] CsrMultiplyTests_Poisson7_10_10 dDDI : [PASSED] [ 24/ 167] CsrMultiplyTests_Poisson7_10_10 dFFI : [PASSED] [ 25/ 167] CsrMultiplyTests_Poisson9_100_100 dDDI : [PASSED] [ 26/ 167] CsrMultiplyTests_Poisson9_100_100 dFFI : [PASSED] [ 27/ 167] CsrMultiplyTests_Poisson9_10_10 dDDI : [PASSED] [ 28/ 167] CsrMultiplyTests_Poisson9_10_10 dFFI : [PASSED] [ 29/ 167] CsrSparsityILU1Tests_Poisson27_100_100 dDDI : [PASSED] [ 30/ 167] CsrSparsityILU1Tests_Poisson27_10_10 dDDI : [PASSED] [ 31/ 167] CsrSparsityILU1Tests_Poisson5_100_100 dDDI : [PASSED] [ 32/ 167] CsrSparsityILU1Tests_Poisson5_10_10 dDDI : [PASSED] [ 33/ 167] CsrSparsityILU1Tests_Poisson7_100_100 dDDI : [PASSED] [ 34/ 167] CsrSparsityILU1Tests_Poisson7_10_10 dDDI : [PASSED] [ 35/ 167] CsrSparsityILU1Tests_Poisson9_100_100 dDDI : [PASSED] [ 36/ 167] CsrSparsityILU1Tests_Poisson9_10_10 dDDI : [PASSED] [ 37/ 167] CsrSparsityTests_Poisson27_100_100 dDDI : [PASSED] [ 38/ 167] CsrSparsityTests_Poisson27_10_10 dDDI : [PASSED] [ 39/ 167] CsrSparsityTests_Poisson5_100_100 dDDI : [PASSED] [ 40/ 167] CsrSparsityTests_Poisson5_10_10 dDDI : [PASSED] [ 41/ 167] CsrSparsityTests_Poisson7_100_100 dDDI : [PASSED] [ 42/ 167] CsrSparsityTests_Poisson7_10_10 dDDI : [PASSED] [ 43/ 167] CsrSparsityTests_Poisson9_100_100 dDDI : [PASSED] [ 44/ 167] CsrSparsityTests_Poisson9_10_10 dDDI : [PASSED] [ 45/ 167] DenseLUSolverTest_Factorization_Id_256 dDDI : [PASSED] [ 46/ 167] DenseLUSolverTest_Factorization_Id_256 dFFI : [PASSED] [ 47/ 167] DenseLUSolverTest_Factorization_Id_32 dDDI : [PASSED] [ 48/ 167] DenseLUSolverTest_Factorization_Id_32 dFFI : [PASSED] [ 49/ 167] DenseLUSolverTest_Solve_Id_256 dDDI : [PASSED] [ 50/ 167] DenseLUSolverTest_Solve_Id_256 dFFI : [PASSED] [ 51/ 167] DenseLUSolverTest_Solve_Id_32 dDDI : [PASSED] [ 52/ 167] DenseLUSolverTest_Solve_Id_32 dFFI : [PASSED] [ 53/ 167] DenseLUSolverTest_Solve_Poisson3D dDDI : [PASSED] [ 54/ 167] DiagonalMatrix dDDI : CR: numrows = 6000, numCoarse = 6000 Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

[FAILED] Error opening file 'nonexisting_file.cfg' Error opening file 'nonexisting_file.cfg' Error parsing parameter file: nonexisting_file.cfg Error opening file 'nonexisting_file.cfg' Error opening file 'nonexisting_file.cfg' Error parsing parameter file: nonexisting_file.cfg Error opening file 'nonexisting_file.cfg' Error opening file 'nonexisting_file.cfg' Error parsing parameter file: nonexisting_file.cfg Cannot read file as JSON object, trying as AMGX config Converting config string to current config version Parsing configuration string: determinism_flag=1 ; Error parsing parameter string: Variable 'determinism_flag' not registered

Error parsing parameter string: Error parsing parameter string: determinism_flag=1 ;

Spawning new worker Launching test DiagonalMatrix dDDI in the new process for the second time [ 1/ 114] DiagonalMatrix dDDI : CR: numrows = 6000, numCoarse = 6000 Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

[FAILED] [ 2/ 114] DiagonalMatrix dDFI : CR: numrows = 6000, numCoarse = 6000 Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

[FAILED] Spawning new worker Launching test DiagonalMatrix dDFI in the new process for the second time [ 1/ 113] DiagonalMatrix dDFI : CR: numrows = 6000, numCoarse = 6000 Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

[FAILED] [ 2/ 113] DiagonalMatrix dFFI : CR: numrows = 6000, numCoarse = 6000 Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

[FAILED] Spawning new worker Launching test DiagonalMatrix dFFI in the new process for the second time [ 1/ 112] DiagonalMatrix dFFI : CR: numrows = 6000, numCoarse = 6000 Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

Caught amgx exception: Error, setup must be called before calling solve at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:598 Stack trace:

[FAILED] [ 2/ 112] EigenSolverTest_InverseIteration dDDI : [FAILED] Spawning new worker Launching test EigenSolverTest_InverseIteration dDDI in the new process for the second time [ 1/ 111] EigenSolverTest_InverseIteration dDDI : [FAILED] [ 2/ 111] EigenSolverTest_PowerIteration dDDI : [FAILED] Spawning new worker Launching test EigenSolverTest_PowerIteration dDDI in the new process for the second time [ 1/ 110] EigenSolverTest_PowerIteration dDDI : [FAILED] [ 2/ 110] EnergyminAlgorithmTest dDDI : [FAILED] Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

AMGX version 2.0.0.130-opensource Built on Jan 11 2018, 15:28:36 Compiled with CUDA Runtime 9.1, using CUDA driver 9.1 Spawning new worker Launching test EnergyminAlgorithmTest dDDI in the new process for the second time [ 1/ 109] EnergyminAlgorithmTest dDDI : [FAILED] [ 2/ 109] ExplicitZeroValues dDDI : [FAILED] Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

AMGX version 2.0.0.130-opensource Built on Jan 11 2018, 15:28:36 Compiled with CUDA Runtime 9.1, using CUDA driver 9.1 Spawning new worker Launching test ExplicitZeroValues dDDI in the new process for the second time [ 1/ 108] ExplicitZeroValues dDDI : [FAILED] [ 2/ 108] ExplicitZeroValues dDFI : [FAILED] Spawning new worker Launching test ExplicitZeroValues dDFI in the new process for the second time [ 1/ 107] ExplicitZeroValues dDFI : [FAILED] [ 2/ 107] ExplicitZeroValues dFFI : [FAILED] Spawning new worker Launching test ExplicitZeroValues dFFI in the new process for the second time [ 1/ 106] ExplicitZeroValues dFFI : [FAILED] [ 2/ 106] FGMRESConvergencePoisson dDDI : [PASSED] [ 3/ 106] FGMRESConvergencePoisson dFFI : [PASSED] [ 4/ 106] FactoriesTest hDDI : [FAILED] Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

Spawning new worker Launching test FactoriesTest hDDI in the new process for the second time [ 1/ 103] FactoriesTest hDDI : [FAILED] [ 2/ 103] FactoriesTest hDFI : [FAILED] Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

Spawning new worker Launching test FactoriesTest hDFI in the new process for the second time [ 1/ 102] FactoriesTest hDFI : [FAILED] [ 2/ 102] FactoriesTest hFFI : [FAILED] Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

Spawning new worker Launching test FactoriesTest hFFI in the new process for the second time [ 1/ 101] FactoriesTest hFFI : [FAILED] [ 2/ 101] FactoriesTest dDDI : [FAILED] Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

Spawning new worker Launching test FactoriesTest dDDI in the new process for the second time [ 1/ 100] FactoriesTest dDDI : [FAILED] [ 2/ 100] FactoriesTest dDFI : [FAILED] Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

Spawning new worker Launching test FactoriesTest dDFI in the new process for the second time [ 1/ 99] FactoriesTest dDFI : [FAILED] [ 2/ 99] FactoriesTest dFFI : [FAILED] Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

Spawning new worker Launching test FactoriesTest dFFI in the new process for the second time [ 1/ 98] FactoriesTest dFFI : [FAILED] [ 2/ 98] GeneratedMatrixDistributedIOTest hDDI : [FAILED] Caught amgx exception: SolverFactory 'TEMPLATE_SOLVER' has not been registered

at: D:/NES_Libs/AmgX-src/base/src/solvers/solver.cu:1074 Stack trace:

Spawning new worker Launching test GeneratedMatrixDistributedIOTest hDDI in the new process for the second time [ 1/ 97] GeneratedMatrixDistributedIOTest hDDI : [FAILED] [ 2/ 97] GeneratedMatrixDistributedIOTest hDFI : [FAILED] Spawning new worker Launching test GeneratedMatrixDistributedIOTest hDFI in the new process for the second time [ 1/ 96] GeneratedMatrixDistributedIOTest hDFI : [FAILED] [ 2/ 96] GeneratedMatrixDistributedIOTest hFFI : [FAILED] Spawning new worker Launching test GeneratedMatrixDistributedIOTest hFFI in the new process for the second time [ 1/ 95] GeneratedMatrixDistributedIOTest hFFI : [FAILED] [ 2/ 95] GeneratedMatrixIOTest hDDI : [FAILED] Spawning new worker Launching test GeneratedMatrixIOTest hDDI in the new process for the second time [ 1/ 94] GeneratedMatrixIOTest hDDI : [FAILED] [ 2/ 94] GeneratedMatrixIOTest hDFI : [FAILED] Spawning new worker Launching test GeneratedMatrixIOTest hDFI in the new process for the second time [ 1/ 93] GeneratedMatrixIOTest hDFI : [FAILED] [ 2/ 93] GeneratedMatrixIOTest hFFI : [FAILED] Spawning new worker Launching test GeneratedMatrixIOTest hFFI in the new process for the second time [ 1/ 92] GeneratedMatrixIOTest hFFI : [FAILED] [ 2/ 92] GenericSpMVTest hDDI : [PASSED] [ 3/ 92] GenericSpMVTest hFFI : [PASSED] [ 4/ 92] GenericSpMVTest dDDI : [PASSED] [ 5/ 92] GenericSpMVTest dFFI : [PASSED] [ 6/ 92] IDRConvergencePoisson dDDI : [PASSED] [ 7/ 92] IDRConvergencePoisson dFFI : [PASSED] [ 8/ 92] IDRmsyncConvergencePoisson dDDI : [PASSED] [ 9/ 92] IDRmsyncConvergencePoisson dFFI : [PASSED] [ 10/ 92] ILU1_coloringA dDDI : [FAILED] Spawning new worker Launching test ILU1_coloringA dDDI in the new process for the second time [ 1/ 83] ILU1_coloringA dDDI : [FAILED] [ 2/ 83] ILU1_in_vs_out_diag dDDI : [FAILED] Spawning new worker Launching test ILU1_in_vs_out_diag dDDI in the new process for the second time [ 1/ 82] ILU1_in_vs_out_diag dDDI : [FAILED] [ 2/ 82] ILU_DILU_equivalence dDDI : [FAILED] Spawning new worker Launching test ILU_DILU_equivalence dDDI in the new process for the second time [ 1/ 81] ILU_DILU_equivalence dDDI : [FAILED] [ 2/ 81] ImplicitZeroInDiagonal dDDI : [FAILED] Spawning new worker Launching test ImplicitZeroInDiagonal dDDI in the new process for the second time [ 1/ 80] ImplicitZeroInDiagonal dDDI : [FAILED] [ 2/ 80] ImplicitZeroInDiagonal dDFI : [FAILED] Spawning new worker Launching test ImplicitZeroInDiagonal dDFI in the new process for the second time [ 1/ 79] ImplicitZeroInDiagonal dDFI : [FAILED] [ 2/ 79] ImplicitZeroInDiagonal dFFI : [FAILED] Spawning new worker Launching test ImplicitZeroInDiagonal dFFI in the new process for the second time [ 1/ 78] ImplicitZeroInDiagonal dFFI : [FAILED] [ 2/ 78] LargeMatricesSupport dDDI : [PASSED] [ 3/ 78] LargeMatricesSupport dDFI : [PASSED] [ 4/ 78] LargeMatricesSupport dFFI : [PASSED] [ 5/ 78] LowDegDeterminism dDDI : [PASSED] [ 6/ 78] LowDegDeterminism dDFI : [PASSED] [ 7/ 78] LowDegDeterminism dFFI :

AndreaNES avatar Jan 22 '18 21:01 AndreaNES

I think before @marsaev let's us know which unit tests are actually up to date and supposed to run, it's kind of a stabbing in the dark process.

bnase avatar Jan 22 '18 22:01 bnase

Following on my previous post, I have one question ... is there any platform / combination of build tools which results in the above tests to pass? For example do all the tests pass under Linux ?

Thanks for any feedback,

Andrea

AndreaNES avatar Jan 23 '18 17:01 AndreaNES

@AndreaNES The logs we have attached at original post here are under Linux, so it is definitely not a Windows issue.

bnase avatar Jan 23 '18 18:01 bnase

@bnase thanks for the comment, and sorry for not reading the first line of your post ... :-(

Would be nice to hear what's the situation with AmgX. I am currently using CUSP, but the project seems not to be maintained and I am assuming the natural choice would be to switch to AmgX - given its recent release in public form.

AndreaNES avatar Jan 24 '18 18:01 AndreaNES

Hi guys,

I'm in the process of restoring working state of AMGX unit tests which is required to progress AMGX code (around halfway through). Many of them were failing because were not properly maintained, i'll update the issue when it would be ready.

marsaev avatar Jan 25 '18 15:01 marsaev

Thank you for the feedback!

AndreaNES avatar Jan 26 '18 23:01 AndreaNES

Tracking internally: AMGX-13

marsaev avatar Apr 07 '21 19:04 marsaev

Hi @marsaev! What's the current status of the unittests? Are all supposed to pass or is there a list of cases which are expected to fail?

Slightly related, is there any release plan for upcoming changes? I saw the message by @mattmartineau in #169 which hinted towards some coming changes and it would be interesting to get to know more about these (both in terms of what's to come and what your expected time scope is - if there is any) :)

Thanks!

Samev avatar May 11 '22 18:05 Samev

Currently many of the unit tests do not work. I have finally managed to free up some time and I'm planning a number of changes to the library over the coming months. A very high level set of (that may change) of priorities for medium term development:

  • Remove parts of the code that have high maintenance cost and low value
  • Start to address issues with tests / docs / CI
  • Broad changes to the API (possibly introducing a new C++ API)
  • Performance optimisation:
    • Exact coarse solve for unsmoothed aggregation
    • Improved sparse operator performance (i.e. SpGEMM optimisations, currently seeing 2-6x in internal PoC developments depending on the operation)
    • Reducing the redundant compute in resetup
    • Adding capability to cache preconditioner
    • Fixing / optimising smoothers (particularly MULTICOLOR_GS)
    • Potentially hybrid threaded CPU/GPU execution mode

The list is actually much longer than this but these issues are at the top. We have lots of features we would like to explore but aren't committing to, e.g. smoothed aggregation.

mattmartineau avatar Jun 01 '22 08:06 mattmartineau