AMGX Seeking advice for setting up solver for finite element

Environment information:

OS: Windows 11 CUDA runtime: CUDA 12.4. AMGX version or commit hash: 2.5.0 NVIDIA driver: 551.78 NVIDIA GPU: NVIDIA RTX 3060 Ti

AMGX solver configuration: This is one of AMG configuration that I tried to use but failed to converged. { "config_version": 2, "solver": { "matrix_coloring_scheme": "MIN_MAX", "max_uncolored_percentage": 0.15, "algorithm": "AGGREGATION", "obtain_timings": 1, "solver": "AMG", "smoother": "MULTICOLOR_DILU", "print_solve_stats": 1, "presweeps": 1, "selector": "SIZE_2", "coarsest_sweeps": 2, "max_iters": 1000, "monitor_residual": 1, "scope": "main", "max_levels": 1000, "postsweeps": 1, "tolerance": 0.1, "print_grid_stats": 1, "norm": "L1", "cycle": "V" } }

Matrix is attached

Reproduction steps

I ran this with the command line as in the quick start document :"amgx_capi -m identity_system.mtx -c AGGREGATION_DILU.json

It got diverged. Samte thing with AMG_CLASSICAL_CG.json. Using AMG_AGGRREGATION_CG it manage to converge but very slow

If I use PCG_AGGREGATION_JACOBI i got the best convergence speed.

Can you please tell me which option should I choose for the type of matrix that I attached? This is basically a 10 nodes tet mesh generated from a 3D solid dishes.

Thanks,

Kien

coarse.zip

Apr 26 '24 20:04 kiendangtor

@kiendangtor I apologize for the slow response.

This matrix is quite small to exploit full GPU power. Would you consider direct solve approach here?

Other than that, configurations from provided examples are typical for this type of meshes (of course also depending on what are you trying to solve). I had in mind something like "config scanner" to find best solver options for given matrix, but no ETA on that.

Jul 03 '24 21:07 marsaev

Thank you for your reply. "config scanner" could be awesome. I had bigger mesh with 20mil of non zero but to make the diagnosis easier I attached a very coarse mesh. But the behavior are the same. I was able to successfully run the solver with many configs but all the one that has "MULTICOLOR_DILU" did not work either as a smoother or as preconditioner. Do we need to provide graph separately? I look at the code and it doesnt seem like it need that.

Jul 09 '24 20:07 kiendangtor

Understood, thanks. Right, starting with smaller matrix is often easier to put aside what's clearly not working. Just wanted to clarify some things

Do we need to provide graph separately? At this moment the linear system (A, rhs, optional x_0) is the only input AMGX is working with.
I was able to successfully run the solver with many configs So for the larger system (i.e. 20m NNZ, as you have mentioned above) many configs work, but they are still not fast enough? Or what kind of problem you experience there? Can you provide output of the best performing config for your larger case?
all the one that has "MULTICOLOR_DILU" did not work either as a smoother or as preconditioner. Can you elaborate what you mean here? In general, there is no reason to always use stronger smoother (like ILU), since they can be significantly slower than other smoothers, and we can try to compensate convergence rate by adjusting multigrid structure.

Thanks,

Jul 09 '24 21:07 marsaev

Thank you for your fast response. I have upload the larger system here: https://we.tl/t-ymNm1kBuan 2. My issue is that the CG solver without preconditioner is the best option so far in my case.. Here is my best configuration (CG without Precond, took ~ 6s) { "config_version": 2, "solver": { "solver": "CG", "print_solve_stats": 1, "obtain_timings": 1, "max_iters": 2000, "monitor_residual": 1, "convergence": "RELATIVE_INI_CORE", "scope": "main", "tolerance": 1e-6, "norm": "L2" } }

Here is the output: AMGX version 2.5.0 Built on Apr 25 2024, 13:20:11 Compiled with CUDA Runtime 12.4, using CUDA driver 12.4 iter Mem Usage (GB) residual rate ---------------------------------------------------------------------- Ini 0 1.353886e+01 0 0 1.262998e+01 0.9329 1 0.0000 1.344963e+01 1.0649 2 0.0000 1.341992e+01 0.9978 3 0.0000 1.287418e+01 0.9593 4 0.0000 1.291408e+01 1.0031 5 0.0000 1.116468e+01 0.8645 6 0.0000 1.006132e+01 0.9012 7 0.0000 9.782631e+00 0.9723 8 0.0000 9.901049e+00 1.0121 9 0.0000 9.289660e+00 0.9383 10 0.0000 8.494891e+00 0.9144 11 0.0000 8.746236e+00 1.0296 12 0.0000 9.184710e+00 1.0501 13 0.0000 8.967623e+00 0.9764 14 0.0000 8.210301e+00 0.9155 15 0.0000 7.656633e+00 0.9326 16 0.0000 7.515435e+00 0.9816 17 0.0000 7.752882e+00 1.0316 18 0.0000 7.573552e+00 0.9769 19 0.0000 6.960823e+00 0.9191 20 0.0000 6.493094e+00 0.9328 21 0.0000 6.856289e+00 1.0559 22 0.0000 7.807127e+00 1.1387 23 0.0000 7.382842e+00 0.9457 24 0.0000 6.450630e+00 0.8737 25 0.0000 6.198494e+00 0.9609 26 0.0000 6.606438e+00 1.0658 27 0.0000 6.915229e+00 1.0467 28 0.0000 6.720466e+00 0.9718 29 0.0000 6.636989e+00 0.9876 30 0.0000 6.287094e+00 0.9473 31 0.0000 5.595264e+00 0.8900 32 0.0000 5.520952e+00 0.9867 33 0.0000 5.446880e+00 0.9866 34 0.0000 5.337425e+00 0.9799 35 0.0000 5.085706e+00 0.9528 36 0.0000 4.873419e+00 0.9583 37 0.0000 4.674906e+00 0.9593 38 0.0000 4.506826e+00 0.9640 39 0.0000 4.389167e+00 0.9739 40 0.0000 4.533550e+00 1.0329 41 0.0000 4.536435e+00 1.0006 42 0.0000 4.155694e+00 0.9161 43 0.0000 4.163612e+00 1.0019 44 0.0000 4.321235e+00 1.0379 45 0.0000 3.934455e+00 0.9105 46 0.0000 3.598542e+00 0.9146 47 0.0000 3.606063e+00 1.0021 48 0.0000 3.828375e+00 1.0616 49 0.0000 3.652129e+00 0.9540 50 0.0000 3.340484e+00 0.9147 51 0.0000 3.341184e+00 1.0002 52 0.0000 3.360057e+00 1.0056 53 0.0000 3.161396e+00 0.9409 54 0.0000 3.171314e+00 1.0031 55 0.0000 3.069862e+00 0.9680 56 0.0000 2.932093e+00 0.9551 57 0.0000 2.925210e+00 0.9977 58 0.0000 2.768549e+00 0.9464 59 0.0000 2.684689e+00 0.9697 60 0.0000 2.733094e+00 1.0180 61 0.0000 2.531500e+00 0.9262 62 0.0000 2.409799e+00 0.9519 63 0.0000 2.575164e+00 1.0686 64 0.0000 2.486035e+00 0.9654 65 0.0000 2.201733e+00 0.8856 66 0.0000 2.209115e+00 1.0034 67 0.0000 2.206543e+00 0.9988 68 0.0000 2.200250e+00 0.9971 69 0.0000 2.254593e+00 1.0247 70 0.0000 2.201090e+00 0.9763 71 0.0000 2.113318e+00 0.9601 72 0.0000 2.080655e+00 0.9845 73 0.0000 2.179618e+00 1.0476 74 0.0000 2.060706e+00 0.9454 75 0.0000 2.057177e+00 0.9983 76 0.0000 2.110698e+00 1.0260 77 0.0000 1.945716e+00 0.9218 78 0.0000 1.900318e+00 0.9767 79 0.0000 1.922642e+00 1.0117 80 0.0000 1.836647e+00 0.9553 81 0.0000 1.834999e+00 0.9991 82 0.0000 1.910327e+00 1.0411 83 0.0000 1.781650e+00 0.9326 84 0.0000 1.714393e+00 0.9622 85 0.0000 1.695226e+00 0.9888 86 0.0000 1.698762e+00 1.0021 87 0.0000 1.756629e+00 1.0341 88 0.0000 1.720690e+00 0.9795 89 0.0000 1.655724e+00 0.9622 90 0.0000 1.667694e+00 1.0072 91 0.0000 1.695344e+00 1.0166 92 0.0000 1.588701e+00 0.9371 93 0.0000 1.541870e+00 0.9705 94 0.0000 1.582440e+00 1.0263 95 0.0000 1.581971e+00 0.9997 96 0.0000 1.522868e+00 0.9626 97 0.0000 1.524990e+00 1.0014 98 0.0000 1.475319e+00 0.9674 99 0.0000 1.471768e+00 0.9976 100 0.0000 1.501872e+00 1.0205 101 0.0000 1.425984e+00 0.9495 102 0.0000 1.397138e+00 0.9798 103 0.0000 1.411637e+00 1.0104 104 0.0000 1.376674e+00 0.9752 105 0.0000 1.334801e+00 0.9696 106 0.0000 1.331577e+00 0.9976 107 0.0000 1.352925e+00 1.0160 108 0.0000 1.282638e+00 0.9480 109 0.0000 1.289760e+00 1.0056 110 0.0000 1.311367e+00 1.0168 111 0.0000 1.271219e+00 0.9694 112 0.0000 1.256974e+00 0.9888 113 0.0000 1.279412e+00 1.0179 114 0.0000 1.257423e+00 0.9828 115 0.0000 1.231465e+00 0.9794 116 0.0000 1.235452e+00 1.0032 117 0.0000 1.186845e+00 0.9607 118 0.0000 1.158272e+00 0.9759 119 0.0000 1.163486e+00 1.0045 120 0.0000 1.146347e+00 0.9853 121 0.0000 1.128234e+00 0.9842 122 0.0000 1.151797e+00 1.0209 123 0.0000 1.163217e+00 1.0099 124 0.0000 1.141058e+00 0.9810 125 0.0000 1.130028e+00 0.9903 126 0.0000 1.090592e+00 0.9651 127 0.0000 1.084568e+00 0.9945 128 0.0000 1.052836e+00 0.9707 129 0.0000 1.031181e+00 0.9794 130 0.0000 1.057783e+00 1.0258 131 0.0000 1.037753e+00 0.9811 132 0.0000 1.003208e+00 0.9667 133 0.0000 1.009502e+00 1.0063 134 0.0000 1.015455e+00 1.0059 135 0.0000 9.903361e-01 0.9753 136 0.0000 9.782567e-01 0.9878 137 0.0000 9.792711e-01 1.0010 138 0.0000 9.522720e-01 0.9724 139 0.0000 9.373981e-01 0.9844 140 0.0000 9.393499e-01 1.0021 141 0.0000 9.349063e-01 0.9953 142 0.0000 9.307010e-01 0.9955 143 0.0000 9.104812e-01 0.9783 144 0.0000 9.000397e-01 0.9885 145 0.0000 8.931340e-01 0.9923 146 0.0000 8.813223e-01 0.9868 147 0.0000 8.854138e-01 1.0046 148 0.0000 8.714043e-01 0.9842 149 0.0000 8.596665e-01 0.9865 150 0.0000 8.388330e-01 0.9758 151 0.0000 8.474289e-01 1.0102 152 0.0000 8.478095e-01 1.0004 153 0.0000 8.403001e-01 0.9911 154 0.0000 8.205706e-01 0.9765 155 0.0000 8.303758e-01 1.0119 156 0.0000 8.393402e-01 1.0108 157 0.0000 8.239402e-01 0.9817 158 0.0000 8.310023e-01 1.0086 159 0.0000 8.476804e-01 1.0201 160 0.0000 8.022486e-01 0.9464 161 0.0000 7.757737e-01 0.9670 162 0.0000 7.889112e-01 1.0169 163 0.0000 7.860369e-01 0.9964 164 0.0000 7.505643e-01 0.9549 165 0.0000 7.480393e-01 0.9966 166 0.0000 7.573304e-01 1.0124 167 0.0000 7.290384e-01 0.9626 168 0.0000 7.477997e-01 1.0257 169 0.0000 7.526946e-01 1.0065 170 0.0000 7.233232e-01 0.9610 171 0.0000 7.462722e-01 1.0317 172 0.0000 7.483545e-01 1.0028 173 0.0000 7.120085e-01 0.9514 174 0.0000 7.181118e-01 1.0086 175 0.0000 7.167547e-01 0.9981 176 0.0000 7.207807e-01 1.0056 177 0.0000 7.161876e-01 0.9936 178 0.0000 6.931766e-01 0.9679 179 0.0000 6.684345e-01 0.9643 180 0.0000 6.566061e-01 0.9823 181 0.0000 6.584297e-01 1.0028 182 0.0000 6.757532e-01 1.0263 183 0.0000 6.572882e-01 0.9727 184 0.0000 6.698679e-01 1.0191 185 0.0000 6.785304e-01 1.0129 186 0.0000 6.390069e-01 0.9418 187 0.0000 6.447940e-01 1.0091 188 0.0000 6.370597e-01 0.9880 189 0.0000 6.382035e-01 1.0018 190 0.0000 6.326952e-01 0.9914 191 0.0000 6.173935e-01 0.9758 192 0.0000 6.126585e-01 0.9923 193 0.0000 6.200593e-01 1.0121 194 0.0000 6.183008e-01 0.9972 195 0.0000 6.195359e-01 1.0020 196 0.0000 6.215696e-01 1.0033 197 0.0000 6.009018e-01 0.9667 198 0.0000 6.136047e-01 1.0211 199 0.0000 6.118257e-01 0.9971 200 0.0000 6.168415e-01 1.0082 201 0.0000 6.159987e-01 0.9986 202 0.0000 5.853574e-01 0.9503 203 0.0000 5.884180e-01 1.0052 204 0.0000 6.030999e-01 1.0250 205 0.0000 5.890910e-01 0.9768 206 0.0000 5.823547e-01 0.9886 207 0.0000 5.855304e-01 1.0055 208 0.0000 5.724553e-01 0.9777 209 0.0000 5.614496e-01 0.9808 210 0.0000 5.762443e-01 1.0264 211 0.0000 5.673486e-01 0.9846 212 0.0000 5.650837e-01 0.9960 213 0.0000 5.709320e-01 1.0103 214 0.0000 5.884823e-01 1.0307 215 0.0000 5.737960e-01 0.9750 216 0.0000 5.553936e-01 0.9679 217 0.0000 5.511891e-01 0.9924 218 0.0000 5.436915e-01 0.9864 219 0.0000 5.523357e-01 1.0159 220 0.0000 5.481563e-01 0.9924 221 0.0000 5.295897e-01 0.9661 222 0.0000 5.276222e-01 0.9963 223 0.0000 5.382623e-01 1.0202 224 0.0000 5.356678e-01 0.9952 225 0.0000 5.328757e-01 0.9948 226 0.0000 5.263735e-01 0.9878 227 0.0000 5.179160e-01 0.9839 228 0.0000 5.029985e-01 0.9712 229 0.0000 4.939617e-01 0.9820 230 0.0000 4.967452e-01 1.0056 231 0.0000 4.767040e-01 0.9597 232 0.0000 4.719018e-01 0.9899 233 0.0000 4.827788e-01 1.0230 234 0.0000 4.641570e-01 0.9614 235 0.0000 4.580175e-01 0.9868 236 0.0000 4.626222e-01 1.0101 237 0.0000 4.577362e-01 0.9894 238 0.0000 4.635887e-01 1.0128 239 0.0000 4.624222e-01 0.9975 240 0.0000 4.438046e-01 0.9597 241 0.0000 4.463173e-01 1.0057 242 0.0000 4.554095e-01 1.0204 243 0.0000 4.301062e-01 0.9444 244 0.0000 4.336583e-01 1.0083 245 0.0000 4.284497e-01 0.9880 246 0.0000 4.189278e-01 0.9778 247 0.0000 4.373496e-01 1.0440 248 0.0000 4.332623e-01 0.9907 249 0.0000 4.345460e-01 1.0030 250 0.0000 4.393239e-01 1.0110 251 0.0000 4.207781e-01 0.9578 252 0.0000 4.332030e-01 1.0295 253 0.0000 4.239820e-01 0.9787 254 0.0000 4.211489e-01 0.9933 255 0.0000 4.192285e-01 0.9954 256 0.0000 4.034146e-01 0.9623 257 0.0000 4.089019e-01 1.0136 258 0.0000 4.069869e-01 0.9953 259 0.0000 4.020965e-01 0.9880 260 0.0000 4.101955e-01 1.0201 261 0.0000 4.054649e-01 0.9885 262 0.0000 3.963023e-01 0.9774 263 0.0000 3.990551e-01 1.0069 264 0.0000 3.803718e-01 0.9532 265 0.0000 3.792435e-01 0.9970 266 0.0000 3.828624e-01 1.0095 267 0.0000 3.575243e-01 0.9338 268 0.0000 3.442248e-01 0.9628 269 0.0000 3.424394e-01 0.9948 270 0.0000 3.271007e-01 0.9552 271 0.0000 3.143270e-01 0.9609 272 0.0000 3.144425e-01 1.0004 273 0.0000 3.037625e-01 0.9660 274 0.0000 2.861743e-01 0.9421 275 0.0000 2.769139e-01 0.9676 276 0.0000 2.671295e-01 0.9647 277 0.0000 2.532142e-01 0.9479 278 0.0000 2.375813e-01 0.9383 279 0.0000 2.234230e-01 0.9404 280 0.0000 2.069208e-01 0.9261 281 0.0000 1.929073e-01 0.9323 282 0.0000 1.848349e-01 0.9582 283 0.0000 1.809836e-01 0.9792 284 0.0000 1.736374e-01 0.9594 285 0.0000 1.527659e-01 0.8798 286 0.0000 1.417858e-01 0.9281 287 0.0000 1.315394e-01 0.9277 288 0.0000 1.235922e-01 0.9396 289 0.0000 1.162488e-01 0.9406 290 0.0000 1.119325e-01 0.9629 291 0.0000 1.068759e-01 0.9548 292 0.0000 9.955425e-02 0.9315 293 0.0000 9.073929e-02 0.9115 294 0.0000 8.643774e-02 0.9526 295 0.0000 8.299320e-02 0.9602 296 0.0000 8.080298e-02 0.9736 297 0.0000 7.763006e-02 0.9607 298 0.0000 7.273561e-02 0.9370 299 0.0000 7.232574e-02 0.9944 300 0.0000 6.658044e-02 0.9206 301 0.0000 6.364191e-02 0.9559 302 0.0000 6.233454e-02 0.9795 303 0.0000 6.132779e-02 0.9838 304 0.0000 5.801469e-02 0.9460 305 0.0000 5.795389e-02 0.9990 306 0.0000 5.633932e-02 0.9721 307 0.0000 5.434833e-02 0.9647 308 0.0000 5.382773e-02 0.9904 309 0.0000 5.311001e-02 0.9867 310 0.0000 5.027714e-02 0.9467 311 0.0000 4.726738e-02 0.9401 312 0.0000 4.629796e-02 0.9795 313 0.0000 4.521235e-02 0.9766 314 0.0000 4.273949e-02 0.9453 315 0.0000 4.155467e-02 0.9723 316 0.0000 4.217253e-02 1.0149 317 0.0000 3.988379e-02 0.9457 318 0.0000 3.859674e-02 0.9677 319 0.0000 3.897464e-02 1.0098 320 0.0000 3.746127e-02 0.9612 321 0.0000 3.647192e-02 0.9736 322 0.0000 3.558145e-02 0.9756 323 0.0000 3.408042e-02 0.9578 324 0.0000 3.336385e-02 0.9790 325 0.0000 3.280365e-02 0.9832 326 0.0000 3.194300e-02 0.9738 327 0.0000 3.162545e-02 0.9901 328 0.0000 3.207835e-02 1.0143 329 0.0000 3.056155e-02 0.9527 330 0.0000 2.968799e-02 0.9714 331 0.0000 2.903696e-02 0.9781 332 0.0000 2.764549e-02 0.9521 333 0.0000 2.757215e-02 0.9973 334 0.0000 2.787487e-02 1.0110 335 0.0000 2.687063e-02 0.9640 336 0.0000 2.648189e-02 0.9855 337 0.0000 2.687394e-02 1.0148 338 0.0000 2.693819e-02 1.0024 339 0.0000 2.627692e-02 0.9755 340 0.0000 2.556245e-02 0.9728 341 0.0000 2.536271e-02 0.9922 342 0.0000 2.513220e-02 0.9909 343 0.0000 2.541615e-02 1.0113 344 0.0000 2.598969e-02 1.0226 345 0.0000 2.565233e-02 0.9870 346 0.0000 2.455921e-02 0.9574 347 0.0000 2.338488e-02 0.9522 348 0.0000 2.328266e-02 0.9956 349 0.0000 2.367848e-02 1.0170 350 0.0000 2.278399e-02 0.9622 351 0.0000 2.238886e-02 0.9827 352 0.0000 2.245145e-02 1.0028 353 0.0000 2.184556e-02 0.9730 354 0.0000 2.186281e-02 1.0008 355 0.0000 2.197417e-02 1.0051 356 0.0000 2.193714e-02 0.9983 357 0.0000 2.215427e-02 1.0099 358 0.0000 2.194692e-02 0.9906 359 0.0000 2.112797e-02 0.9627 360 0.0000 2.072562e-02 0.9810 361 0.0000 2.087301e-02 1.0071 362 0.0000 2.067195e-02 0.9904 363 0.0000 2.124003e-02 1.0275 364 0.0000 2.074068e-02 0.9765 365 0.0000 1.932746e-02 0.9319 366 0.0000 1.964174e-02 1.0163 367 0.0000 1.952642e-02 0.9941 368 0.0000 1.912590e-02 0.9795 369 0.0000 1.956800e-02 1.0231 370 0.0000 1.892145e-02 0.9670 371 0.0000 1.880505e-02 0.9938 372 0.0000 1.935910e-02 1.0295 373 0.0000 1.886556e-02 0.9745 374 0.0000 1.892023e-02 1.0029 375 0.0000 1.862439e-02 0.9844 376 0.0000 1.825186e-02 0.9800 377 0.0000 1.813391e-02 0.9935 378 0.0000 1.811987e-02 0.9992 379 0.0000 1.783458e-02 0.9843 380 0.0000 1.785637e-02 1.0012 381 0.0000 1.832706e-02 1.0264 382 0.0000 1.819248e-02 0.9927 383 0.0000 1.717503e-02 0.9441 384 0.0000 1.694985e-02 0.9869 385 0.0000 1.674220e-02 0.9877 386 0.0000 1.661223e-02 0.9922 387 0.0000 1.686434e-02 1.0152 388 0.0000 1.687965e-02 1.0009 389 0.0000 1.698021e-02 1.0060 390 0.0000 1.645316e-02 0.9690 391 0.0000 1.632727e-02 0.9923 392 0.0000 1.677217e-02 1.0272 393 0.0000 1.567514e-02 0.9346 394 0.0000 1.549359e-02 0.9884 395 0.0000 1.568206e-02 1.0122 396 0.0000 1.544631e-02 0.9850 397 0.0000 1.578358e-02 1.0218 398 0.0000 1.582254e-02 1.0025 399 0.0000 1.573691e-02 0.9946 400 0.0000 1.550577e-02 0.9853 401 0.0000 1.548083e-02 0.9984 402 0.0000 1.535429e-02 0.9918 403 0.0000 1.572514e-02 1.0242 404 0.0000 1.604480e-02 1.0203 405 0.0000 1.601420e-02 0.9981 406 0.0000 1.585728e-02 0.9902 407 0.0000 1.531246e-02 0.9656 408 0.0000 1.513600e-02 0.9885 409 0.0000 1.563596e-02 1.0330 410 0.0000 1.568023e-02 1.0028 411 0.0000 1.608473e-02 1.0258 412 0.0000 1.529724e-02 0.9510 413 0.0000 1.491083e-02 0.9747 414 0.0000 1.598739e-02 1.0722 415 0.0000 1.578798e-02 0.9875 416 0.0000 1.515374e-02 0.9598 417 0.0000 1.520244e-02 1.0032 418 0.0000 1.470155e-02 0.9671 419 0.0000 1.456586e-02 0.9908 420 0.0000 1.469726e-02 1.0090 421 0.0000 1.468928e-02 0.9995 422 0.0000 1.437507e-02 0.9786 423 0.0000 1.442039e-02 1.0032 424 0.0000 1.458873e-02 1.0117 425 0.0000 1.410793e-02 0.9670 426 0.0000 1.374792e-02 0.9745 427 0.0000 1.376393e-02 1.0012 428 0.0000 1.362299e-02 0.9898 429 0.0000 1.357906e-02 0.9968 430 0.0000 1.374911e-02 1.0125 431 0.0000 1.360199e-02 0.9893 432 0.0000 1.310474e-02 0.9634 433 0.0000 1.307918e-02 0.9980 434 0.0000 1.313347e-02 1.0042 435 0.0000 1.338300e-02 1.0190 436 0.0000 1.370956e-02 1.0244 437 0.0000 1.287422e-02 0.9391 438 0.0000 1.235340e-02 0.9595 439 0.0000 1.245901e-02 1.0085 440 0.0000 1.242344e-02 0.9971 441 0.0000 1.247323e-02 1.0040 442 0.0000 1.298080e-02 1.0407 443 0.0000 1.291163e-02 0.9947 444 0.0000 1.274599e-02 0.9872 445 0.0000 1.255987e-02 0.9854 446 0.0000 1.237453e-02 0.9852 447 0.0000 1.247371e-02 1.0080 448 0.0000 1.263250e-02 1.0127 449 0.0000 1.271840e-02 1.0068 450 0.0000 1.222111e-02 0.9609 451 0.0000 1.199371e-02 0.9814 452 0.0000 1.223683e-02 1.0203 453 0.0000 1.206892e-02 0.9863 454 0.0000 1.180749e-02 0.9783 455 0.0000 1.122894e-02 0.9510 456 0.0000 1.112267e-02 0.9905 457 0.0000 1.132348e-02 1.0181 458 0.0000 1.109869e-02 0.9801 459 0.0000 1.095065e-02 0.9867 460 0.0000 1.078360e-02 0.9847 461 0.0000 1.050138e-02 0.9738 462 0.0000 1.037224e-02 0.9877 463 0.0000 1.042789e-02 1.0054 464 0.0000 1.052245e-02 1.0091 465 0.0000 1.026729e-02 0.9758 466 0.0000 9.842529e-03 0.9586 467 0.0000 9.732925e-03 0.9889 468 0.0000 9.525361e-03 0.9787 469 0.0000 9.552203e-03 1.0028 470 0.0000 9.502360e-03 0.9948 471 0.0000 9.599928e-03 1.0103 472 0.0000 9.534268e-03 0.9932 473 0.0000 9.076386e-03 0.9520 474 0.0000 8.959994e-03 0.9872 475 0.0000 8.939976e-03 0.9978 476 0.0000 8.795774e-03 0.9839 477 0.0000 8.727889e-03 0.9923 478 0.0000 8.782370e-03 1.0062 479 0.0000 8.983462e-03 1.0229 480 0.0000 9.075476e-03 1.0102 481 0.0000 8.419634e-03 0.9277 482 0.0000 8.401494e-03 0.9978 483 0.0000 8.477547e-03 1.0091 484 0.0000 8.349500e-03 0.9849 485 0.0000 8.223551e-03 0.9849 486 0.0000 8.072738e-03 0.9817 487 0.0000 7.854028e-03 0.9729 488 0.0000 7.722498e-03 0.9833 489 0.0000 7.667248e-03 0.9928 490 0.0000 7.634887e-03 0.9958 491 0.0000 7.656882e-03 1.0029 492 0.0000 7.697057e-03 1.0052 493 0.0000 7.669751e-03 0.9965 494 0.0000 7.357037e-03 0.9592 495 0.0000 7.097171e-03 0.9647 496 0.0000 7.206420e-03 1.0154 497 0.0000 7.076045e-03 0.9819 498 0.0000 6.735162e-03 0.9518 499 0.0000 6.765319e-03 1.0045 500 0.0000 6.794066e-03 1.0042 501 0.0000 6.688231e-03 0.9844 502 0.0000 6.849964e-03 1.0242 503 0.0000 6.645858e-03 0.9702 504 0.0000 6.440888e-03 0.9692 505 0.0000 6.320286e-03 0.9813 506 0.0000 6.248014e-03 0.9886 507 0.0000 6.337786e-03 1.0144 508 0.0000 6.349945e-03 1.0019 509 0.0000 6.349234e-03 0.9999 510 0.0000 6.388300e-03 1.0062 511 0.0000 6.395770e-03 1.0012 512 0.0000 6.289377e-03 0.9834 513 0.0000 6.241988e-03 0.9925 514 0.0000 6.170202e-03 0.9885 515 0.0000 6.312940e-03 1.0231 516 0.0000 6.636119e-03 1.0512 517 0.0000 6.341295e-03 0.9556 518 0.0000 6.353455e-03 1.0019 519 0.0000 6.296889e-03 0.9911 520 0.0000 6.241926e-03 0.9913 521 0.0000 6.386142e-03 1.0231 522 0.0000 6.247452e-03 0.9783 523 0.0000 6.186063e-03 0.9902 524 0.0000 6.187890e-03 1.0003 525 0.0000 6.105420e-03 0.9867 526 0.0000 6.226842e-03 1.0199 527 0.0000 6.319307e-03 1.0148 528 0.0000 6.334320e-03 1.0024 529 0.0000 6.464382e-03 1.0205 530 0.0000 6.301481e-03 0.9748 531 0.0000 6.273641e-03 0.9956 532 0.0000 6.388512e-03 1.0183 533 0.0000 6.345122e-03 0.9932 534 0.0000 6.208837e-03 0.9785 535 0.0000 6.146809e-03 0.9900 536 0.0000 6.202854e-03 1.0091 537 0.0000 6.234117e-03 1.0050 538 0.0000 6.113430e-03 0.9806 539 0.0000 6.066137e-03 0.9923 540 0.0000 6.066997e-03 1.0001 541 0.0000 6.048647e-03 0.9970 542 0.0000 6.134174e-03 1.0141 543 0.0000 6.160458e-03 1.0043 544 0.0000 6.227700e-03 1.0109 545 0.0000 6.318068e-03 1.0145 546 0.0000 6.169826e-03 0.9765 547 0.0000 6.221217e-03 1.0083 548 0.0000 6.264204e-03 1.0069 549 0.0000 6.186179e-03 0.9875 550 0.0000 6.309242e-03 1.0199 551 0.0000 6.353479e-03 1.0070 552 0.0000 6.357793e-03 1.0007 553 0.0000 6.353085e-03 0.9993 554 0.0000 6.362545e-03 1.0015 555 0.0000 6.307434e-03 0.9913 556 0.0000 6.260310e-03 0.9925 557 0.0000 6.375725e-03 1.0184 558 0.0000 6.262039e-03 0.9822 559 0.0000 6.339788e-03 1.0124 560 0.0000 6.373469e-03 1.0053 561 0.0000 6.169795e-03 0.9680 562 0.0000 6.452991e-03 1.0459 563 0.0000 6.528671e-03 1.0117 564 0.0000 6.262205e-03 0.9592 565 0.0000 6.303491e-03 1.0066 566 0.0000 6.412179e-03 1.0172 567 0.0000 6.303144e-03 0.9830 568 0.0000 6.147397e-03 0.9753 569 0.0000 6.136071e-03 0.9982 570 0.0000 6.294733e-03 1.0259 571 0.0000 6.012360e-03 0.9551 572 0.0000 5.891027e-03 0.9798 573 0.0000 5.988166e-03 1.0165 574 0.0000 5.891136e-03 0.9838 575 0.0000 5.977501e-03 1.0147 576 0.0000 5.716100e-03 0.9563 577 0.0000 5.635954e-03 0.9860 578 0.0000 5.594633e-03 0.9927 579 0.0000 5.477058e-03 0.9790 580 0.0000 5.440232e-03 0.9933 581 0.0000 5.456996e-03 1.0031 582 0.0000 5.406281e-03 0.9907 583 0.0000 5.364208e-03 0.9922 584 0.0000 5.343921e-03 0.9962 585 0.0000 5.195428e-03 0.9722 586 0.0000 5.294369e-03 1.0190 587 0.0000 5.331546e-03 1.0070 588 0.0000 5.083982e-03 0.9536 589 0.0000 5.033618e-03 0.9901 590 0.0000 4.946860e-03 0.9828 591 0.0000 4.898466e-03 0.9902 592 0.0000 4.911797e-03 1.0027 593 0.0000 4.852804e-03 0.9880 594 0.0000 4.802611e-03 0.9897 595 0.0000 4.875350e-03 1.0151 596 0.0000 4.836140e-03 0.9920 597 0.0000 4.760769e-03 0.9844 598 0.0000 4.619881e-03 0.9704 599 0.0000 4.380142e-03 0.9481 600 0.0000 4.388399e-03 1.0019 601 0.0000 4.401163e-03 1.0029 602 0.0000 4.292870e-03 0.9754 603 0.0000 4.236514e-03 0.9869 604 0.0000 4.266072e-03 1.0070 605 0.0000 4.292588e-03 1.0062 606 0.0000 4.262486e-03 0.9930 607 0.0000 4.249779e-03 0.9970 608 0.0000 4.102022e-03 0.9652 609 0.0000 4.095270e-03 0.9984 610 0.0000 4.047164e-03 0.9883 611 0.0000 3.910163e-03 0.9661 612 0.0000 3.914555e-03 1.0011 613 0.0000 3.781342e-03 0.9660 614 0.0000 3.725926e-03 0.9853 615 0.0000 3.809098e-03 1.0223 616 0.0000 3.795154e-03 0.9963 617 0.0000 3.777809e-03 0.9954 618 0.0000 3.726416e-03 0.9864 619 0.0000 3.552350e-03 0.9533 620 0.0000 3.618827e-03 1.0187 621 0.0000 3.681537e-03 1.0173 622 0.0000 3.627929e-03 0.9854 623 0.0000 3.571412e-03 0.9844 624 0.0000 3.417510e-03 0.9569 625 0.0000 3.426221e-03 1.0025 626 0.0000 3.458219e-03 1.0093 627 0.0000 3.438220e-03 0.9942 628 0.0000 3.536513e-03 1.0286 629 0.0000 3.362617e-03 0.9508 630 0.0000 3.245816e-03 0.9653 631 0.0000 3.292259e-03 1.0143 632 0.0000 3.292518e-03 1.0001 633 0.0000 3.260045e-03 0.9901 634 0.0000 3.379410e-03 1.0366 635 0.0000 3.318488e-03 0.9820 636 0.0000 3.299100e-03 0.9942 637 0.0000 3.323998e-03 1.0075 638 0.0000 3.165984e-03 0.9525 639 0.0000 3.162043e-03 0.9988 640 0.0000 3.219586e-03 1.0182 641 0.0000 3.144344e-03 0.9766 642 0.0000 3.141328e-03 0.9990 643 0.0000 3.223903e-03 1.0263 644 0.0000 3.233494e-03 1.0030 645 0.0000 3.247744e-03 1.0044 646 0.0000 3.141434e-03 0.9673 647 0.0000 3.092526e-03 0.9844 648 0.0000 3.080675e-03 0.9962 649 0.0000 3.050095e-03 0.9901 650 0.0000 3.121482e-03 1.0234 651 0.0000 3.205082e-03 1.0268 652 0.0000 3.096339e-03 0.9661 653 0.0000 3.014102e-03 0.9734 654 0.0000 2.980229e-03 0.9888 655 0.0000 3.016261e-03 1.0121 656 0.0000 3.047112e-03 1.0102 657 0.0000 3.096890e-03 1.0163 658 0.0000 3.090594e-03 0.9980 659 0.0000 3.049689e-03 0.9868 660 0.0000 3.119639e-03 1.0229 661 0.0000 3.104548e-03 0.9952 662 0.0000 3.063124e-03 0.9867 663 0.0000 3.071626e-03 1.0028 664 0.0000 2.994386e-03 0.9749 665 0.0000 3.074427e-03 1.0267 666 0.0000 3.042785e-03 0.9897 667 0.0000 3.002771e-03 0.9868 668 0.0000 3.071673e-03 1.0229 669 0.0000 3.020934e-03 0.9835 670 0.0000 3.068377e-03 1.0157 671 0.0000 3.032896e-03 0.9884 672 0.0000 2.933976e-03 0.9674 673 0.0000 2.964433e-03 1.0104 674 0.0000 2.952689e-03 0.9960 675 0.0000 2.885523e-03 0.9773 676 0.0000 2.882944e-03 0.9991 677 0.0000 2.935961e-03 1.0184 678 0.0000 2.920453e-03 0.9947 679 0.0000 2.858204e-03 0.9787 680 0.0000 2.762709e-03 0.9666 681 0.0000 2.707928e-03 0.9802 682 0.0000 2.716042e-03 1.0030 683 0.0000 2.721017e-03 1.0018 684 0.0000 2.720301e-03 0.9997 685 0.0000 2.696040e-03 0.9911 686 0.0000 2.657746e-03 0.9858 687 0.0000 2.664420e-03 1.0025 688 0.0000 2.610177e-03 0.9796 689 0.0000 2.645394e-03 1.0135 690 0.0000 2.699657e-03 1.0205 691 0.0000 2.609284e-03 0.9665 692 0.0000 2.629226e-03 1.0076 693 0.0000 2.611216e-03 0.9932 694 0.0000 2.630112e-03 1.0072 695 0.0000 2.653442e-03 1.0089 696 0.0000 2.538814e-03 0.9568 697 0.0000 2.502645e-03 0.9858 698 0.0000 2.504065e-03 1.0006 699 0.0000 2.687520e-03 1.0733 700 0.0000 2.551698e-03 0.9495 701 0.0000 2.461522e-03 0.9647 702 0.0000 2.531003e-03 1.0282 703 0.0000 2.430953e-03 0.9605 704 0.0000 2.430052e-03 0.9996 705 0.0000 2.504488e-03 1.0306 706 0.0000 2.466978e-03 0.9850 707 0.0000 2.456033e-03 0.9956 708 0.0000 2.491324e-03 1.0144 709 0.0000 2.426681e-03 0.9741 710 0.0000 2.391890e-03 0.9857 711 0.0000 2.478016e-03 1.0360 712 0.0000 2.511699e-03 1.0136 713 0.0000 2.449575e-03 0.9753 714 0.0000 2.481302e-03 1.0130 715 0.0000 2.505996e-03 1.0100 716 0.0000 2.501104e-03 0.9980 717 0.0000 2.451500e-03 0.9802 718 0.0000 2.477735e-03 1.0107 719 0.0000 2.442478e-03 0.9858 720 0.0000 2.383028e-03 0.9757 721 0.0000 2.382710e-03 0.9999 722 0.0000 2.370649e-03 0.9949 723 0.0000 2.415416e-03 1.0189 724 0.0000 2.438737e-03 1.0097 725 0.0000 2.407252e-03 0.9871 726 0.0000 2.346697e-03 0.9748 727 0.0000 2.321115e-03 0.9891 728 0.0000 2.333413e-03 1.0053 729 0.0000 2.351854e-03 1.0079 730 0.0000 2.409359e-03 1.0245 731 0.0000 2.344550e-03 0.9731 732 0.0000 2.293627e-03 0.9783 733 0.0000 2.371450e-03 1.0339 734 0.0000 2.287999e-03 0.9648 735 0.0000 2.313765e-03 1.0113 736 0.0000 2.260754e-03 0.9771 737 0.0000 2.188349e-03 0.9680 738 0.0000 2.238853e-03 1.0231 739 0.0000 2.291927e-03 1.0237 740 0.0000 2.258121e-03 0.9853 741 0.0000 2.213533e-03 0.9803 742 0.0000 2.192415e-03 0.9905 743 0.0000 2.159613e-03 0.9850 744 0.0000 2.152427e-03 0.9967 745 0.0000 2.170189e-03 1.0083 746 0.0000 2.223195e-03 1.0244 747 0.0000 2.307541e-03 1.0379 748 0.0000 2.181823e-03 0.9455 749 0.0000 2.163917e-03 0.9918 750 0.0000 2.167458e-03 1.0016 751 0.0000 2.156657e-03 0.9950 752 0.0000 2.125397e-03 0.9855 753 0.0000 2.142835e-03 1.0082 754 0.0000 2.217114e-03 1.0347 755 0.0000 2.107002e-03 0.9503 756 0.0000 2.074550e-03 0.9846 757 0.0000 2.047449e-03 0.9869 758 0.0000 2.003972e-03 0.9788 759 0.0000 2.014668e-03 1.0053 760 0.0000 1.975443e-03 0.9805 761 0.0000 1.945485e-03 0.9848 762 0.0000 1.952831e-03 1.0038 763 0.0000 1.937567e-03 0.9922 764 0.0000 1.894101e-03 0.9776 765 0.0000 1.868663e-03 0.9866 766 0.0000 1.876975e-03 1.0044 767 0.0000 1.855223e-03 0.9884 768 0.0000 1.856640e-03 1.0008 769 0.0000 1.873003e-03 1.0088 770 0.0000 1.825525e-03 0.9747 771 0.0000 1.772884e-03 0.9712 772 0.0000 1.791424e-03 1.0105 773 0.0000 1.761388e-03 0.9832 774 0.0000 1.754499e-03 0.9961 775 0.0000 1.749571e-03 0.9972 776 0.0000 1.732596e-03 0.9903 777 0.0000 1.727874e-03 0.9973 778 0.0000 1.732001e-03 1.0024 779 0.0000 1.727282e-03 0.9973 780 0.0000 1.700106e-03 0.9843 781 0.0000 1.703887e-03 1.0022 782 0.0000 1.680089e-03 0.9860 783 0.0000 1.628074e-03 0.9690 784 0.0000 1.628501e-03 1.0003 785 0.0000 1.636634e-03 1.0050 786 0.0000 1.628870e-03 0.9953 787 0.0000 1.641730e-03 1.0079 788 0.0000 1.625949e-03 0.9904 789 0.0000 1.581212e-03 0.9725 790 0.0000 1.532030e-03 0.9689 791 0.0000 1.519634e-03 0.9919 792 0.0000 1.535160e-03 1.0102 793 0.0000 1.550009e-03 1.0097 794 0.0000 1.563274e-03 1.0086 795 0.0000 1.493288e-03 0.9552 796 0.0000 1.459549e-03 0.9774 797 0.0000 1.509726e-03 1.0344 798 0.0000 1.470356e-03 0.9739 799 0.0000 1.436431e-03 0.9769 800 0.0000 1.411141e-03 0.9824 801 0.0000 1.403195e-03 0.9944 802 0.0000 1.402421e-03 0.9994 803 0.0000 1.385873e-03 0.9882 804 0.0000 1.353841e-03 0.9769 805 0.0000 1.340951e-03 0.9905 806 0.0000 1.341311e-03 1.0003 807 0.0000 1.349036e-03 1.0058 808 0.0000 1.350563e-03 1.0011 809 0.0000 1.285433e-03 0.9518 810 0.0000 1.302456e-03 1.0132 811 0.0000 1.317619e-03 1.0116 812 0.0000 1.323645e-03 1.0046 813 0.0000 1.322994e-03 0.9995 814 0.0000 1.356128e-03 1.0250 815 0.0000 1.317021e-03 0.9712 816 0.0000 1.286394e-03 0.9767 817 0.0000 1.293315e-03 1.0054 818 0.0000 1.301398e-03 1.0062 819 0.0000 1.347228e-03 1.0352 820 0.0000 1.294844e-03 0.9611 821 0.0000 1.260692e-03 0.9736 822 0.0000 1.269896e-03 1.0073 823 0.0000 1.234638e-03 0.9722 824 0.0000 1.231516e-03 0.9975 825 0.0000 1.250195e-03 1.0152 826 0.0000 1.237134e-03 0.9896 827 0.0000 1.258198e-03 1.0170 828 0.0000 1.279474e-03 1.0169 829 0.0000 1.274285e-03 0.9959 830 0.0000 1.252230e-03 0.9827 831 0.0000 1.216491e-03 0.9715 832 0.0000 1.204917e-03 0.9905 833 0.0000 1.167398e-03 0.9689 834 0.0000 1.161045e-03 0.9946 835 0.0000 1.173960e-03 1.0111 836 0.0000 1.182561e-03 1.0073 837 0.0000 1.192346e-03 1.0083 838 0.0000 1.148196e-03 0.9630 839 0.0000 1.092183e-03 0.9512 840 0.0000 1.058793e-03 0.9694 841 0.0000 1.045892e-03 0.9878 842 0.0000 1.038956e-03 0.9934 843 0.0000 1.037247e-03 0.9984 844 0.0000 1.042432e-03 1.0050 845 0.0000 1.049370e-03 1.0067 846 0.0000 1.064085e-03 1.0140 847 0.0000 1.021784e-03 0.9602 848 0.0000 9.960429e-04 0.9748 849 0.0000 9.829912e-04 0.9869 850 0.0000 9.908452e-04 1.0080 851 0.0000 1.005408e-03 1.0147 852 0.0000 9.850166e-04 0.9797 853 0.0000 9.731450e-04 0.9879 854 0.0000 9.615552e-04 0.9881 855 0.0000 9.421366e-04 0.9798 856 0.0000 9.616912e-04 1.0208 857 0.0000 9.832599e-04 1.0224 858 0.0000 9.875020e-04 1.0043 859 0.0000 9.857665e-04 0.9982 860 0.0000 9.712259e-04 0.9852 861 0.0000 9.714164e-04 1.0002 862 0.0000 9.580410e-04 0.9862 863 0.0000 9.725060e-04 1.0151 864 0.0000 9.726826e-04 1.0002 865 0.0000 9.394663e-04 0.9659 866 0.0000 9.413713e-04 1.0020 867 0.0000 9.396075e-04 0.9981 868 0.0000 9.409370e-04 1.0014 869 0.0000 9.498303e-04 1.0095 870 0.0000 9.261593e-04 0.9751 871 0.0000 9.199120e-04 0.9933 872 0.0000 9.200373e-04 1.0001 873 0.0000 9.041397e-04 0.9827 874 0.0000 9.123642e-04 1.0091 875 0.0000 9.192381e-04 1.0075 876 0.0000 9.188214e-04 0.9995 877 0.0000 9.000381e-04 0.9796 878 0.0000 8.806142e-04 0.9784 879 0.0000 8.855791e-04 1.0056 880 0.0000 8.676648e-04 0.9798 881 0.0000 8.677054e-04 1.0000 882 0.0000 8.807259e-04 1.0150 883 0.0000 8.676048e-04 0.9851 884 0.0000 8.676534e-04 1.0001 885 0.0000 8.309741e-04 0.9577 886 0.0000 8.163647e-04 0.9824 887 0.0000 8.268292e-04 1.0128 888 0.0000 8.249113e-04 0.9977 889 0.0000 8.431913e-04 1.0222 890 0.0000 8.093790e-04 0.9599 891 0.0000 8.053845e-04 0.9951 892 0.0000 8.313124e-04 1.0322 893 0.0000 7.973871e-04 0.9592 894 0.0000 7.883449e-04 0.9887 895 0.0000 7.711701e-04 0.9782 896 0.0000 7.718840e-04 1.0009 897 0.0000 7.925186e-04 1.0267 898 0.0000 8.114793e-04 1.0239 899 0.0000 8.119894e-04 1.0006 900 0.0000 7.745538e-04 0.9539 901 0.0000 7.748123e-04 1.0003 902 0.0000 7.678743e-04 0.9910 903 0.0000 7.640441e-04 0.9950 904 0.0000 7.755048e-04 1.0150 905 0.0000 7.891542e-04 1.0176 906 0.0000 7.742804e-04 0.9812 907 0.0000 7.525522e-04 0.9719 908 0.0000 7.566220e-04 1.0054 909 0.0000 7.469665e-04 0.9872 910 0.0000 7.290903e-04 0.9761 911 0.0000 7.522938e-04 1.0318 912 0.0000 7.442352e-04 0.9893 913 0.0000 7.108979e-04 0.9552 914 0.0000 7.095701e-04 0.9981 915 0.0000 7.070365e-04 0.9964 916 0.0000 7.088656e-04 1.0026 917 0.0000 6.982154e-04 0.9850 918 0.0000 6.753450e-04 0.9672 919 0.0000 6.675135e-04 0.9884 920 0.0000 6.671596e-04 0.9995 921 0.0000 6.708132e-04 1.0055 922 0.0000 6.444531e-04 0.9607 923 0.0000 6.226790e-04 0.9662 924 0.0000 6.230278e-04 1.0006 925 0.0000 6.163252e-04 0.9892 926 0.0000 6.022176e-04 0.9771 927 0.0000 5.991566e-04 0.9949 928 0.0000 5.912835e-04 0.9869 929 0.0000 5.831130e-04 0.9862 930 0.0000 5.700193e-04 0.9775 931 0.0000 5.571408e-04 0.9774 932 0.0000 5.691536e-04 1.0216 933 0.0000 5.579242e-04 0.9803 934 0.0000 5.528541e-04 0.9909 935 0.0000 5.490729e-04 0.9932 936 0.0000 5.333079e-04 0.9713 937 0.0000 5.326629e-04 0.9988 938 0.0000 5.286821e-04 0.9925 939 0.0000 5.300768e-04 1.0026 940 0.0000 5.217623e-04 0.9843 941 0.0000 5.058423e-04 0.9695 942 0.0000 4.945977e-04 0.9778 943 0.0000 4.943199e-04 0.9994 944 0.0000 5.014849e-04 1.0145 945 0.0000 4.948979e-04 0.9869 946 0.0000 4.834331e-04 0.9768 947 0.0000 4.703484e-04 0.9729 948 0.0000 4.760264e-04 1.0121 949 0.0000 4.698934e-04 0.9871 950 0.0000 4.603654e-04 0.9797 951 0.0000 4.505246e-04 0.9786 952 0.0000 4.449649e-04 0.9877 953 0.0000 4.477592e-04 1.0063 954 0.0000 4.486911e-04 1.0021 955 0.0000 4.348044e-04 0.9691 956 0.0000 4.218062e-04 0.9701 957 0.0000 4.101086e-04 0.9723 958 0.0000 4.066455e-04 0.9916 959 0.0000 4.043486e-04 0.9944 960 0.0000 3.920222e-04 0.9695 961 0.0000 3.847938e-04 0.9816 962 0.0000 3.794788e-04 0.9862 963 0.0000 3.799792e-04 1.0013 964 0.0000 3.839192e-04 1.0104 965 0.0000 3.715595e-04 0.9678 966 0.0000 3.647828e-04 0.9818 967 0.0000 3.574942e-04 0.9800 968 0.0000 3.566596e-04 0.9977 969 0.0000 3.477963e-04 0.9751 970 0.0000 3.457679e-04 0.9942 971 0.0000 3.517328e-04 1.0173 972 0.0000 3.450002e-04 0.9809 973 0.0000 3.401140e-04 0.9858 974 0.0000 3.331621e-04 0.9796 975 0.0000 3.234024e-04 0.9707 976 0.0000 3.258192e-04 1.0075 977 0.0000 3.350614e-04 1.0284 978 0.0000 3.334387e-04 0.9952 979 0.0000 3.293452e-04 0.9877 980 0.0000 3.211299e-04 0.9751 981 0.0000 3.210761e-04 0.9998 982 0.0000 3.164181e-04 0.9855 983 0.0000 3.050857e-04 0.9642 984 0.0000 3.012009e-04 0.9873 985 0.0000 3.024822e-04 1.0043 986 0.0000 3.026344e-04 1.0005 987 0.0000 3.086870e-04 1.0200 988 0.0000 3.049964e-04 0.9880 989 0.0000 2.873506e-04 0.9421 990 0.0000 2.834352e-04 0.9864 991 0.0000 2.808062e-04 0.9907 992 0.0000 2.793854e-04 0.9949 993 0.0000 2.865596e-04 1.0257 994 0.0000 2.866176e-04 1.0002 995 0.0000 2.788491e-04 0.9729 996 0.0000 2.758293e-04 0.9892 997 0.0000 2.763111e-04 1.0017 998 0.0000 2.682861e-04 0.9710 999 0.0000 2.601792e-04 0.9698 1000 0.0000 2.596169e-04 0.9978 1001 0.0000 2.616959e-04 1.0080 1002 0.0000 2.535893e-04 0.9690 1003 0.0000 2.456781e-04 0.9688 1004 0.0000 2.442945e-04 0.9944 1005 0.0000 2.418638e-04 0.9900 1006 0.0000 2.453883e-04 1.0146 1007 0.0000 2.476002e-04 1.0090 1008 0.0000 2.420985e-04 0.9778 1009 0.0000 2.323620e-04 0.9598 1010 0.0000 2.233616e-04 0.9613 1011 0.0000 2.289190e-04 1.0249 1012 0.0000 2.321657e-04 1.0142 1013 0.0000 2.274925e-04 0.9799 1014 0.0000 2.191199e-04 0.9632 1015 0.0000 2.175438e-04 0.9928 1016 0.0000 2.220332e-04 1.0206 1017 0.0000 2.172046e-04 0.9783 1018 0.0000 2.097659e-04 0.9658 1019 0.0000 2.036157e-04 0.9707 1020 0.0000 1.994712e-04 0.9796 1021 0.0000 1.959702e-04 0.9824 1022 0.0000 1.930542e-04 0.9851 1023 0.0000 1.909361e-04 0.9890 1024 0.0000 1.903897e-04 0.9971 1025 0.0000 1.917316e-04 1.0070 1026 0.0000 1.910562e-04 0.9965 1027 0.0000 1.833002e-04 0.9594 1028 0.0000 1.776087e-04 0.9689 1029 0.0000 1.754119e-04 0.9876 1030 0.0000 1.745686e-04 0.9952 1031 0.0000 1.718461e-04 0.9844 1032 0.0000 1.664330e-04 0.9685 1033 0.0000 1.644253e-04 0.9879 1034 0.0000 1.606550e-04 0.9771 1035 0.0000 1.561615e-04 0.9720 1036 0.0000 1.522687e-04 0.9751 1037 0.0000 1.486872e-04 0.9765 1038 0.0000 1.492920e-04 1.0041 1039 0.0000 1.525888e-04 1.0221 1040 0.0000 1.458966e-04 0.9561 1041 0.0000 1.426591e-04 0.9778 1042 0.0000 1.399706e-04 0.9812 1043 0.0000 1.379055e-04 0.9852 1044 0.0000 1.389959e-04 1.0079 1045 0.0000 1.386503e-04 0.9975 1046 0.0000 1.380247e-04 0.9955 1047 0.0000 1.357530e-04 0.9835 1048 0.0000 1.356600e-04 0.9993 1049 0.0000 1.340695e-04 0.9883 1050 0.0000 1.305877e-04 0.9740 1051 0.0000 1.261571e-04 0.9661 1052 0.0000 1.238009e-04 0.9813 1053 0.0000 1.238562e-04 1.0004 1054 0.0000 1.280320e-04 1.0337 1055 0.0000 1.288540e-04 1.0064 1056 0.0000 1.192187e-04 0.9252 1057 0.0000 1.171484e-04 0.9826 1058 0.0000 1.177575e-04 1.0052 1059 0.0000 1.160407e-04 0.9854 1060 0.0000 1.192555e-04 1.0277 1061 0.0000 1.145663e-04 0.9607 1062 0.0000 1.119191e-04 0.9769 1063 0.0000 1.113072e-04 0.9945 1064 0.0000 1.066629e-04 0.9583 1065 0.0000 1.082836e-04 1.0152 1066 0.0000 1.094147e-04 1.0104 1067 0.0000 1.078406e-04 0.9856 1068 0.0000 1.059389e-04 0.9824 1069 0.0000 1.025855e-04 0.9683 1070 0.0000 1.065688e-04 1.0388 1071 0.0000 1.038323e-04 0.9743 1072 0.0000 1.005078e-04 0.9680 1073 0.0000 9.960116e-05 0.9910 1074 0.0000 9.555465e-05 0.9594 1075 0.0000 9.389676e-05 0.9826 1076 0.0000 9.329190e-05 0.9936 1077 0.0000 9.168408e-05 0.9828 1078 0.0000 9.045320e-05 0.9866 1079 0.0000 9.012966e-05 0.9964 1080 0.0000 8.915993e-05 0.9892 1081 0.0000 8.807006e-05 0.9878 1082 0.0000 8.689666e-05 0.9867 1083 0.0000 8.928345e-05 1.0275 1084 0.0000 8.881869e-05 0.9948 1085 0.0000 8.305333e-05 0.9351 1086 0.0000 8.164746e-05 0.9831 1087 0.0000 8.361787e-05 1.0241 1088 0.0000 8.482257e-05 1.0144 1089 0.0000 8.102409e-05 0.9552 1090 0.0000 7.804316e-05 0.9632 1091 0.0000 7.777590e-05 0.9966 1092 0.0000 7.698070e-05 0.9898 1093 0.0000 7.617721e-05 0.9896 1094 0.0000 7.785015e-05 1.0220 1095 0.0000 7.908359e-05 1.0158 1096 0.0000 7.586426e-05 0.9593 1097 0.0000 7.311162e-05 0.9637 1098 0.0000 7.077980e-05 0.9681 1099 0.0000 7.057252e-05 0.9971 1100 0.0000 7.018112e-05 0.9945 1101 0.0000 6.975981e-05 0.9940 1102 0.0000 7.069918e-05 1.0135 1103 0.0000 6.917076e-05 0.9784 1104 0.0000 6.798418e-05 0.9828 1105 0.0000 6.724114e-05 0.9891 1106 0.0000 6.722599e-05 0.9998 1107 0.0000 6.703655e-05 0.9972 1108 0.0000 6.766579e-05 1.0094 1109 0.0000 6.728036e-05 0.9943 1110 0.0000 6.337604e-05 0.9420 1111 0.0000 6.185643e-05 0.9760 1112 0.0000 6.180634e-05 0.9992 1113 0.0000 6.237878e-05 1.0093 1114 0.0000 6.059123e-05 0.9713 1115 0.0000 5.896654e-05 0.9732 1116 0.0000 5.900299e-05 1.0006 1117 0.0000 5.988532e-05 1.0150 1118 0.0000 5.821689e-05 0.9721 1119 0.0000 5.568219e-05 0.9565 1120 0.0000 5.459460e-05 0.9805 1121 0.0000 5.404070e-05 0.9899 1122 0.0000 5.327976e-05 0.9859 1123 0.0000 5.329041e-05 1.0002 1124 0.0000 5.310982e-05 0.9966 1125 0.0000 5.209220e-05 0.9808 1126 0.0000 5.205710e-05 0.9993 1127 0.0000 5.152420e-05 0.9898 1128 0.0000 5.079353e-05 0.9858 1129 0.0000 4.902484e-05 0.9652 1130 0.0000 4.714705e-05 0.9617 1131 0.0000 4.671814e-05 0.9909 1132 0.0000 4.722716e-05 1.0109 1133 0.0000 4.672086e-05 0.9893 1134 0.0000 4.591729e-05 0.9828 1135 0.0000 4.543921e-05 0.9896 1136 0.0000 4.455681e-05 0.9806 1137 0.0000 4.512239e-05 1.0127 1138 0.0000 4.406280e-05 0.9765 1139 0.0000 4.304828e-05 0.9770 1140 0.0000 4.228005e-05 0.9822 1141 0.0000 4.117294e-05 0.9738 1142 0.0000 4.094077e-05 0.9944 1143 0.0000 4.121272e-05 1.0066 1144 0.0000 4.008860e-05 0.9727 1145 0.0000 3.926912e-05 0.9796 1146 0.0000 3.884349e-05 0.9892 1147 0.0000 3.774906e-05 0.9718 1148 0.0000 3.682124e-05 0.9754 1149 0.0000 3.645388e-05 0.9900 1150 0.0000 3.626818e-05 0.9949 1151 0.0000 3.691003e-05 1.0177 1152 0.0000 3.568361e-05 0.9668 1153 0.0000 3.417539e-05 0.9577 1154 0.0000 3.359662e-05 0.9831 1155 0.0000 3.366826e-05 1.0021 1156 0.0000 3.365647e-05 0.9996 1157 0.0000 3.293674e-05 0.9786 1158 0.0000 3.176295e-05 0.9644 1159 0.0000 3.052760e-05 0.9611 1160 0.0000 3.032254e-05 0.9933 1161 0.0000 2.970485e-05 0.9796 1162 0.0000 2.880702e-05 0.9698 1163 0.0000 2.869126e-05 0.9960 1164 0.0000 2.866974e-05 0.9992 1165 0.0000 2.925789e-05 1.0205 1166 0.0000 2.978360e-05 1.0180 1167 0.0000 2.862327e-05 0.9610 1168 0.0000 2.773773e-05 0.9691 1169 0.0000 2.713071e-05 0.9781 1170 0.0000 2.739368e-05 1.0097 1171 0.0000 2.752526e-05 1.0048 1172 0.0000 2.726573e-05 0.9906 1173 0.0000 2.669564e-05 0.9791 1174 0.0000 2.588392e-05 0.9696 1175 0.0000 2.542371e-05 0.9822 1176 0.0000 2.491846e-05 0.9801 1177 0.0000 2.492559e-05 1.0003 1178 0.0000 2.416940e-05 0.9697 1179 0.0000 2.377181e-05 0.9835 1180 0.0000 2.363671e-05 0.9943 1181 0.0000 2.398205e-05 1.0146 1182 0.0000 2.405632e-05 1.0031 1183 0.0000 2.271367e-05 0.9442 1184 0.0000 2.249473e-05 0.9904 1185 0.0000 2.231221e-05 0.9919 1186 0.0000 2.193788e-05 0.9832 1187 0.0000 2.178802e-05 0.9932 1188 0.0000 2.121793e-05 0.9738 1189 0.0000 2.104937e-05 0.9921 1190 0.0000 2.149159e-05 1.0210 1191 0.0000 2.074685e-05 0.9653 1192 0.0000 2.000348e-05 0.9642 1193 0.0000 2.010386e-05 1.0050 1194 0.0000 1.991699e-05 0.9907 1195 0.0000 1.964748e-05 0.9865 1196 0.0000 1.946989e-05 0.9910 1197 0.0000 1.925539e-05 0.9890 1198 0.0000 1.917586e-05 0.9959 1199 0.0000 1.894290e-05 0.9879 1200 0.0000 1.899356e-05 1.0027 1201 0.0000 1.884265e-05 0.9921 1202 0.0000 1.807620e-05 0.9593 1203 0.0000 1.796752e-05 0.9940 1204 0.0000 1.803827e-05 1.0039 1205 0.0000 1.774020e-05 0.9835 1206 0.0000 1.721043e-05 0.9701 1207 0.0000 1.674415e-05 0.9729 1208 0.0000 1.658830e-05 0.9907 1209 0.0000 1.667041e-05 1.0050 1210 0.0000 1.703125e-05 1.0216 1211 0.0000 1.649411e-05 0.9685 1212 0.0000 1.617734e-05 0.9808 1213 0.0000 1.657556e-05 1.0246 1214 0.0000 1.634708e-05 0.9862 1215 0.0000 1.587665e-05 0.9712 1216 0.0000 1.564958e-05 0.9857 1217 0.0000 1.550977e-05 0.9911 1218 0.0000 1.568440e-05 1.0113 1219 0.0000 1.520927e-05 0.9697 1220 0.0000 1.467077e-05 0.9646 1221 0.0000 1.451754e-05 0.9896 1222 0.0000 1.433151e-05 0.9872 1223 0.0000 1.443223e-05 1.0070 1224 0.0000 1.450418e-05 1.0050 1225 0.0000 1.426128e-05 0.9833 1226 0.0000 1.393596e-05 0.9772 1227 0.0000 1.368879e-05 0.9823 1228 0.0000 1.374039e-05 1.0038 1229 0.0000 1.369593e-05 0.9968 1230 0.0000 1.353448e-05 0.9882 ---------------------------------------------------------------------- Total Iterations: 1231 Avg Convergence Rate: 0.9888 Final Residual: 1.353448e-05 Total Reduction in Residual: 9.996766e-07 Maximum Memory Usage: 0.000 GB ---------------------------------------------------------------------- Total Time: 5.72287 setup: 0.000170048 s solve: 5.7227 s solve(per iteration): 0.00464882 s

If I use GMRES_AMG_D2.json it took 16s AMGX version 2.5.0 Built on Apr 25 2024, 13:20:11 Compiled with CUDA Runtime 12.4, using CUDA driver 12.4 AMG Grid: Number of Levels: 9 LVL ROWS NNZ PARTS SPRSTY Mem (GB) ---------------------------------------------------------------------- 0(D) 1507876 113305856 1 4.98e-05 1.3 1(D) 281692 31103364 1 0.000392 0.706 2(D) 39859 4883517 1 0.00307 0.111 3(D) 4793 328295 1 0.0143 0.00753 4(D) 762 35364 1 0.0609 0.000822 5(D) 198 7160 1 0.183 0.000168 6(D) 60 1622 1 0.451 3.88e-05 7(D) 18 254 1 0.784 6.45e-06 8(D) 6 36 1 1 9.98e-07 ---------------------------------------------------------------------- Grid Complexity: 1.21712 Operator Complexity: 1.3209 Total Memory Usage: 2.12977 GB ---------------------------------------------------------------------- iter Mem Usage (GB) residual rate ---------------------------------------------------------------------- Ini 4.50732 1.353886e+01 0 4.50732 9.552636e+00 0.7056 1 4.5073 6.826744e+00 0.7146 2 4.5073 5.451789e+00 0.7986 3 4.5073 4.193551e+00 0.7692 4 4.5073 3.318682e+00 0.7914 5 4.5073 2.696656e+00 0.8126 6 4.5073 2.253779e+00 0.8358 7 4.5073 1.897404e+00 0.8419 8 4.5073 1.605763e+00 0.8463 9 4.5073 1.371794e+00 0.8543 10 4.5073 1.280317e+00 0.9333 11 4.5073 1.165342e+00 0.9102 12 4.5073 1.044937e+00 0.8967 13 4.5073 9.311771e-01 0.8911 14 4.5073 8.317691e-01 0.8932 15 4.5073 7.104873e-01 0.8542 16 4.5073 6.342603e-01 0.8927 17 4.5073 5.645147e-01 0.8900 18 4.5073 5.062215e-01 0.8967 19 4.5073 4.567174e-01 0.9022 20 4.5073 4.388230e-01 0.9608 21 4.5073 4.211402e-01 0.9597 22 4.5073 3.973845e-01 0.9436 23 4.5073 3.710982e-01 0.9339 24 4.5073 3.424965e-01 0.9229 25 4.5073 3.155958e-01 0.9215 26 4.5073 2.966112e-01 0.9398 27 4.5073 2.795076e-01 0.9423 28 4.5073 2.635899e-01 0.9431 29 4.5073 2.461252e-01 0.9337 30 4.5073 2.365946e-01 0.9613 31 4.5073 2.264181e-01 0.9570 32 4.5073 2.144454e-01 0.9471 33 4.5073 2.034822e-01 0.9489 34 4.5073 1.939666e-01 0.9532 35 4.5073 1.830933e-01 0.9439 36 4.5073 1.759580e-01 0.9610 37 4.5073 1.689407e-01 0.9601 38 4.5073 1.625273e-01 0.9620 39 4.5073 1.567911e-01 0.9647 40 4.5073 1.523330e-01 0.9716 41 4.5073 1.482169e-01 0.9730 42 4.5073 1.430960e-01 0.9655 43 4.5073 1.379123e-01 0.9638 44 4.5073 1.318838e-01 0.9563 45 4.5073 1.262892e-01 0.9576 46 4.5073 1.216729e-01 0.9634 47 4.5073 1.172642e-01 0.9638 48 4.5073 1.136271e-01 0.9690 49 4.5073 1.098468e-01 0.9667 50 4.5073 1.073779e-01 0.9775 51 4.5073 1.044496e-01 0.9727 52 4.5073 1.007739e-01 0.9648 53 4.5073 9.737754e-02 0.9663 54 4.5073 9.470861e-02 0.9726 55 4.5073 9.166303e-02 0.9678 56 4.5073 8.969872e-02 0.9786 57 4.5073 8.761253e-02 0.9767 58 4.5073 8.539046e-02 0.9746 59 4.5073 8.346770e-02 0.9775 60 4.5073 8.175577e-02 0.9795 61 4.5073 8.011439e-02 0.9799 62 4.5073 7.851187e-02 0.9800 63 4.5073 7.699035e-02 0.9806 64 4.5073 7.499845e-02 0.9741 65 4.5073 7.315360e-02 0.9754 66 4.5073 7.119191e-02 0.9732 67 4.5073 6.927591e-02 0.9731 68 4.5073 6.783679e-02 0.9792 69 4.5073 6.647737e-02 0.9800 70 4.5073 6.545250e-02 0.9846 71 4.5073 6.426370e-02 0.9818 72 4.5073 6.257041e-02 0.9737 73 4.5073 6.100275e-02 0.9749 74 4.5073 5.987852e-02 0.9816 75 4.5073 5.862508e-02 0.9791 76 4.5073 5.780494e-02 0.9860 77 4.5073 5.687329e-02 0.9839 78 4.5073 5.563071e-02 0.9782 79 4.5073 5.456461e-02 0.9808 80 4.5073 5.351356e-02 0.9807 81 4.5073 5.255514e-02 0.9821 82 4.5073 5.177287e-02 0.9851 83 4.5073 5.106999e-02 0.9864 84 4.5073 5.012493e-02 0.9815 85 4.5073 4.921204e-02 0.9818 86 4.5073 4.799043e-02 0.9752 87 4.5073 4.670630e-02 0.9732 88 4.5073 4.583710e-02 0.9814 89 4.5073 4.502878e-02 0.9824 90 4.5073 4.445102e-02 0.9872 91 4.5073 4.367765e-02 0.9826 92 4.5073 4.254507e-02 0.9741 93 4.5073 4.147386e-02 0.9748 94 4.5073 4.078047e-02 0.9833 95 4.5073 4.004918e-02 0.9821 96 4.5073 3.958986e-02 0.9885 97 4.5073 3.904670e-02 0.9863 98 4.5073 3.819285e-02 0.9781 99 4.5073 3.743150e-02 0.9801 100 4.5073 3.671366e-02 0.9808 101 4.5073 3.597912e-02 0.9800 102 4.5073 3.552583e-02 0.9874 103 4.5073 3.510992e-02 0.9883 104 4.5073 3.452025e-02 0.9832 105 4.5073 3.394837e-02 0.9834 106 4.5073 3.301806e-02 0.9726 107 4.5073 3.206440e-02 0.9711 108 4.5073 3.147588e-02 0.9816 109 4.5073 3.095063e-02 0.9833 110 4.5073 3.051699e-02 0.9860 111 4.5073 3.003388e-02 0.9842 112 4.5073 2.918968e-02 0.9719 113 4.5073 2.837800e-02 0.9722 114 4.5073 2.792080e-02 0.9839 115 4.5073 2.745213e-02 0.9832 116 4.5073 2.715536e-02 0.9892 117 4.5073 2.684906e-02 0.9887 118 4.5073 2.618969e-02 0.9754 119 4.5073 2.561850e-02 0.9782 120 4.5073 2.506056e-02 0.9782 121 4.5073 2.451296e-02 0.9781 122 4.5073 2.423323e-02 0.9886 123 4.5073 2.397209e-02 0.9892 124 4.5073 2.358096e-02 0.9837 125 4.5073 2.317335e-02 0.9827 126 4.5073 2.252357e-02 0.9720 127 4.5073 2.182169e-02 0.9688 128 4.5073 2.142175e-02 0.9817 129 4.5073 2.106657e-02 0.9834 130 4.5073 2.076983e-02 0.9859 131 4.5073 2.043719e-02 0.9840 132 4.5073 1.982875e-02 0.9702 133 4.5073 1.925887e-02 0.9713 134 4.5073 1.892543e-02 0.9827 135 4.5073 1.861500e-02 0.9836 136 4.5073 1.841600e-02 0.9893 137 4.5073 1.822721e-02 0.9897 138 4.5073 1.774987e-02 0.9738 139 4.5073 1.723753e-02 0.9711 140 4.5073 1.675373e-02 0.9719 141 4.5073 1.636719e-02 0.9769 142 4.5073 1.617550e-02 0.9883 143 4.5073 1.600921e-02 0.9897 144 4.5073 1.578544e-02 0.9860 145 4.5073 1.549496e-02 0.9816 146 4.5073 1.508306e-02 0.9734 147 4.5073 1.456481e-02 0.9656 148 4.5073 1.427043e-02 0.9798 149 4.5073 1.402647e-02 0.9829 150 4.5073 1.384524e-02 0.9871 151 4.5073 1.359538e-02 0.9820 152 4.5073 1.315202e-02 0.9674 153 4.5073 1.278672e-02 0.9722 154 4.5073 1.253887e-02 0.9806 155 4.5073 1.235779e-02 0.9856 156 4.5073 1.223345e-02 0.9899 157 4.5073 1.210488e-02 0.9895 158 4.5073 1.180894e-02 0.9756 159 4.5073 1.147988e-02 0.9721 160 4.5073 1.116957e-02 0.9730 161 4.5073 1.092455e-02 0.9781 162 4.5073 1.079219e-02 0.9879 163 4.5073 1.069048e-02 0.9906 164 4.5073 1.054792e-02 0.9867 165 4.5073 1.033906e-02 0.9802 166 4.5073 1.006528e-02 0.9735 167 4.5073 9.688645e-03 0.9626 168 4.5073 9.485966e-03 0.9791 169 4.5073 9.321227e-03 0.9826 170 4.5073 9.195419e-03 0.9865 171 4.5073 9.021929e-03 0.9811 172 4.5073 8.718005e-03 0.9663 173 4.5073 8.466200e-03 0.9711 174 4.5073 8.285718e-03 0.9787 175 4.5073 8.173952e-03 0.9865 176 4.5073 8.095984e-03 0.9905 177 4.5073 8.004101e-03 0.9887 178 4.5073 7.829951e-03 0.9782 179 4.5073 7.643146e-03 0.9761 180 4.5073 7.462735e-03 0.9764 181 4.5073 7.309771e-03 0.9795 182 4.5073 7.223806e-03 0.9882 183 4.5073 7.155744e-03 0.9906 184 4.5073 7.058219e-03 0.9864 185 4.5073 6.910895e-03 0.9791 186 4.5073 6.705845e-03 0.9703 187 4.5073 6.452930e-03 0.9623 188 4.5073 6.324550e-03 0.9801 189 4.5073 6.209451e-03 0.9818 190 4.5073 6.117331e-03 0.9852 191 4.5073 6.002762e-03 0.9813 192 4.5073 5.795177e-03 0.9654 193 4.5073 5.617591e-03 0.9694 194 4.5073 5.492606e-03 0.9778 195 4.5073 5.417069e-03 0.9862 196 4.5073 5.365974e-03 0.9906 197 4.5073 5.306792e-03 0.9890 198 4.5073 5.199231e-03 0.9797 199 4.5073 5.089314e-03 0.9789 200 4.5073 4.984597e-03 0.9794 201 4.5073 4.885234e-03 0.9801 202 4.5073 4.831392e-03 0.9890 203 4.5073 4.785620e-03 0.9905 204 4.5073 4.720878e-03 0.9865 205 4.5073 4.617083e-03 0.9780 206 4.5073 4.473098e-03 0.9688 207 4.5073 4.298896e-03 0.9611 208 4.5073 4.211524e-03 0.9797 209 4.5073 4.126273e-03 0.9798 210 4.5073 4.054650e-03 0.9826 211 4.5073 3.976860e-03 0.9808 212 4.5073 3.832365e-03 0.9637 213 4.5073 3.712143e-03 0.9686 214 4.5073 3.627296e-03 0.9771 215 4.5073 3.576847e-03 0.9861 216 4.5073 3.542947e-03 0.9905 217 4.5073 3.507197e-03 0.9899 218 4.5073 3.438890e-03 0.9805 219 4.5073 3.373210e-03 0.9809 220 4.5073 3.309594e-03 0.9811 221 4.5073 3.246149e-03 0.9808 222 4.5073 3.212992e-03 0.9898 223 4.5073 3.182623e-03 0.9905 224 4.5073 3.138007e-03 0.9860 225 4.5073 3.063484e-03 0.9763 226 4.5073 2.962185e-03 0.9669 227 4.5073 2.842631e-03 0.9596 228 4.5073 2.782643e-03 0.9789 229 4.5073 2.721328e-03 0.9780 230 4.5073 2.669796e-03 0.9811 231 4.5073 2.615614e-03 0.9797 232 4.5073 2.516624e-03 0.9622 233 4.5073 2.433075e-03 0.9668 234 4.5073 2.372982e-03 0.9753 235 4.5073 2.338936e-03 0.9857 236 4.5073 2.316631e-03 0.9905 237 4.5073 2.293694e-03 0.9901 238 4.5073 2.248138e-03 0.9801 239 4.5073 2.206995e-03 0.9817 240 4.5073 2.167269e-03 0.9820 241 4.5073 2.124825e-03 0.9804 242 4.5073 2.103450e-03 0.9899 243 4.5073 2.083674e-03 0.9906 244 4.5073 2.054267e-03 0.9859 245 4.5073 2.001011e-03 0.9741 246 4.5073 1.930307e-03 0.9647 247 4.5073 1.848721e-03 0.9577 248 4.5073 1.808268e-03 0.9781 249 4.5073 1.763726e-03 0.9754 250 4.5073 1.726539e-03 0.9789 251 4.5073 1.689947e-03 0.9788 252 4.5073 1.624000e-03 0.9610 253 4.5073 1.565009e-03 0.9637 254 4.5073 1.522738e-03 0.9730 255 4.5073 1.500731e-03 0.9855 256 4.5073 1.486607e-03 0.9906 257 4.5073 1.472336e-03 0.9904 258 4.5073 1.442995e-03 0.9801 259 4.5073 1.401880e-03 0.9715 260 4.5073 1.363245e-03 0.9724 261 4.5073 1.338839e-03 0.9821 262 4.5073 1.324971e-03 0.9896 263 4.5073 1.314374e-03 0.9920 264 4.5073 1.297219e-03 0.9869 265 4.5073 1.265595e-03 0.9756 266 4.5073 1.219403e-03 0.9635 267 4.5073 1.165827e-03 0.9561 268 4.5073 1.136158e-03 0.9746 269 4.5073 1.106047e-03 0.9735 270 4.5073 1.081060e-03 0.9774 271 4.5073 1.057849e-03 0.9785 272 4.5073 1.011653e-03 0.9563 273 4.5073 9.744879e-04 0.9633 274 4.5073 9.499882e-04 0.9749 275 4.5073 9.369585e-04 0.9863 276 4.5073 9.290940e-04 0.9916 277 4.5073 9.204583e-04 0.9907 278 4.5073 9.048393e-04 0.9830 279 4.5073 8.898536e-04 0.9834 280 4.5073 8.755330e-04 0.9839 281 4.5073 8.611554e-04 0.9836 282 4.5073 8.530492e-04 0.9906 283 4.5073 8.459542e-04 0.9917 284 4.5073 8.339620e-04 0.9858 285 4.5073 8.123608e-04 0.9741 286 4.5073 7.823079e-04 0.9630 287 4.5073 7.475207e-04 0.9555 288 4.5073 7.289601e-04 0.9752 289 4.5073 7.082735e-04 0.9716 290 4.5073 6.906207e-04 0.9751 291 4.5073 6.748595e-04 0.9772 292 4.5073 6.470612e-04 0.9588 293 4.5073 6.235908e-04 0.9637 294 4.5073 6.062682e-04 0.9722 295 4.5073 5.972097e-04 0.9851 296 4.5073 5.916863e-04 0.9908 297 4.5073 5.861331e-04 0.9906 298 4.5073 5.771284e-04 0.9846 299 4.5073 5.698246e-04 0.9873 300 4.5073 5.629160e-04 0.9879 301 4.5073 5.548604e-04 0.9857 302 4.5073 5.496628e-04 0.9906 303 4.5073 5.443566e-04 0.9903 304 4.5073 5.351942e-04 0.9832 305 4.5073 5.197481e-04 0.9711 306 4.5073 5.013518e-04 0.9646 307 4.5073 4.820704e-04 0.9615 308 4.5073 4.703863e-04 0.9758 309 4.5073 4.551120e-04 0.9675 310 4.5073 4.420310e-04 0.9713 311 4.5073 4.321963e-04 0.9778 312 4.5073 4.173260e-04 0.9656 313 4.5073 4.027740e-04 0.9651 314 4.5073 3.902635e-04 0.9689 315 4.5073 3.834007e-04 0.9824 316 4.5073 3.794524e-04 0.9897 317 4.5073 3.759048e-04 0.9907 318 4.5073 3.706544e-04 0.9860 319 4.5073 3.662881e-04 0.9882 320 4.5073 3.621661e-04 0.9887 321 4.5073 3.575503e-04 0.9873 322 4.5073 3.542326e-04 0.9907 323 4.5073 3.505049e-04 0.9895 324 4.5073 3.438591e-04 0.9810 325 4.5073 3.327386e-04 0.9677 326 4.5073 3.209327e-04 0.9645 327 4.5073 3.097232e-04 0.9651 328 4.5073 3.025474e-04 0.9768 329 4.5073 2.915968e-04 0.9638 330 4.5073 2.822276e-04 0.9679 331 4.5073 2.758958e-04 0.9776 332 4.5073 2.670500e-04 0.9679 333 4.5073 2.577056e-04 0.9650 334 4.5073 2.490600e-04 0.9665 335 4.5073 2.442267e-04 0.9806 336 4.5073 2.416747e-04 0.9896 337 4.5073 2.395729e-04 0.9913 338 4.5073 2.366283e-04 0.9877 339 4.5073 2.329755e-04 0.9846 340 4.5073 2.295243e-04 0.9852 341 4.5073 2.271336e-04 0.9896 342 4.5073 2.249478e-04 0.9904 343 4.5073 2.225149e-04 0.9892 344 4.5073 2.183825e-04 0.9814 345 4.5073 2.115974e-04 0.9689 346 4.5073 2.037608e-04 0.9630 347 4.5073 1.961096e-04 0.9624 348 4.5073 1.915623e-04 0.9768 349 4.5073 1.844891e-04 0.9631 350 4.5073 1.789415e-04 0.9699 351 4.5073 1.747135e-04 0.9764 352 4.5073 1.685326e-04 0.9646 353 4.5073 1.625619e-04 0.9646 354 4.5073 1.573789e-04 0.9681 355 4.5073 1.543450e-04 0.9807 356 4.5073 1.527411e-04 0.9896 357 4.5073 1.514816e-04 0.9918 358 4.5073 1.498667e-04 0.9893 359 4.5073 1.482822e-04 0.9894 360 4.5073 1.467254e-04 0.9895 361 4.5073 1.453226e-04 0.9904 362 4.5073 1.440473e-04 0.9912 363 4.5073 1.425693e-04 0.9897 364 4.5073 1.399659e-04 0.9817 365 4.5073 1.354891e-04 0.9680 366 4.5073 1.304395e-04 0.9627 367 4.5073 1.255801e-04 0.9627 368 4.5073 1.224384e-04 0.9750 369 4.5073 1.178715e-04 0.9627 370 4.5073 1.141439e-04 0.9684 371 4.5073 1.114363e-04 0.9763 372 4.5073 1.074004e-04 0.9638 373 4.5073 1.037559e-04 0.9661 374 4.5073 1.005130e-04 0.9687 375 4.5073 9.864468e-05 0.9814 376 4.5073 9.765982e-05 0.9900 377 4.5073 9.693787e-05 0.9926 378 4.5073 9.603671e-05 0.9907 379 4.5073 9.536281e-05 0.9930 380 4.5073 9.470601e-05 0.9931 381 4.5073 9.386835e-05 0.9912 382 4.5073 9.312684e-05 0.9921 383 4.5073 9.220860e-05 0.9901 384 4.5073 9.051789e-05 0.9817 385 4.5073 8.757986e-05 0.9675 386 4.5073 8.441697e-05 0.9639 387 4.5073 8.117176e-05 0.9616 388 4.5073 7.917326e-05 0.9754 389 4.5073 7.650414e-05 0.9663 390 4.5073 7.415814e-05 0.9693 391 4.5073 7.243978e-05 0.9768 392 4.5073 6.992122e-05 0.9652 393 4.5073 6.755464e-05 0.9662 394 4.5073 6.550298e-05 0.9696 395 4.5073 6.442420e-05 0.9835 396 4.5073 6.386114e-05 0.9913 397 4.5073 6.343917e-05 0.9934 398 4.5073 6.291857e-05 0.9918 399 4.5073 6.255101e-05 0.9942 400 4.5073 6.218764e-05 0.9942 401 4.5073 6.169533e-05 0.9921 402 4.5073 6.126233e-05 0.9930 403 4.5073 6.074052e-05 0.9915 404 4.5073 5.976100e-05 0.9839 405 4.5073 5.783132e-05 0.9677 406 4.5073 5.582332e-05 0.9653 407 4.5073 5.381541e-05 0.9640 408 4.5073 5.256528e-05 0.9768 409 4.5073 5.088652e-05 0.9681 410 4.5073 4.942387e-05 0.9713 411 4.5073 4.836018e-05 0.9785 412 4.5073 4.681841e-05 0.9681 413 4.5073 4.530922e-05 0.9678 414 4.5073 4.403902e-05 0.9720 415 4.5073 4.342640e-05 0.9861 416 4.5073 4.311669e-05 0.9929 417 4.5073 4.288333e-05 0.9946 418 4.5073 4.258149e-05 0.9930 419 4.5073 4.234146e-05 0.9944 420 4.5073 4.210422e-05 0.9944 421 4.5073 4.182142e-05 0.9933 422 4.5073 4.157963e-05 0.9942 423 4.5073 4.129297e-05 0.9931 424 4.5073 4.075008e-05 0.9869 425 4.5073 3.955404e-05 0.9706 426 4.5073 3.826284e-05 0.9674 427 4.5073 3.703658e-05 0.9680 428 4.5073 3.627819e-05 0.9795 429 4.5073 3.518842e-05 0.9700 430 4.5073 3.427321e-05 0.9740 431 4.5073 3.360933e-05 0.9806 432 4.5073 3.266358e-05 0.9719 433 4.5073 3.170647e-05 0.9707 434 4.5073 3.092739e-05 0.9754 435 4.5073 3.057961e-05 0.9888 436 4.5073 3.040358e-05 0.9942 437 4.5073 3.027417e-05 0.9957 438 4.5073 3.010033e-05 0.9943 439 4.5073 2.991853e-05 0.9940 440 4.5073 2.973998e-05 0.9940 441 4.5073 2.958330e-05 0.9947 442 4.5073 2.944665e-05 0.9954 443 4.5073 2.928091e-05 0.9944 444 4.5073 2.897940e-05 0.9897 445 4.5073 2.826589e-05 0.9754 446 4.5073 2.743132e-05 0.9705 447 4.5073 2.667133e-05 0.9723 448 4.5073 2.615545e-05 0.9807 449 4.5073 2.540920e-05 0.9715 450 4.5073 2.485503e-05 0.9782 451 4.5073 2.436528e-05 0.9803 452 4.5073 2.373976e-05 0.9743 453 4.5073 2.310008e-05 0.9731 454 4.5073 2.261866e-05 0.9792 455 4.5073 2.240689e-05 0.9906 456 4.5073 2.229318e-05 0.9949 457 4.5073 2.221321e-05 0.9964 458 4.5073 2.210959e-05 0.9953 459 4.5073 2.200217e-05 0.9951 460 4.5073 2.189792e-05 0.9953 461 4.5073 2.180671e-05 0.9958 462 4.5073 2.171675e-05 0.9959 463 4.5073 2.161412e-05 0.9953 464 4.5073 2.143114e-05 0.9915 465 4.5073 2.097284e-05 0.9786 466 4.5073 2.038027e-05 0.9717 467 4.5073 1.986920e-05 0.9749 468 4.5073 1.948462e-05 0.9806 469 4.5073 1.898508e-05 0.9744 470 4.5073 1.857710e-05 0.9785 471 4.5073 1.821628e-05 0.9806 472 4.5073 1.782268e-05 0.9784 473 4.5073 1.736033e-05 0.9741 474 4.5073 1.698073e-05 0.9781 475 4.5073 1.684377e-05 0.9919 476 4.5073 1.676710e-05 0.9954 477 4.5073 1.670770e-05 0.9965 478 4.5073 1.663904e-05 0.9959 479 4.5073 1.657661e-05 0.9962 480 4.5073 1.651758e-05 0.9964 481 4.5073 1.646211e-05 0.9966 482 4.5073 1.639271e-05 0.9958 483 4.5073 1.633027e-05 0.9962 484 4.5073 1.622154e-05 0.9933 485 4.5073 1.587925e-05 0.9789 486 4.5073 1.541874e-05 0.9710 487 4.5073 1.509442e-05 0.9790 488 4.5073 1.477210e-05 0.9786 489 4.5073 1.440651e-05 0.9753 490 4.5073 1.411757e-05 0.9799 491 4.5073 1.379559e-05 0.9772 492 4.5073 1.354723e-05 0.9820 493 4.5073 1.314833e-05 0.9706 ---------------------------------------------------------------------- Total Iterations: 494 Avg Convergence Rate: 0.9724 Final Residual: 1.314833e-05 Total Reduction in Residual: 9.711549e-07 Maximum Memory Usage: 4.507 GB ---------------------------------------------------------------------- Total Time: 16.6462 setup: 0.555882 s solve: 16.0903 s solve(per iteration): 0.0325714 s

Output for PCG_AGGREGATION_JACOBI.json (~16s as well): AMGX version 2.5.0 Built on Apr 25 2024, 13:20:11 Compiled with CUDA Runtime 12.4, using CUDA driver 12.4 AMG Grid: Number of Levels: 9 LVL ROWS NNZ PARTS SPRSTY Mem (GB) ---------------------------------------------------------------------- 0(D) 1507876 113305856 1 4.98e-05 1.3 1(D) 281692 31103364 1 0.000392 0.706 2(D) 39859 4883517 1 0.00307 0.111 3(D) 4793 328295 1 0.0143 0.00753 4(D) 762 35364 1 0.0609 0.000822 5(D) 198 7160 1 0.183 0.000168 6(D) 60 1622 1 0.451 3.88e-05 7(D) 18 254 1 0.784 6.45e-06 8(D) 6 36 1 1 9.98e-07 ---------------------------------------------------------------------- Grid Complexity: 1.21712 Operator Complexity: 1.3209 Total Memory Usage: 2.12977 GB ---------------------------------------------------------------------- iter Mem Usage (GB) residual rate ---------------------------------------------------------------------- Ini 4.50732 1.353886e+01 0 4.50732 9.552636e+00 0.7056 1 4.5073 6.826744e+00 0.7146 2 4.5073 5.451789e+00 0.7986 3 4.5073 4.193551e+00 0.7692 4 4.5073 3.318682e+00 0.7914 5 4.5073 2.696656e+00 0.8126 6 4.5073 2.253779e+00 0.8358 7 4.5073 1.897404e+00 0.8419 8 4.5073 1.605763e+00 0.8463 9 4.5073 1.371794e+00 0.8543 10 4.5073 1.280317e+00 0.9333 11 4.5073 1.165342e+00 0.9102 12 4.5073 1.044937e+00 0.8967 13 4.5073 9.311771e-01 0.8911 14 4.5073 8.317691e-01 0.8932 15 4.5073 7.104873e-01 0.8542 16 4.5073 6.342603e-01 0.8927 17 4.5073 5.645147e-01 0.8900 18 4.5073 5.062215e-01 0.8967 19 4.5073 4.567174e-01 0.9022 20 4.5073 4.388230e-01 0.9608 21 4.5073 4.211402e-01 0.9597 22 4.5073 3.973845e-01 0.9436 23 4.5073 3.710982e-01 0.9339 24 4.5073 3.424965e-01 0.9229 25 4.5073 3.155958e-01 0.9215 26 4.5073 2.966112e-01 0.9398 27 4.5073 2.795076e-01 0.9423 28 4.5073 2.635899e-01 0.9431 29 4.5073 2.461252e-01 0.9337 30 4.5073 2.365946e-01 0.9613 31 4.5073 2.264181e-01 0.9570 32 4.5073 2.144454e-01 0.9471 33 4.5073 2.034822e-01 0.9489 34 4.5073 1.939666e-01 0.9532 35 4.5073 1.830933e-01 0.9439 36 4.5073 1.759580e-01 0.9610 37 4.5073 1.689407e-01 0.9601 38 4.5073 1.625273e-01 0.9620 39 4.5073 1.567911e-01 0.9647 40 4.5073 1.523330e-01 0.9716 41 4.5073 1.482169e-01 0.9730 42 4.5073 1.430960e-01 0.9655 43 4.5073 1.379123e-01 0.9638 44 4.5073 1.318838e-01 0.9563 45 4.5073 1.262892e-01 0.9576 46 4.5073 1.216729e-01 0.9634 47 4.5073 1.172642e-01 0.9638 48 4.5073 1.136271e-01 0.9690 49 4.5073 1.098468e-01 0.9667 50 4.5073 1.073779e-01 0.9775 51 4.5073 1.044496e-01 0.9727 52 4.5073 1.007739e-01 0.9648 53 4.5073 9.737754e-02 0.9663 54 4.5073 9.470861e-02 0.9726 55 4.5073 9.166303e-02 0.9678 56 4.5073 8.969872e-02 0.9786 57 4.5073 8.761253e-02 0.9767 58 4.5073 8.539046e-02 0.9746 59 4.5073 8.346770e-02 0.9775 60 4.5073 8.175577e-02 0.9795 61 4.5073 8.011439e-02 0.9799 62 4.5073 7.851187e-02 0.9800 63 4.5073 7.699035e-02 0.9806 64 4.5073 7.499845e-02 0.9741 65 4.5073 7.315360e-02 0.9754 66 4.5073 7.119191e-02 0.9732 67 4.5073 6.927591e-02 0.9731 68 4.5073 6.783679e-02 0.9792 69 4.5073 6.647737e-02 0.9800 70 4.5073 6.545250e-02 0.9846 71 4.5073 6.426370e-02 0.9818 72 4.5073 6.257041e-02 0.9737 73 4.5073 6.100275e-02 0.9749 74 4.5073 5.987852e-02 0.9816 75 4.5073 5.862508e-02 0.9791 76 4.5073 5.780494e-02 0.9860 77 4.5073 5.687329e-02 0.9839 78 4.5073 5.563071e-02 0.9782 79 4.5073 5.456461e-02 0.9808 80 4.5073 5.351356e-02 0.9807 81 4.5073 5.255514e-02 0.9821 82 4.5073 5.177287e-02 0.9851 83 4.5073 5.106999e-02 0.9864 84 4.5073 5.012493e-02 0.9815 85 4.5073 4.921204e-02 0.9818 86 4.5073 4.799043e-02 0.9752 87 4.5073 4.670630e-02 0.9732 88 4.5073 4.583710e-02 0.9814 89 4.5073 4.502878e-02 0.9824 90 4.5073 4.445102e-02 0.9872 91 4.5073 4.367765e-02 0.9826 92 4.5073 4.254507e-02 0.9741 93 4.5073 4.147386e-02 0.9748 94 4.5073 4.078047e-02 0.9833 95 4.5073 4.004918e-02 0.9821 96 4.5073 3.958986e-02 0.9885 97 4.5073 3.904670e-02 0.9863 98 4.5073 3.819285e-02 0.9781 99 4.5073 3.743150e-02 0.9801 100 4.5073 3.671366e-02 0.9808 101 4.5073 3.597912e-02 0.9800 102 4.5073 3.552583e-02 0.9874 103 4.5073 3.510992e-02 0.9883 104 4.5073 3.452025e-02 0.9832 105 4.5073 3.394837e-02 0.9834 106 4.5073 3.301806e-02 0.9726 107 4.5073 3.206440e-02 0.9711 108 4.5073 3.147588e-02 0.9816 109 4.5073 3.095063e-02 0.9833 110 4.5073 3.051699e-02 0.9860 111 4.5073 3.003388e-02 0.9842 112 4.5073 2.918968e-02 0.9719 113 4.5073 2.837800e-02 0.9722 114 4.5073 2.792080e-02 0.9839 115 4.5073 2.745213e-02 0.9832 116 4.5073 2.715536e-02 0.9892 117 4.5073 2.684906e-02 0.9887 118 4.5073 2.618969e-02 0.9754 119 4.5073 2.561850e-02 0.9782 120 4.5073 2.506056e-02 0.9782 121 4.5073 2.451296e-02 0.9781 122 4.5073 2.423323e-02 0.9886 123 4.5073 2.397209e-02 0.9892 124 4.5073 2.358096e-02 0.9837 125 4.5073 2.317335e-02 0.9827 126 4.5073 2.252357e-02 0.9720 127 4.5073 2.182169e-02 0.9688 128 4.5073 2.142175e-02 0.9817 129 4.5073 2.106657e-02 0.9834 130 4.5073 2.076983e-02 0.9859 131 4.5073 2.043719e-02 0.9840 132 4.5073 1.982875e-02 0.9702 133 4.5073 1.925887e-02 0.9713 134 4.5073 1.892543e-02 0.9827 135 4.5073 1.861500e-02 0.9836 136 4.5073 1.841600e-02 0.9893 137 4.5073 1.822721e-02 0.9897 138 4.5073 1.774987e-02 0.9738 139 4.5073 1.723753e-02 0.9711 140 4.5073 1.675373e-02 0.9719 141 4.5073 1.636719e-02 0.9769 142 4.5073 1.617550e-02 0.9883 143 4.5073 1.600921e-02 0.9897 144 4.5073 1.578544e-02 0.9860 145 4.5073 1.549496e-02 0.9816 146 4.5073 1.508306e-02 0.9734 147 4.5073 1.456481e-02 0.9656 148 4.5073 1.427043e-02 0.9798 149 4.5073 1.402647e-02 0.9829 150 4.5073 1.384524e-02 0.9871 151 4.5073 1.359538e-02 0.9820 152 4.5073 1.315202e-02 0.9674 153 4.5073 1.278672e-02 0.9722 154 4.5073 1.253887e-02 0.9806 155 4.5073 1.235779e-02 0.9856 156 4.5073 1.223345e-02 0.9899 157 4.5073 1.210488e-02 0.9895 158 4.5073 1.180894e-02 0.9756 159 4.5073 1.147988e-02 0.9721 160 4.5073 1.116957e-02 0.9730 161 4.5073 1.092455e-02 0.9781 162 4.5073 1.079219e-02 0.9879 163 4.5073 1.069048e-02 0.9906 164 4.5073 1.054792e-02 0.9867 165 4.5073 1.033906e-02 0.9802 166 4.5073 1.006528e-02 0.9735 167 4.5073 9.688645e-03 0.9626 168 4.5073 9.485966e-03 0.9791 169 4.5073 9.321227e-03 0.9826 170 4.5073 9.195419e-03 0.9865 171 4.5073 9.021929e-03 0.9811 172 4.5073 8.718005e-03 0.9663 173 4.5073 8.466200e-03 0.9711 174 4.5073 8.285718e-03 0.9787 175 4.5073 8.173952e-03 0.9865 176 4.5073 8.095984e-03 0.9905 177 4.5073 8.004101e-03 0.9887 178 4.5073 7.829951e-03 0.9782 179 4.5073 7.643146e-03 0.9761 180 4.5073 7.462735e-03 0.9764 181 4.5073 7.309771e-03 0.9795 182 4.5073 7.223806e-03 0.9882 183 4.5073 7.155744e-03 0.9906 184 4.5073 7.058219e-03 0.9864 185 4.5073 6.910895e-03 0.9791 186 4.5073 6.705845e-03 0.9703 187 4.5073 6.452930e-03 0.9623 188 4.5073 6.324550e-03 0.9801 189 4.5073 6.209451e-03 0.9818 190 4.5073 6.117331e-03 0.9852 191 4.5073 6.002762e-03 0.9813 192 4.5073 5.795177e-03 0.9654 193 4.5073 5.617591e-03 0.9694 194 4.5073 5.492606e-03 0.9778 195 4.5073 5.417069e-03 0.9862 196 4.5073 5.365974e-03 0.9906 197 4.5073 5.306792e-03 0.9890 198 4.5073 5.199231e-03 0.9797 199 4.5073 5.089314e-03 0.9789 200 4.5073 4.984597e-03 0.9794 201 4.5073 4.885234e-03 0.9801 202 4.5073 4.831392e-03 0.9890 203 4.5073 4.785620e-03 0.9905 204 4.5073 4.720878e-03 0.9865 205 4.5073 4.617083e-03 0.9780 206 4.5073 4.473098e-03 0.9688 207 4.5073 4.298896e-03 0.9611 208 4.5073 4.211524e-03 0.9797 209 4.5073 4.126273e-03 0.9798 210 4.5073 4.054650e-03 0.9826 211 4.5073 3.976860e-03 0.9808 212 4.5073 3.832365e-03 0.9637 213 4.5073 3.712143e-03 0.9686 214 4.5073 3.627296e-03 0.9771 215 4.5073 3.576847e-03 0.9861 216 4.5073 3.542947e-03 0.9905 217 4.5073 3.507197e-03 0.9899 218 4.5073 3.438890e-03 0.9805 219 4.5073 3.373210e-03 0.9809 220 4.5073 3.309594e-03 0.9811 221 4.5073 3.246149e-03 0.9808 222 4.5073 3.212992e-03 0.9898 223 4.5073 3.182623e-03 0.9905 224 4.5073 3.138007e-03 0.9860 225 4.5073 3.063484e-03 0.9763 226 4.5073 2.962185e-03 0.9669 227 4.5073 2.842631e-03 0.9596 228 4.5073 2.782643e-03 0.9789 229 4.5073 2.721328e-03 0.9780 230 4.5073 2.669796e-03 0.9811 231 4.5073 2.615614e-03 0.9797 232 4.5073 2.516624e-03 0.9622 233 4.5073 2.433075e-03 0.9668 234 4.5073 2.372982e-03 0.9753 235 4.5073 2.338936e-03 0.9857 236 4.5073 2.316631e-03 0.9905 237 4.5073 2.293694e-03 0.9901 238 4.5073 2.248138e-03 0.9801 239 4.5073 2.206995e-03 0.9817 240 4.5073 2.167269e-03 0.9820 241 4.5073 2.124825e-03 0.9804 242 4.5073 2.103450e-03 0.9899 243 4.5073 2.083674e-03 0.9906 244 4.5073 2.054267e-03 0.9859 245 4.5073 2.001011e-03 0.9741 246 4.5073 1.930307e-03 0.9647 247 4.5073 1.848721e-03 0.9577 248 4.5073 1.808268e-03 0.9781 249 4.5073 1.763726e-03 0.9754 250 4.5073 1.726539e-03 0.9789 251 4.5073 1.689947e-03 0.9788 252 4.5073 1.624000e-03 0.9610 253 4.5073 1.565009e-03 0.9637 254 4.5073 1.522738e-03 0.9730 255 4.5073 1.500731e-03 0.9855 256 4.5073 1.486607e-03 0.9906 257 4.5073 1.472336e-03 0.9904 258 4.5073 1.442995e-03 0.9801 259 4.5073 1.401880e-03 0.9715 260 4.5073 1.363245e-03 0.9724 261 4.5073 1.338839e-03 0.9821 262 4.5073 1.324971e-03 0.9896 263 4.5073 1.314374e-03 0.9920 264 4.5073 1.297219e-03 0.9869 265 4.5073 1.265595e-03 0.9756 266 4.5073 1.219403e-03 0.9635 267 4.5073 1.165827e-03 0.9561 268 4.5073 1.136158e-03 0.9746 269 4.5073 1.106047e-03 0.9735 270 4.5073 1.081060e-03 0.9774 271 4.5073 1.057849e-03 0.9785 272 4.5073 1.011653e-03 0.9563 273 4.5073 9.744879e-04 0.9633 274 4.5073 9.499882e-04 0.9749 275 4.5073 9.369585e-04 0.9863 276 4.5073 9.290940e-04 0.9916 277 4.5073 9.204583e-04 0.9907 278 4.5073 9.048393e-04 0.9830 279 4.5073 8.898536e-04 0.9834 280 4.5073 8.755330e-04 0.9839 281 4.5073 8.611554e-04 0.9836 282 4.5073 8.530492e-04 0.9906 283 4.5073 8.459542e-04 0.9917 284 4.5073 8.339620e-04 0.9858 285 4.5073 8.123608e-04 0.9741 286 4.5073 7.823079e-04 0.9630 287 4.5073 7.475207e-04 0.9555 288 4.5073 7.289601e-04 0.9752 289 4.5073 7.082735e-04 0.9716 290 4.5073 6.906207e-04 0.9751 291 4.5073 6.748595e-04 0.9772 292 4.5073 6.470612e-04 0.9588 293 4.5073 6.235908e-04 0.9637 294 4.5073 6.062682e-04 0.9722 295 4.5073 5.972097e-04 0.9851 296 4.5073 5.916863e-04 0.9908 297 4.5073 5.861331e-04 0.9906 298 4.5073 5.771284e-04 0.9846 299 4.5073 5.698246e-04 0.9873 300 4.5073 5.629160e-04 0.9879 301 4.5073 5.548604e-04 0.9857 302 4.5073 5.496628e-04 0.9906 303 4.5073 5.443566e-04 0.9903 304 4.5073 5.351942e-04 0.9832 305 4.5073 5.197481e-04 0.9711 306 4.5073 5.013518e-04 0.9646 307 4.5073 4.820704e-04 0.9615 308 4.5073 4.703863e-04 0.9758 309 4.5073 4.551120e-04 0.9675 310 4.5073 4.420310e-04 0.9713 311 4.5073 4.321963e-04 0.9778 312 4.5073 4.173260e-04 0.9656 313 4.5073 4.027740e-04 0.9651 314 4.5073 3.902635e-04 0.9689 315 4.5073 3.834007e-04 0.9824 316 4.5073 3.794524e-04 0.9897 317 4.5073 3.759048e-04 0.9907 318 4.5073 3.706544e-04 0.9860 319 4.5073 3.662881e-04 0.9882 320 4.5073 3.621661e-04 0.9887 321 4.5073 3.575503e-04 0.9873 322 4.5073 3.542326e-04 0.9907 323 4.5073 3.505049e-04 0.9895 324 4.5073 3.438591e-04 0.9810 325 4.5073 3.327386e-04 0.9677 326 4.5073 3.209327e-04 0.9645 327 4.5073 3.097232e-04 0.9651 328 4.5073 3.025474e-04 0.9768 329 4.5073 2.915968e-04 0.9638 330 4.5073 2.822276e-04 0.9679 331 4.5073 2.758958e-04 0.9776 332 4.5073 2.670500e-04 0.9679 333 4.5073 2.577056e-04 0.9650 334 4.5073 2.490600e-04 0.9665 335 4.5073 2.442267e-04 0.9806 336 4.5073 2.416747e-04 0.9896 337 4.5073 2.395729e-04 0.9913 338 4.5073 2.366283e-04 0.9877 339 4.5073 2.329755e-04 0.9846 340 4.5073 2.295243e-04 0.9852 341 4.5073 2.271336e-04 0.9896 342 4.5073 2.249478e-04 0.9904 343 4.5073 2.225149e-04 0.9892 344 4.5073 2.183825e-04 0.9814 345 4.5073 2.115974e-04 0.9689 346 4.5073 2.037608e-04 0.9630 347 4.5073 1.961096e-04 0.9624 348 4.5073 1.915623e-04 0.9768 349 4.5073 1.844891e-04 0.9631 350 4.5073 1.789415e-04 0.9699 351 4.5073 1.747135e-04 0.9764 352 4.5073 1.685326e-04 0.9646 353 4.5073 1.625619e-04 0.9646 354 4.5073 1.573789e-04 0.9681 355 4.5073 1.543450e-04 0.9807 356 4.5073 1.527411e-04 0.9896 357 4.5073 1.514816e-04 0.9918 358 4.5073 1.498667e-04 0.9893 359 4.5073 1.482822e-04 0.9894 360 4.5073 1.467254e-04 0.9895 361 4.5073 1.453226e-04 0.9904 362 4.5073 1.440473e-04 0.9912 363 4.5073 1.425693e-04 0.9897 364 4.5073 1.399659e-04 0.9817 365 4.5073 1.354891e-04 0.9680 366 4.5073 1.304395e-04 0.9627 367 4.5073 1.255801e-04 0.9627 368 4.5073 1.224384e-04 0.9750 369 4.5073 1.178715e-04 0.9627 370 4.5073 1.141439e-04 0.9684 371 4.5073 1.114363e-04 0.9763 372 4.5073 1.074004e-04 0.9638 373 4.5073 1.037559e-04 0.9661 374 4.5073 1.005130e-04 0.9687 375 4.5073 9.864467e-05 0.9814 376 4.5073 9.765982e-05 0.9900 377 4.5073 9.693787e-05 0.9926 378 4.5073 9.603671e-05 0.9907 379 4.5073 9.536281e-05 0.9930 380 4.5073 9.470601e-05 0.9931 381 4.5073 9.386835e-05 0.9912 382 4.5073 9.312684e-05 0.9921 383 4.5073 9.220860e-05 0.9901 384 4.5073 9.051789e-05 0.9817 385 4.5073 8.757986e-05 0.9675 386 4.5073 8.441697e-05 0.9639 387 4.5073 8.117176e-05 0.9616 388 4.5073 7.917326e-05 0.9754 389 4.5073 7.650414e-05 0.9663 390 4.5073 7.415814e-05 0.9693 391 4.5073 7.243978e-05 0.9768 392 4.5073 6.992122e-05 0.9652 393 4.5073 6.755464e-05 0.9662 394 4.5073 6.550298e-05 0.9696 395 4.5073 6.442420e-05 0.9835 396 4.5073 6.386114e-05 0.9913 397 4.5073 6.343917e-05 0.9934 398 4.5073 6.291857e-05 0.9918 399 4.5073 6.255101e-05 0.9942 400 4.5073 6.218764e-05 0.9942 401 4.5073 6.169533e-05 0.9921 402 4.5073 6.126233e-05 0.9930 403 4.5073 6.074052e-05 0.9915 404 4.5073 5.976100e-05 0.9839 405 4.5073 5.783132e-05 0.9677 406 4.5073 5.582332e-05 0.9653 407 4.5073 5.381541e-05 0.9640 408 4.5073 5.256528e-05 0.9768 409 4.5073 5.088652e-05 0.9681 410 4.5073 4.942387e-05 0.9713 411 4.5073 4.836018e-05 0.9785 412 4.5073 4.681841e-05 0.9681 413 4.5073 4.530922e-05 0.9678 414 4.5073 4.403902e-05 0.9720 415 4.5073 4.342640e-05 0.9861 416 4.5073 4.311669e-05 0.9929 417 4.5073 4.288333e-05 0.9946 418 4.5073 4.258149e-05 0.9930 419 4.5073 4.234146e-05 0.9944 420 4.5073 4.210422e-05 0.9944 421 4.5073 4.182142e-05 0.9933 422 4.5073 4.157963e-05 0.9942 423 4.5073 4.129297e-05 0.9931 424 4.5073 4.075008e-05 0.9869 425 4.5073 3.955404e-05 0.9706 426 4.5073 3.826284e-05 0.9674 427 4.5073 3.703658e-05 0.9680 428 4.5073 3.627819e-05 0.9795 429 4.5073 3.518842e-05 0.9700 430 4.5073 3.427321e-05 0.9740 431 4.5073 3.360933e-05 0.9806 432 4.5073 3.266358e-05 0.9719 433 4.5073 3.170647e-05 0.9707 434 4.5073 3.092739e-05 0.9754 435 4.5073 3.057961e-05 0.9888 436 4.5073 3.040358e-05 0.9942 437 4.5073 3.027417e-05 0.9957 438 4.5073 3.010033e-05 0.9943 439 4.5073 2.991853e-05 0.9940 440 4.5073 2.973998e-05 0.9940 441 4.5073 2.958330e-05 0.9947 442 4.5073 2.944665e-05 0.9954 443 4.5073 2.928091e-05 0.9944 444 4.5073 2.897939e-05 0.9897 445 4.5073 2.826589e-05 0.9754 446 4.5073 2.743131e-05 0.9705 447 4.5073 2.667133e-05 0.9723 448 4.5073 2.615545e-05 0.9807 449 4.5073 2.540920e-05 0.9715 450 4.5073 2.485503e-05 0.9782 451 4.5073 2.436528e-05 0.9803 452 4.5073 2.373976e-05 0.9743 453 4.5073 2.310008e-05 0.9731 454 4.5073 2.261866e-05 0.9792 455 4.5073 2.240689e-05 0.9906 456 4.5073 2.229318e-05 0.9949 457 4.5073 2.221321e-05 0.9964 458 4.5073 2.210959e-05 0.9953 459 4.5073 2.200217e-05 0.9951 460 4.5073 2.189792e-05 0.9953 461 4.5073 2.180671e-05 0.9958 462 4.5073 2.171675e-05 0.9959 463 4.5073 2.161412e-05 0.9953 464 4.5073 2.143114e-05 0.9915 465 4.5073 2.097284e-05 0.9786 466 4.5073 2.038027e-05 0.9717 467 4.5073 1.986920e-05 0.9749 468 4.5073 1.948462e-05 0.9806 469 4.5073 1.898509e-05 0.9744 470 4.5073 1.857711e-05 0.9785 471 4.5073 1.821628e-05 0.9806 472 4.5073 1.782268e-05 0.9784 473 4.5073 1.736033e-05 0.9741 474 4.5073 1.698073e-05 0.9781 475 4.5073 1.684377e-05 0.9919 476 4.5073 1.676710e-05 0.9954 477 4.5073 1.670770e-05 0.9965 478 4.5073 1.663904e-05 0.9959 479 4.5073 1.657661e-05 0.9962 480 4.5073 1.651758e-05 0.9964 481 4.5073 1.646211e-05 0.9966 482 4.5073 1.639271e-05 0.9958 483 4.5073 1.633027e-05 0.9962 484 4.5073 1.622154e-05 0.9933 485 4.5073 1.587925e-05 0.9789 486 4.5073 1.541874e-05 0.9710 487 4.5073 1.509443e-05 0.9790 488 4.5073 1.477211e-05 0.9786 489 4.5073 1.440650e-05 0.9753 490 4.5073 1.411756e-05 0.9799 491 4.5073 1.379558e-05 0.9772 492 4.5073 1.354723e-05 0.9820 493 4.5073 1.314832e-05 0.9706 ---------------------------------------------------------------------- Total Iterations: 494 Avg Convergence Rate: 0.9724 Final Residual: 1.314832e-05 Total Reduction in Residual: 9.711542e-07 Maximum Memory Usage: 4.507 GB ---------------------------------------------------------------------- Total Time: 16.4863 setup: 0.550713 s solve: 15.9356 s solve(per iteration): 0.0322583 s

Adding strength threshold = 0.6 i manage to get GMRES_AMD_D2 to get to 7s but still longer than CG without preconditioner. { "config_version": 2, "determinism_flag": 1, "exception_handling" : 1, "solver": { "print_grid_stats": 1, "store_res_history": 0, "solver": "GMRES", "print_solve_stats": 1, "obtain_timings": 1, "preconditioner": { "interpolator": "D2", "print_grid_stats": 1, "solver": "AMG", "smoother": "JACOBI_L1", "presweeps": 2, "selector": "PMIS", "coarsest_sweeps": 2, "coarse_solver": "NOSOLVER", "max_iters": 1, "interp_max_elements": 4, "min_coarse_rows": 2, "scope": "amg_solver", "max_levels": 24, "cycle": "V", "postsweeps": 2, "max_row_sum": 0.9, "strength_threshold": 0.6 }, "max_iters": 1000, "monitor_residual": 1, "gmres_n_restart": 10, "convergence": "RELATIVE_INI_CORE", "tolerance": 1e-06, "norm": "L2" } } Here is the out put: AMGX version 2.5.0 Built on Apr 25 2024, 13:20:11 Compiled with CUDA Runtime 12.4, using CUDA driver 12.4 AMG Grid: Number of Levels: 8 LVL ROWS NNZ PARTS SPRSTY Mem (GB) ---------------------------------------------------------------------- 0(D) 1507876 113305856 1 4.98e-05 1.31 1(D) 427806 45533312 1 0.000249 1.04 2(D) 137879 22690729 1 0.00119 0.513 3(D) 38257 6494695 1 0.00444 0.147 4(D) 9256 1114344 1 0.013 0.0253 5(D) 1820 124304 1 0.0375 0.00285 6(D) 351 12935 1 0.105 0.000303 7(D) 47 451 1 0.204 1.15e-05 ---------------------------------------------------------------------- Grid Complexity: 1.40813 Operator Complexity: 1.67049 Total Memory Usage: 3.03017 GB ---------------------------------------------------------------------- iter Mem Usage (GB) residual rate ---------------------------------------------------------------------- Ini 4.52881 1.353886e+01 0 4.52881 8.768310e+00 0.6476 1 4.5288 5.875893e+00 0.6701 2 4.5288 4.114999e+00 0.7003 3 4.5288 2.921429e+00 0.7099 4 4.5288 2.161990e+00 0.7400 5 4.5288 1.646122e+00 0.7614 6 4.5288 1.277718e+00 0.7762 7 4.5288 1.001020e+00 0.7834 8 4.5288 7.980492e-01 0.7972 9 4.5288 6.559208e-01 0.8219 10 4.5288 5.960254e-01 0.9087 11 4.5288 5.332775e-01 0.8947 12 4.5288 4.738347e-01 0.8885 13 4.5288 4.126436e-01 0.8709 14 4.5288 3.498271e-01 0.8478 15 4.5288 2.893011e-01 0.8270 16 4.5288 2.489482e-01 0.8605 17 4.5288 2.146483e-01 0.8622 18 4.5288 1.837306e-01 0.8560 19 4.5288 1.614325e-01 0.8786 20 4.5288 1.520154e-01 0.9417 21 4.5288 1.407548e-01 0.9259 22 4.5288 1.286014e-01 0.9137 23 4.5288 1.176749e-01 0.9150 24 4.5288 1.072537e-01 0.9114 25 4.5288 9.944967e-02 0.9272 26 4.5288 9.291163e-02 0.9343 27 4.5288 8.549769e-02 0.9202 28 4.5288 7.896385e-02 0.9236 29 4.5288 7.237665e-02 0.9166 30 4.5288 6.818068e-02 0.9420 31 4.5288 6.386154e-02 0.9367 32 4.5288 5.982216e-02 0.9367 33 4.5288 5.638698e-02 0.9426 34 4.5288 5.283324e-02 0.9370 35 4.5288 4.850473e-02 0.9181 36 4.5288 4.445382e-02 0.9165 37 4.5288 4.021743e-02 0.9047 38 4.5288 3.686018e-02 0.9165 39 4.5288 3.463181e-02 0.9395 40 4.5288 3.264420e-02 0.9426 41 4.5288 3.038673e-02 0.9308 42 4.5288 2.781750e-02 0.9154 43 4.5288 2.567715e-02 0.9231 44 4.5288 2.368404e-02 0.9224 45 4.5288 2.225356e-02 0.9396 46 4.5288 2.094205e-02 0.9411 47 4.5288 1.957047e-02 0.9345 48 4.5288 1.835569e-02 0.9379 49 4.5288 1.715982e-02 0.9349 50 4.5288 1.621576e-02 0.9450 51 4.5288 1.527343e-02 0.9419 52 4.5288 1.438522e-02 0.9418 53 4.5288 1.351747e-02 0.9397 54 4.5288 1.264552e-02 0.9355 55 4.5288 1.153628e-02 0.9123 56 4.5288 1.046056e-02 0.9068 57 4.5288 9.460004e-03 0.9043 58 4.5288 8.685592e-03 0.9181 59 4.5288 8.173298e-03 0.9410 60 4.5288 7.713984e-03 0.9438 61 4.5288 7.156656e-03 0.9278 62 4.5288 6.530942e-03 0.9126 63 4.5288 5.991247e-03 0.9174 64 4.5288 5.501422e-03 0.9182 65 4.5288 5.160612e-03 0.9381 66 4.5288 4.843384e-03 0.9385 67 4.5288 4.559390e-03 0.9414 68 4.5288 4.290216e-03 0.9410 69 4.5288 4.035412e-03 0.9406 70 4.5288 3.819968e-03 0.9466 71 4.5288 3.611904e-03 0.9455 72 4.5288 3.419467e-03 0.9467 73 4.5288 3.208093e-03 0.9382 74 4.5288 2.993778e-03 0.9332 75 4.5288 2.726845e-03 0.9108 76 4.5288 2.457223e-03 0.9011 77 4.5288 2.225714e-03 0.9058 78 4.5288 2.033611e-03 0.9137 79 4.5288 1.914184e-03 0.9413 80 4.5288 1.799697e-03 0.9402 81 4.5288 1.670334e-03 0.9281 82 4.5288 1.522858e-03 0.9117 83 4.5288 1.388566e-03 0.9118 84 4.5288 1.275700e-03 0.9187 85 4.5288 1.194108e-03 0.9360 86 4.5288 1.119639e-03 0.9376 87 4.5288 1.061499e-03 0.9481 88 4.5288 1.000335e-03 0.9424 89 4.5288 9.452505e-04 0.9449 90 4.5288 8.985699e-04 0.9506 91 4.5288 8.504361e-04 0.9464 92 4.5288 8.098885e-04 0.9523 93 4.5288 7.613204e-04 0.9400 94 4.5288 7.080914e-04 0.9301 95 4.5288 6.480337e-04 0.9152 96 4.5288 5.812936e-04 0.8970 97 4.5288 5.259771e-04 0.9048 98 4.5288 4.851439e-04 0.9224 99 4.5288 4.556539e-04 0.9392 100 4.5288 4.293960e-04 0.9424 101 4.5288 4.009284e-04 0.9337 102 4.5288 3.650740e-04 0.9106 103 4.5288 3.336130e-04 0.9138 104 4.5288 3.087990e-04 0.9256 105 4.5288 2.887910e-04 0.9352 106 4.5288 2.724659e-04 0.9435 107 4.5288 2.612934e-04 0.9590 108 4.5288 2.473665e-04 0.9467 109 4.5288 2.350677e-04 0.9503 110 4.5288 2.251407e-04 0.9578 111 4.5288 2.138501e-04 0.9499 112 4.5288 2.055829e-04 0.9613 113 4.5288 1.951170e-04 0.9491 114 4.5288 1.826712e-04 0.9362 115 4.5288 1.693517e-04 0.9271 116 4.5288 1.546645e-04 0.9133 117 4.5288 1.414307e-04 0.9144 118 4.5288 1.325003e-04 0.9369 119 4.5288 1.265273e-04 0.9549 120 4.5288 1.208508e-04 0.9551 121 4.5288 1.144998e-04 0.9474 122 4.5288 1.060693e-04 0.9264 123 4.5288 9.925553e-05 0.9358 124 4.5288 9.370971e-05 0.9441 125 4.5288 8.940171e-05 0.9540 126 4.5288 8.594241e-05 0.9613 127 4.5288 8.353282e-05 0.9720 128 4.5288 8.024825e-05 0.9607 129 4.5288 7.731457e-05 0.9634 130 4.5288 7.480387e-05 0.9675 131 4.5288 7.216509e-05 0.9647 132 4.5288 7.037042e-05 0.9751 133 4.5288 6.794323e-05 0.9655 134 4.5288 6.514613e-05 0.9588 135 4.5288 6.205491e-05 0.9525 136 4.5288 5.863003e-05 0.9448 137 4.5288 5.502975e-05 0.9386 138 4.5288 5.254025e-05 0.9548 139 4.5288 5.086614e-05 0.9681 140 4.5288 4.932885e-05 0.9698 141 4.5288 4.746658e-05 0.9622 142 4.5288 4.485311e-05 0.9449 143 4.5288 4.297201e-05 0.9581 144 4.5288 4.136565e-05 0.9626 145 4.5288 4.015194e-05 0.9707 146 4.5288 3.900880e-05 0.9715 147 4.5288 3.820714e-05 0.9794 148 4.5288 3.706657e-05 0.9701 149 4.5288 3.596322e-05 0.9702 150 4.5288 3.497656e-05 0.9726 151 4.5288 3.398798e-05 0.9717 152 4.5288 3.333172e-05 0.9807 153 4.5288 3.242272e-05 0.9727 154 4.5288 3.145855e-05 0.9703 155 4.5288 3.036427e-05 0.9652 156 4.5288 2.914779e-05 0.9599 157 4.5288 2.764016e-05 0.9483 158 4.5288 2.659135e-05 0.9621 159 4.5288 2.590356e-05 0.9741 160 4.5288 2.516156e-05 0.9714 161 4.5288 2.434308e-05 0.9675 162 4.5288 2.309417e-05 0.9487 163 4.5288 2.224793e-05 0.9634 164 4.5288 2.147848e-05 0.9654 165 4.5288 2.090239e-05 0.9732 166 4.5288 2.035908e-05 0.9740 167 4.5288 1.995579e-05 0.9802 168 4.5288 1.940278e-05 0.9723 169 4.5288 1.886533e-05 0.9723 170 4.5288 1.835445e-05 0.9729 171 4.5288 1.786542e-05 0.9734 172 4.5288 1.753188e-05 0.9813 173 4.5288 1.706333e-05 0.9733 174 4.5288 1.659876e-05 0.9728 175 4.5288 1.604605e-05 0.9667 176 4.5288 1.542272e-05 0.9612 177 4.5288 1.468205e-05 0.9520 178 4.5288 1.418284e-05 0.9660 179 4.5288 1.381454e-05 0.9740 180 4.5288 1.343885e-05 0.9728 ---------------------------------------------------------------------- Total Iterations: 181 Avg Convergence Rate: 0.9265 Final Residual: 1.343885e-05 Total Reduction in Residual: 9.926137e-07 Maximum Memory Usage: 4.529 GB ---------------------------------------------------------------------- Total Time: 7.59087 setup: 0.583601 s solve: 7.00727 s solve(per iteration): 0.0387142 s

Thats why I tried with ILU smoother to see if it can improve the performance (Using FSAI as complex smoother in HYPRE helped in my case) and when I found that all the configuration involving ILU did not work for my matrix, so I wonder if there may be additional info needed for Multicolor ILU. Here is the output I got when using this configuration (based on GMRES_AMG_D2.json but just change smoother to ILU) { "config_version": 2, "determinism_flag": 1, "exception_handling" : 1, "solver": { "print_grid_stats": 1, "store_res_history": 1, "solver": "GMRES", "print_solve_stats": 1, "obtain_timings": 1, "preconditioner": { "interpolator": "D2", "print_grid_stats": 1, "solver": "AMG", "smoother": "MULTICOLOR_DILU", "presweeps": 2, "selector": "PMIS", "coarsest_sweeps": 2, "coarse_solver": "NOSOLVER", "max_iters": 1, "interp_max_elements": 4, "min_coarse_rows": 2, "scope": "amg_solver", "max_levels": 24, "cycle": "V", "postsweeps": 2 }, "max_iters": 100, "monitor_residual": 1, "gmres_n_restart": 10, "convergence": "RELATIVE_INI_CORE", "tolerance": 1e-06, "norm": "L2" } } Here is the output: AMGX version 2.5.0 Built on Apr 25 2024, 13:20:11 Compiled with CUDA Runtime 12.4, using CUDA driver 12.4 AMG Grid: Number of Levels: 9 LVL ROWS NNZ PARTS SPRSTY Mem (GB) ---------------------------------------------------------------------- 0(D) 1507876 113305856 1 4.98e-05 1.3 1(D) 281692 31103364 1 0.000392 0.706 2(D) 39859 4883517 1 0.00307 0.111 3(D) 4793 328295 1 0.0143 0.00753 4(D) 762 35364 1 0.0609 0.000822 5(D) 198 7160 1 0.183 0.000168 6(D) 60 1622 1 0.451 3.88e-05 7(D) 18 254 1 0.784 6.45e-06 8(D) 6 36 1 1 9.98e-07 ---------------------------------------------------------------------- Grid Complexity: 1.21712 Operator Complexity: 1.3209 Total Memory Usage: 2.12977 GB ---------------------------------------------------------------------- iter Mem Usage (GB) residual rate ---------------------------------------------------------------------- Ini 4.52295 1.353886e+01 0 4.52295 1.353814e+01 0.9999 1 4.5229 1.353091e+01 0.9995 2 4.5229 1.353089e+01 1.0000 3 4.5229 1.353076e+01 1.0000 4 4.5229 1.353076e+01 1.0000 5 4.5229 1.351923e+01 0.9991 6 4.5229 1.351887e+01 1.0000 7 4.5229 1.350736e+01 0.9991 8 4.5229 1.350569e+01 0.9999 9 4.5229 1.350562e+01 1.0000 10 4.5229 1.350562e+01 1.0000 11 4.5229 1.350562e+01 1.0000 12 4.5229 1.350562e+01 1.0000 13 4.5229 1.350562e+01 1.0000 14 4.5229 1.350562e+01 1.0000 15 4.5229 1.350562e+01 1.0000 16 4.5229 1.350562e+01 1.0000 17 4.5229 1.350562e+01 1.0000 18 4.5229 1.350562e+01 1.0000 19 4.5229 1.350555e+01 1.0000 20 4.5229 1.350555e+01 1.0000 21 4.5229 1.350555e+01 1.0000 22 4.5229 1.350555e+01 1.0000 23 4.5229 1.350555e+01 1.0000 24 4.5229 1.350555e+01 1.0000 25 4.5229 1.350555e+01 1.0000 26 4.5229 1.350555e+01 1.0000 27 4.5229 1.350555e+01 1.0000 28 4.5229 1.350555e+01 1.0000 29 4.5229 1.350552e+01 1.0000 30 4.5229 1.350552e+01 1.0000 31 4.5229 1.350552e+01 1.0000 32 4.5229 1.350552e+01 1.0000 33 4.5229 1.350552e+01 1.0000 34 4.5229 1.350552e+01 1.0000 35 4.5229 1.350552e+01 1.0000 36 4.5229 1.350552e+01 1.0000 37 4.5229 1.350552e+01 1.0000 38 4.5229 1.350552e+01 1.0000 39 4.5229 1.350551e+01 1.0000 40 4.5229 1.350551e+01 1.0000 41 4.5229 1.350551e+01 1.0000 42 4.5229 1.350551e+01 1.0000 43 4.5229 1.350551e+01 1.0000 44 4.5229 1.350551e+01 1.0000 45 4.5229 1.350551e+01 1.0000 46 4.5229 1.350551e+01 1.0000 47 4.5229 1.350551e+01 1.0000 48 4.5229 1.350551e+01 1.0000 49 4.5229 1.350551e+01 1.0000 50 4.5229 1.350551e+01 1.0000 51 4.5229 1.350551e+01 1.0000 52 4.5229 1.350551e+01 1.0000 53 4.5229 1.350551e+01 1.0000 54 4.5229 1.350551e+01 1.0000 55 4.5229 1.350551e+01 1.0000 56 4.5229 1.350551e+01 1.0000 57 4.5229 1.350551e+01 1.0000 58 4.5229 1.350551e+01 1.0000 59 4.5229 1.350551e+01 1.0000 60 4.5229 1.350551e+01 1.0000 61 4.5229 1.350551e+01 1.0000 62 4.5229 1.350551e+01 1.0000 63 4.5229 1.350551e+01 1.0000 64 4.5229 1.350551e+01 1.0000 65 4.5229 1.350551e+01 1.0000 66 4.5229 1.350551e+01 1.0000 67 4.5229 1.350551e+01 1.0000 68 4.5229 1.350551e+01 1.0000 69 4.5229 1.350551e+01 1.0000 70 4.5229 1.350551e+01 1.0000 71 4.5229 1.350551e+01 1.0000 72 4.5229 1.350551e+01 1.0000 73 4.5229 1.350551e+01 1.0000 74 4.5229 1.350551e+01 1.0000 75 4.5229 1.350551e+01 1.0000 76 4.5229 1.350551e+01 1.0000 77 4.5229 1.350551e+01 1.0000 78 4.5229 1.350551e+01 1.0000 79 4.5229 1.350551e+01 1.0000 80 4.5229 1.350551e+01 1.0000 81 4.5229 1.350551e+01 1.0000 82 4.5229 1.350551e+01 1.0000 83 4.5229 1.350551e+01 1.0000 84 4.5229 1.350551e+01 1.0000 85 4.5229 1.350551e+01 1.0000 86 4.5229 1.350551e+01 1.0000 87 4.5229 1.350551e+01 1.0000 88 4.5229 1.350551e+01 1.0000 89 4.5229 1.350551e+01 1.0000 90 4.5229 1.350551e+01 1.0000 91 4.5229 1.350551e+01 1.0000 92 4.5229 1.350551e+01 1.0000 93 4.5229 1.350551e+01 1.0000 94 4.5229 1.350551e+01 1.0000 95 4.5229 1.350551e+01 1.0000 96 4.5229 1.350551e+01 1.0000 97 4.5229 1.350551e+01 1.0000 98 4.5229 1.350551e+01 1.0000 99 4.5229 1.350551e+01 1.0000 ---------------------------------------------------------------------- Total Iterations: 100 Avg Convergence Rate: 1.0000 Final Residual: 1.350551e+01 Total Reduction in Residual: 9.975368e-01 Maximum Memory Usage: 4.523 GB ---------------------------------------------------------------------- Total Time: 40.3522 setup: 1.15667 s solve: 39.1955 s

When I tried to used PCG_ILU.json, it return nan.

AMGX version 2.5.0 Built on Apr 25 2024, 13:20:11 Compiled with CUDA Runtime 12.4, using CUDA driver 12.4 iter Mem Usage (GB) residual rate ---------------------------------------------------------------------- Ini 0 1.353886e+01 0 0 nan nan 1 0.0000 -nan(ind) -nan(ind) 2 0.0000 nan nan 3 0.0000 -nan(ind) -nan(ind) 4 0.0000 nan nan 5 0.0000 -nan(ind) -nan(ind) 6 0.0000 nan nan 7 0.0000 -nan(ind) -nan(ind) 8 0.0000 nan nan 9 0.0000 -nan(ind) -nan(ind) 10 0.0000 nan nan 11 0.0000 -nan(ind) -nan(ind) 12 0.0000 nan nan 13 0.0000 -nan(ind) -nan(ind) 14 0.0000 nan nan 15 0.0000 -nan(ind) -nan(ind) 16 0.0000 nan nan 17 0.0000 -nan(ind) -nan(ind) 18 0.0000 nan nan 19 0.0000 -nan(ind) -nan(ind) ---------------------------------------------------------------------- Total Iterations: 20 Avg Convergence Rate: -nan(ind) Final Residual: -nan(ind) Total Reduction in Residual: -nan(ind) Maximum Memory Usage: 0.000 GB ---------------------------------------------------------------------- Total Time: 92.0629 setup: 0.282855 s solve: 91.78 s

Basically I am still struggling to get the right configuration for the matrix. The best results I got is from CG without any preconditioner. I hope this should clarify my issues.

Jul 10 '24 19:07 kiendangtor

AMGX AMGX copied to clipboard

Seeking advice for setting up solver for finite element

AMGX
AMGX copied to clipboard