Error while redistributing A on the 2D process mesh

[aithalab]

Hi,

I have been using superlu_dist to solve a complex-valued block-tridiagonal linear system that arises from discretizing a 2D variable coefficient Poisson equation using 2nd order central difference scheme for a few months. The coefficient matrix, A, is symmetric and positive definite. The linear system is complex valued because a 3D Poisson equation has been reduced to 2D by performing Fourier transform. I have also installed METIS and ParMETIS. I have tested other direct solvers for sparse matrices such as MUMPS and PaStiX (which are developed for symmteric matrices) and found that superlu_dist outperformed them both in terms of computational speed and memory footprint.

The library performs without any issues when the dimension of A is O(1e6). This is the case when the 3D grid consists of (256,128,64) grid points. However, when I increase the grid size to (512,256,128) grid points, I get the following error message: Calloc fails for SPA dense[]. at line 761 in file /libraries/SuperLU_DIST_5.1.3/SRC/pzdistribute.c

Do you have any suggestions to fix this? Also, since the coefficient matrix, A, is symmetric and positive definite I expect U to be the transpose of L and thus there is no need to store them both. Is there a way to incorporate this in the solver?

Thanks

[aithalab]

Tagging contributors @xiaoyeli @gchavez2 @jrobcary

[xiaoyeli]

How many nodes / processes are you using?

You can try one of the two things:

1. Use parallel symbolic factorization option, see FAQ http://crd-legacy.lbl.gov/~xiaoye/SuperLU/faq.html#superlu_dist:parsymbfact
2. Reduce the panel size, i.e., the #3 parameter in SRC/sp_ienv.c(), which can be reset by env. variable "NSUP". (The default is 128)

Sherry Li

On Fri, Jan 26, 2018 at 9:48 AM, aithalab [email protected] wrote:

Hi,

I have been using superlu_dist to solve a complex-valued block-tridiagonal linear system that arises from discretizing a 2D variable coefficient Poisson equation using 2nd order central difference scheme for a few months. The coefficient matrix, A, is symmetric and positive definite. The linear system is complex valued because a 3D Poisson equation has been reduced to 2D by performing Fourier transform. I have also installed METIS and ParMETIS. I have tested other direct solvers for sparse matrices such as MUMPS and PaStiX (which are developed for symmteric matrices) and found that superlu_dist outperformed them both in terms of computational speed and memory footprint.

The library performs without any issues when the dimension of A is O(1e6). This is the case when the 3D grid consists of (256,128,64) grid points. However, when I increase the grid size to (512,256,128) grid points, I get the following error message: Calloc fails for SPA dense[]. at line 761 in file /libraries/SuperLU_DIST_5.1.3/SRC/pzdistribute.c

Do you have any suggestions to fix this? Also, since the coefficient matrix, A, is symmetric and positive definite I expect U to be the transpose of L and thus there is no need to store them both. Is there a way to incorporate this in the solver?

Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/xiaoyeli/superlu_dist/issues/26, or mute the thread https://github.com/notifications/unsubscribe-auth/ALMq9wxUsOQ5I9MbbUiNJ-fI4i9UkYLLks5tOg_ngaJpZM4RupZm .