apbs
apbs copied to clipboard
Published
20 hours ago •
Electrostatics
Electrostatics
Very slow boundary element execution
Hi @lwwilson1 --
Could you please help @intendo and me with a performance issue in BEM? The details follow.
From the output below you can see that the bem-pKa test (15) that was switched from MSMS to NanoShaper skin mode is 75 minutes:
Test project /Users/d3c572/git/apbs/build
Start 1: born_test
1/19 Test #1: born_test ........................ Passed 12.83 sec
Start 2: actin-dimer-auto_test
2/19 Test #2: actin-dimer-auto_test ............ Passed 91.52 sec
Start 3: actin-dimer-parallel_test
3/19 Test #3: actin-dimer-parallel_test ........ Passed 290.72 sec
Start 4: alkanes_test
4/19 Test #4: alkanes_test ..................... Passed 7.79 sec
Start 5: FKBP_test
5/19 Test #5: FKBP_test ........................ Passed 24.73 sec
Start 6: hca-bind_test
6/19 Test #6: hca-bind_test .................... Passed 11.11 sec
Start 7: ionize_test
7/19 Test #7: ionize_test ...................... Passed 4.32 sec
Start 8: ion-pmf_test
8/19 Test #8: ion-pmf_test ..................... Passed 4.73 sec
Start 9: ion-protein_test
9/19 Test #9: ion-protein_test ................. Passed 30.82 sec
Start 10: pka-lig_test
10/19 Test #10: pka-lig_test ..................... Passed 30.68 sec
Start 11: point-pmf_test
11/19 Test #11: point-pmf_test ................... Passed 2.00 sec
Start 12: solv_test
12/19 Test #12: solv_test ........................ Passed 1.49 sec
Start 13: geoflow_test
13/19 Test #13: geoflow_test ..................... Passed 4.58 sec
Start 14: bem_test
14/19 Test #14: bem_test ......................... Passed 29.56 sec
Start 15: bem-pKa_test
15/19 Test #15: bem-pKa_test ..................... Passed 4472.47 sec
Start 16: bem-binding-energy_test
16/19 Test #16: bem-binding-energy_test .......... Passed 519.04 sec
Start 17: pbam_test
17/19 Test #17: pbam_test ........................ Passed 0.12 sec
Start 18: pbsam_test
18/19 Test #18: pbsam_test ....................... Passed 5.59 sec
Start 19: protein-rna_test
19/19 Test #19: protein-rna_test ................. Passed 170.10 sec
100% tests passed, 0 tests failed out of 19
Total Test time (real) = 5714.23 sec
There seems like there are some large calculations in one of the bem-pKa test, 2LZT-ASH66.in. Is this due to the large number of atoms (1961), vertices (931532), or triangles (1862940)? It really slows down when it reaches: Creating tree for 1862940 particles with max 500 per node
(apbs) (base) WE36241:bem-pKa d3c572$ ../../build/bin/apbs 2LZT-ASH66.in
[snip header]
This executable compiled on Aug 28 2020 at 16:52:59
Parsing input file 2LZT-ASH66.in...
rank 0 size 1...
Parsed input file.
Got paths for 1 molecules
Reading PQR-format atom data from test_proteins/2LZT-ASH66.pqr.
1961 atoms
Centered at (1.044e+00, 1.408e+01, 2.125e+01)
Net charge 9.00e+00 e
Preparing to run 1 PBE calculations.
----------------------------------------
CALCULATION #1 (comp_solv): BOUNDARY ELEMENT
Setting up problem...
Molecule ID: 1
Linearized traditional PBE
Zero boundary conditions
2 ion species (0.000 M ionic strength):
0.950 A-radius, 1.000 e-charge, 0.000 M concentration
1.810 A-radius, -1.000 e-charge, 0.000 M concentration
Solute dielectric: 20.000
Solvent dielectric: 78.540
Using "molecular" surface definition; no smoothing
Solvent probe radius: 1.400 A
Temperature: 298.000 K
Treecode order: 3
Max # of nodes: 500
MAC (theta): 0.800000
Mesh flag: 2
Setting up the TABI input...
Running NanoShaper...
<<INFO>> Starting NanoShaper 0.7.5
<<INFO>> Loading atoms....
<<INFO>> Read 1961 atoms
<<INFO>> Geometric baricenter -> 0.9775 14.163 21.2475
<<INFO>> Grid is 537
<<INFO>> MAX 27.7775 40.963 48.0475
<<INFO>> MIN -25.8225 -12.637 -5.5525
<<INFO>> Perfil 90 %
<<INFO>> Rmaxdim 48.335
<<INFO>> Allocating memory...ok!
<<INFO>> Initialization completed
<<INFO>> Building skin surface..
<<INFO>> Regular triangulation....ok!
<<INFO>> Computing 3-Delaunay patches...ok!
<<INFO>> Computing 0-Delaunay patches...ok!
<<INFO>> Computing 1-Delaunay patches...ok!
<<INFO>> Computing 2-Delaunay patches...ok!
<<INFO>> Surface build-up time.. 0 [s]
<<INFO>> Shrinking value 0.45
<<INFO>> Number of mixed cells -> 32660
<<INFO>> Number of del_point/vor_cell -> 1345
<<INFO>> Number of del_edge/vor_facet -> 9759
<<INFO>> Number of del_facet/vor_edge -> 17200
<<INFO>> Number of del_cell/vor_point -> 8601
<<INFO>> Use load balancing: 1
<<INFO>> Detected 12 logical cores
<<INFO>> Setting 12 threads
<<INFO>> Inside id value is 5
<<INFO>> Ray-tracing panel 0...ok!
<<INFO>> Ray-tracing panel 1...ok!
<<INFO>> Ray-tracing panel 2...ok!
<<INFO>> Ray-tracing computation time.. 1 [s]
<<INFO>> Approximated 0 rays (0.00000 %)
<<INFO>> Cavity detection time is 0 [s]
<<INFO>> Assembling octrees..ok!
<<INFO>> Surface computation time.. 2 [s]
<<INFO>> Estimated volume 16143.8 [A^3]
<<INFO>> Triangulating Surface...
<<INFO>> Setting 12 threads
<<INFO>> Generating MC vertices...ok!
<<INFO>> MC added 0 non analytical vertices
<<INFO>> Triangles done
<<INFO>> MC time is 1 [s]
<<INFO>> Total, grid conformant, surface area is 6209.03 [A^2]
<<INFO>> Number of vertices 931532 number of triangles 1862940
<<INFO>> Saving in MSMS format, no patch info, no nearest atom..
<<INFO>> Smoothing surface...
<<INFO>> Saving in MSMS format, no patch info, no nearest atom..ok!
<<INFO>> Total Triangulation time 5 [s]
<<INFO>> Cleaning memory...ok!
<<INFO>> If you use NanoShaper please cite these works:
<<CITATION>> S. Decherchi, W. Rocchia, "A general and Robust Ray-Casting-Based Algorithm for Triangulating Surfaces at the Nanoscale"; PlosOne
<<CITATION>> link: http://www.plosone.org/article/metrics/info%3Adoi%2F10.1371%2Fjournal.pone.0059744
Finished reading .vert file...
Finished reading .face file...
0 faces have been deleted...
Total suface area = 6198.26093813872466853
Initializing treecode...
Setting up arrays for Taylor expansion...
x-limits of box: -15.472667, 17.427667
y-limits of box: -6.159000, 34.483667
z-limits of box: -2.914333, 45.409667
Creating tree for 1862940 particles with max 500 per node
iteration no.=1, error=7.924293e-02
iteration no.=2, error=1.900933e-02
iteration no.=3, error=5.577112e-03
iteration no.=4, error=1.981469e-03
iteration no.=5, error=6.735350e-04
iteration no.=6, error=2.060250e-04
iteration no.=7, error=6.499776e-05
Cleaning up the tree structure...
Memory has been cleaned for TABIPB!
Solvation energy = -363.465631 kJ/mol
Free energy = -6849.581187 kJ/mol
The max and min potential and normal derivatives on elements area:
potential 28.210307 -9.041446
norm derv 32.618396 -43.390486
The max and min potential and normal derivatives on vertices area:
potential 28.019487 -8.977629
norm derv 32.276886 -42.726383
Returning to APBS caller...
Solvation energy and Coulombic energy in kJ/mol...
Global net ELEC energy = -3.634656314849E+02
Global net COULOMBIC energy = -6.486115555583E+03
Calculating forces...
----------------------------------------
CLEANING UP AND SHUTTING DOWN...
Destroying force arrays.
No energy arrays to destroy.
Destroying multigrid structures.
Destroying finite element structures.
Destroying 1 molecules
Final memory usage: 0.001 MB total, 1.170 MB high water
Thanks for using APBS!
Here is the list of output files from the tests:
(apbs) (base) WE36241:bem-pKa d3c572$ ls -alFst
total 229896
24 -rw-r--r-- 1 d3c572 admin 8501 Aug 28 18:16 2LZT-noASH66.out
264 -rw-r--r-- 1 d3c572 admin 73078 Aug 28 18:16 io.mc
229504 -rw-r--r-- 1 d3c572 admin 116793839 Aug 28 18:16 surface_potential.dat
0 drwxr-xr-x 15 d3c572 admin 480 Aug 28 17:40 ./
24 -rw-r--r-- 1 d3c572 admin 8497 Aug 28 17:40 2LZT-ASH66.out
24 -rw-r--r-- 1 d3c572 admin 8372 Aug 28 17:02 ASH66.out
8 -rw-r--r-- 1 d3c572 admin 863 Aug 25 20:22 ASP66.in
8 -rw-r--r-- 1 d3c572 admin 857 Aug 25 20:22 ASH66.in
8 -rw-r--r-- 1 d3c572 admin 889 Aug 25 20:22 2LZT-noASP66.in
8 -rw-r--r-- 1 d3c572 admin 889 Aug 25 20:22 2LZT-noASH66.in
8 -rw-r--r-- 1 d3c572 admin 887 Aug 25 20:22 2LZT-ASP66.in
8 -rw-r--r-- 1 d3c572 admin 887 Aug 25 20:22 2LZT-ASH66.in
0 drwxr-xr-x 27 d3c572 admin 864 Aug 21 09:39 ../
0 drwxr-xr-x 8 d3c572 admin 256 Aug 21 09:39 test_proteins/
8 -rw-r--r-- 1 d3c572 admin 1085 Aug 21 09:39 README.md
My bad on this--It had slipped my mind the value of the sdens parameter should be updated on the examples. The density parameter in msms has a slightly different meaning than the one in NanoShaper (in msms, number of surface elements scales roughly linearly with sdens; in Nanoshaper, it scales quadratically). I'm fairly certain sdens 2.47 instead of sdens 10 will result in the tests taking about the same time they used to, but I'll test to make sure before I change anything and update the tests.
