Circuitscape
Improve current map performance
This is what I get on the 6m problem. Even though much of the time is spent in solvers, current map creation is currently taking as much time as the solve. It would be good to get to the bottom of where it spends its time. Note that I am using an SSD.
Virals-MacBook-Pro 12:43:49 {master} ~/CIRCUITSCAPE/BigTests/6m$ time python ../../circuitscape/src/csrun.py 6m-1solve.ini
11/13/2013 12.43.53.PM:INFO: Reading maps
11/13/2013 12.43.56.PM:INFO: Processing maps
3 sec: load_maps
11/13/2013 12.43.56.PM:DEBUG:Resistance/conductance map has 5250648 nodes
0 sec: _construct_node_map
12 sec: _construct_g_graph
14 sec: _construct_component_map
11/13/2013 12.44.11.PM:DEBUG: Graph has 5250648 nodes, 2 focal points and 7 components.
0 sec: _construct_node_map
12 sec: _construct_g_graph
1 sec: laplacian
11/13/2013 12.44.26.PM:INFO: solving focal pair 1 of 1.
multilevel_solver
Number of Levels: 4
Operator Complexity: 1.145
Grid Complexity: 1.118
Coarse Solver: 'splu'
level unknowns nonzeros
0 5208900 46633606 [87.35%]
1 564938 6028046 [11.29%]
2 48399 677925 [ 1.27%]
3 3285 46111 [ 0.09%]
25 sec: create_amg_hierarchy
21 sec: solve_linear_system
21 sec: single_ground_solver
0 sec: _create_voltage_map
18 sec: _create_current_maps
86 sec: single_ground_all_pair_resistances
104 sec: pairwise_module
11/13/2013 12.45.40.PM:DEBUG: Job took 1 minutes 47 seconds to complete.
107 sec: compute_raster
107 sec: compute
(array([[ 0. , 1. , 2. ],
[ 1. , 0. , 104.05655076],
[ 2. , 104.05655076, 0. ]]), False)
real 1m47.936s
user 1m30.988s
sys 0m17.085s
@bmcrae We need to discuss this on a call and see if there is a way to get higher performance here.
Sure. Right now we go through a lot of steps to produce current maps, there certainly might be ways to reorganize and make them more efficient.
Brad McRae, Ph.D. The Nature Conservancy North America Region Tel: 541-223-1170 email: [email protected] http://www.nature.org/ourscience/brad-mcrae.xml
On Sun, Nov 24, 2013 at 3:11 AM, Viral B. Shah [email protected]:
@bmcrae https://github.com/bmcrae We need to discuss this on a call and see if there is a way to get higher performance here.
— Reply to this email directly or view it on GitHubhttps://github.com/Circuitscape/Circuitscape/issues/21#issuecomment-29153195 .
Alternatively, could you explain here how the current maps are computed? It seemed to me that it may be doing some redundant work, and it may be worthwhile to write it from first principles.
It is certainly taking more steps than needed, and this resulted from my efforts to minimize memory usage. I'll dig in to the code to refresh myself on the steps and summarize. I'd put this lower on the priority list than network enhancements and user guide. Sound OK?
On Sun, Nov 24, 2013 at 10:02 AM, Viral B. Shah [email protected]:
Alternatively, could you explain here how the current maps are computed? It seemed to me that it may be doing some redundant work, and it may be worthwhile to write it from first principles.
— Reply to this email directly or view it on GitHubhttps://github.com/Circuitscape/Circuitscape/issues/21#issuecomment-29161374 .
I think we may be able to do better on current map performance if we do not compute the current maps after each solve, and instead do them in a batch.
Computing current maps in a batch could help, but it would require writing voltage maps to disk or storing them in RAM. We can discuss.
@ViralBShah, to answer your question, here are the steps that are needed for current mapping:
- compute voltages at each node
- Compute branch currents (currents flowing through each resistor). For a resistor connecting node a and b, that is computed using I = Rab/Vab, where Vab is the difference in voltages between nodes a and b, and Rab is the graph resistance between them.
The node currents are then computed as the sum of currents into or out of a node. That includes branch currents and any current flowing to ground or from a current source.
Right now the current mapping goes through several steps, which were broken down and experimented with to minimize memory use. I'm sure it could be done more efficiently.
Does that clarify?
Do you think we can undo the steps taken for minimizing memory use, just for experimentation purposes? Given that our memory utilization is lower now, it may be worth simplifying the code, and it should get faster as well. I'd like to revisit this as the last thing as part of the CS 4.0 release.
Yes, but this something that would have to be tried by @tanmaykm. Since this code has been restructured, it would take me quite awhile to understand how it works and try things.
Ok - let's leave it as is for now then, and just get through the 4.0 release. Otherwise, we may end up introducing a bug.