uxarray
uxarray copied to clipboard
Poor performancewhen constructing `edge_node`, `face_edge` and `face_face` connectivity
I've observed very poor performance when constructing the Grid.edge_node_connectivity and Grid.face_edge_connectivity.
The timings below were taken on a single NCAR Derecho CPU node.
- AMD EPYC™ 7763 Milan processors
- Dual-socket nodes, 64 cores per socket
- 256 GB DDR4 memory per node
| Resolution | Nodes | Faces | Edges | Grid Load Time (s) | Connectivity Construction Time (s) | Total Time (s) |
|---|---|---|---|---|---|---|
| 30km | 1,310,720 | 655,362 | 1,966,080 | 2.023 | 9.37 | 11.393 |
| 15km | 5,242,880 | 2,621,442 | 7,864,320 | 7.673 | 39.987 | 47.66 |
| 7.5km | 20,971,520 | 10,485,762 | 31,457,280 | 28.716 | 99.309 | 128.025 |
| 3.75km | 83,886,080 | 41,943,042 | 125,829,120 | 113.943 | 406.8 | 520.743 |
The timing for the 15km grid seems inconsistent with the others, since there's an expected scaling of about 4x. The others follow this trend.
Currently, we have the following implementation.
https://github.com/UXARRAY/uxarray/blob/a6aa6294975f189e8c36dd3d14da745526e53e06/uxarray/grid/connectivity.py#L181-L238