uxarray icon indicating copy to clipboard operation
uxarray copied to clipboard

Poor performancewhen constructing `edge_node`, `face_edge` and `face_face` connectivity

Open philipc2 opened this issue 7 months ago • 1 comments

I've observed very poor performance when constructing the Grid.edge_node_connectivity and Grid.face_edge_connectivity.

The timings below were taken on a single NCAR Derecho CPU node.

  • AMD EPYC™ 7763 Milan processors
  • Dual-socket nodes, 64 cores per socket
  • 256 GB DDR4 memory per node
Resolution Nodes Faces Edges Grid Load Time (s) Connectivity Construction Time (s) Total Time (s)
30km 1,310,720 655,362 1,966,080 2.023 9.37 11.393
15km 5,242,880 2,621,442 7,864,320 7.673 39.987 47.66
7.5km 20,971,520 10,485,762 31,457,280 28.716 99.309 128.025
3.75km 83,886,080 41,943,042 125,829,120 113.943 406.8 520.743

The timing for the 15km grid seems inconsistent with the others, since there's an expected scaling of about 4x. The others follow this trend.

Currently, we have the following implementation.

https://github.com/UXARRAY/uxarray/blob/a6aa6294975f189e8c36dd3d14da745526e53e06/uxarray/grid/connectivity.py#L181-L238

philipc2 avatar Apr 07 '25 04:04 philipc2