uxarray
uxarray copied to clipboard
UXARRAY
Investigate the run-time of `ux.open_grid`
When attempting to construct a Grid using ux.open_grid with a larger dataset (6gb, 56,623,106 faces), it takes about 6 minutes to execute. This time was taken on a single 256gb node on Casper. This specific dataset is ne1024np4_scrip_c20190125.nc which can be found here.
This may be in part due to a lack of optimization in _build_nNodes_per_face() or other functions that are always called.
I noticed this also, possible to have a flag and only perform this operation if needed/specified?
Like xarray, we should load the whole thing only when user performs an operation or queries the ugrid object for specific stuff.
I noticed this also, possible to have a flag and only perform this operation if needed/specified?
Like xarray, we should load the whole thing only when user performs an operation or queries the ugrid object for specific stuff.
We already do this for most, where construction is done at the time a user calls it, not at the object construction. https://github.com/UXARRAY/uxarray/blob/a4263f470874a06230a2c8f1fa165ee6a834330c/uxarray/core/grid.py#L496-L507
However, nNodes_per_face is one that we always construct and wasn't updated to act like the one above, I'll change it.
Will that 6 minute cost show up in some other operation that has to actually read the data?
Will that 6 minute cost show up in some other operation that has to actually read the data?
The 6 minutes is the total system time needed to execute ux.open_grid, meaning that nothing can be done while it's constructing.
For this dataset, and all SCRIP grids, the costly operation is when we find all the unique [lon, lat] pairs and construct the face nodes in the following code block, specifically the np.unique call:
https://github.com/UXARRAY/uxarray/blob/fc2675a93ff54fe9dcaad3c5cc6f205687b44a97/uxarray/io/_scrip.py#L38-L49
From my profiling, that single np.unique call takes up 80% of the execution time to open a grid.
That does seem like the only bit of code in the that does anything significant. I think in these cases we can recommend the user rewrite the mesh in UGRID.
On Sep 25, 2023, at 8:41 PM, Philip Chmielowiec @.***> wrote:
For this dataset, and all SCRIP grids, the costly operation is when we find all the unique [lon, lat] pairs and construct the face nodes in the following code block, specifically the np.unique call:
https://github.com/UXARRAY/uxarray/blob/fc2675a93ff54fe9dcaad3c5cc6f205687b44a97/uxarray/io/_scrip.py#L38-L49
From my profiling, that single np.unique call takes up 80% of the execution time to open a grid.
— Reply to this email directly, view it on GitHub https://github.com/UXARRAY/uxarray/issues/375#issuecomment-1734777134, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIVPZEVAORYVLU5P7KSAOLX4JFHPANCNFSM6AAAAAA2H6KN2Q. You are receiving this because you are subscribed to this thread.
That does seem like the only bit of code in the that does anything significant. I think in these cases we can recommend the user rewrite the mesh in UGRID. … On Sep 25, 2023, at 8:41 PM, Philip Chmielowiec @.***> wrote: For this dataset, and all SCRIP grids, the costly operation is when we find all the unique [lon, lat] pairs and construct the face nodes in the following code block, specifically the np.unique call: https://github.com/UXARRAY/uxarray/blob/fc2675a93ff54fe9dcaad3c5cc6f205687b44a97/uxarray/io/_scrip.py#L38-L49 From my profiling, that single np.unique call takes up 80% of the execution time to open a grid. — Reply to this email directly, view it on GitHub <#375 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIVPZEVAORYVLU5P7KSAOLX4JFHPANCNFSM6AAAAAA2H6KN2Q. You are receiving this because you are subscribed to this thread.
Yep! Either that, or raising a warning message stating that Mesh2_face_nodes is being constructed, which may take some time for large datasets.
@philipc2 maybe setting up an asv benchmark and if there are issues, looking into this through our internship project would be great!
Would decorating this method with numba play well?
It might, especially if the overall time to convert to machine code is shorter than the execution time.
Here is our benchmark that we have for the construction of n_nodes_per_face https://uxarray.github.io/uxarray-asv/#mpas_ocean.ConnectivityConstruction.time_n_nodes_per_face
Just ran a quick check for a 500,000 face grid. I'll run some for larger grids too.
With Numba: 892ms Without Numba: 17ms