uxarray icon indicating copy to clipboard operation
uxarray copied to clipboard

DRAFT: Weighted Average

Open philipc2 opened this issue 1 year ago • 4 comments
trafficstars

Closes #826

Overview

Expected Usage

import uxarray as ux

grid_path = "/path/to/grid.nc"
data_path = "/path/to/data.nc"

uxds = ux.open_dataset(grid_path, data_path)

# this is how you use this function
some_output = uxds.some_function()

# this is another way to use this function
other_output = uxds.some_function(some_param = True)

PR Checklist

General

  • [ ] An issue is linked created and linked
  • [ ] Add appropriate labels
  • [ ] Filled out Overview and Expected Usage (if applicable) sections

Testing

  • [ ] Adequate tests are created if there is new functionality
  • [ ] Tests cover all possible logical paths in your function
  • [ ] Tests are not too basic (such as simply calling a function and nothing else)

Documentation

  • [ ] Docstrings have been added to all new functions
  • [ ] Docstrings have updated with any function changes
  • [ ] Internal functions have a preceding underscore (_) and have been added to docs/internal_api/index.rst
  • [ ] User functions have been added to docs/user_api/index.rst

Examples

  • [ ] Any new notebook examples added to docs/examples/ folder
  • [ ] Clear the output of all cells before committing
  • [ ] New notebook files added to docs/examples.rst toctree
  • [ ] New notebook files added to new entry in docs/gallery.yml with appropriate thumbnail photo in docs/_static/thumbnails/

philipc2 avatar Jul 02 '24 16:07 philipc2

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@rytam2

I've set up the boilerplate for the weighted mean functionality. This should be a good place to get started. We can run over this during today's meeting.

philipc2 avatar Jul 02 '24 16:07 philipc2

@rytam2

We have fixed the issue with the quad-hexagon grid. I've added it back to the test case.

philipc2 avatar Jul 05 '24 18:07 philipc2

ASV Benchmarking

Benchmark Comparison Results

Benchmarks that have improved:

Change Before [b11d0115] After [a0be1c16] Ratio Benchmark (Parameter)
- 445M 375M 0.84 face_bounds.FaceBounds.peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/quad-hexagon/grid.nc'))
- 467M 373M 0.80 mpas_ocean.Integrate.peakmem_integrate('480km')
failed 412±7μs n/a mpas_ocean.WeightedMean.time_weighted_mean_face_centered('120km')
failed 341±6μs n/a mpas_ocean.WeightedMean.time_weighted_mean_face_centered('480km')

Benchmarks that have stayed the same:

Change Before [b11d0115] After [a0be1c16] Ratio Benchmark (Parameter)
375M 376M 1.00 face_bounds.FaceBounds.peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/mpas/QU/oQU480.231010.nc'))
375M 375M 1.00 face_bounds.FaceBounds.peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/scrip/outCSne8/outCSne8.nc'))
400M 379M 0.95 face_bounds.FaceBounds.peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/geoflow-small/grid.nc'))
1.59±0.02s 1.58±0.01s 1.00 face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/mpas/QU/oQU480.231010.nc'))
224±0.9ms 223±4ms 0.99 face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/scrip/outCSne8/outCSne8.nc'))
2.04±0.02s 2.01±0.02s 0.99 face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/geoflow-small/grid.nc'))
7.90±0.3ms 8.07±0.2ms 1.02 face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/quad-hexagon/grid.nc'))
3.02±0.03s 3.08±0.03s 1.02 import.Imports.timeraw_import_uxarray
674±20ms 669±7ms 0.99 mpas_ocean.ConnectivityConstruction.time_face_face_connectivity('120km')
41.9±0.6ms 42.1±0.5ms 1.01 mpas_ocean.ConnectivityConstruction.time_face_face_connectivity('480km')
1.83±0.03ms 1.82±0.03ms 0.99 mpas_ocean.ConnectivityConstruction.time_n_nodes_per_face('120km')
538±10μs 557±20μs 1.03 mpas_ocean.ConnectivityConstruction.time_n_nodes_per_face('480km')
1.12±0μs 1.06±0μs 0.95 mpas_ocean.ConstructTreeStructures.time_ball_tree('120km')
280±1ns 270±2ns 0.96 mpas_ocean.ConstructTreeStructures.time_ball_tree('480km')
770±4ns 759±6ns 0.99 mpas_ocean.ConstructTreeStructures.time_kd_tree('120km')
270±1ns 257±2ns 0.95 mpas_ocean.ConstructTreeStructures.time_kd_tree('480km')
432M 432M 1.00 mpas_ocean.GeoDataFrame.peakmem_to_geodataframe('120km', False)
407M 407M 1.00 mpas_ocean.GeoDataFrame.peakmem_to_geodataframe('120km', True)
379M 379M 1.00 mpas_ocean.GeoDataFrame.peakmem_to_geodataframe('480km', False)
393M 377M 0.96 mpas_ocean.GeoDataFrame.peakmem_to_geodataframe('480km', True)
1.02±0.01s 1.03±0.01s 1.01 mpas_ocean.GeoDataFrame.time_to_geodataframe('120km', False)
53.2±0.4ms 52.7±0.4ms 0.99 mpas_ocean.GeoDataFrame.time_to_geodataframe('120km', True)
78.0±0.3ms 79.3±1ms 1.02 mpas_ocean.GeoDataFrame.time_to_geodataframe('480km', False)
5.50±0.2ms 5.51±0.08ms 1.00 mpas_ocean.GeoDataFrame.time_to_geodataframe('480km', True)
319M 321M 1.01 mpas_ocean.Gradient.peakmem_gradient('120km')
296M 296M 1.00 mpas_ocean.Gradient.peakmem_gradient('480km')
2.79±0.02ms 2.79±0.06ms 1.00 mpas_ocean.Gradient.time_gradient('120km')
308±5μs 320±6μs 1.04 mpas_ocean.Gradient.time_gradient('480km')
389M 389M 1.00 mpas_ocean.Integrate.peakmem_integrate('120km')
182±5ms 177±1ms 0.97 mpas_ocean.Integrate.time_integrate('120km')
12.0±0.04ms 12.1±0.05ms 1.00 mpas_ocean.Integrate.time_integrate('480km')
342±7ms 347±4ms 1.01 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('120km', 'exclude')
348±4ms 348±2ms 1.00 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('120km', 'include')
343±3ms 344±4ms 1.00 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('120km', 'split')
22.8±0.6ms 22.7±0.1ms 0.99 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('480km', 'exclude')
22.8±0.4ms 23.0±0.2ms 1.01 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('480km', 'include')
22.6±0.3ms 23.0±0.2ms 1.01 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('480km', 'split')
56.0±0.1ms 56.5±0.5ms 1.01 mpas_ocean.RemapDownsample.time_inverse_distance_weighted_remapping
45.7±0.2ms 45.9±0.2ms 1.01 mpas_ocean.RemapDownsample.time_nearest_neighbor_remapping
360±0.8ms 361±1ms 1.00 mpas_ocean.RemapUpsample.time_inverse_distance_weighted_remapping
266±2ms 264±0.2ms 0.99 mpas_ocean.RemapUpsample.time_nearest_neighbor_remapping
294M 294M 1.00 quad_hexagon.QuadHexagon.peakmem_open_dataset
291M 291M 1.00 quad_hexagon.QuadHexagon.peakmem_open_grid
5.58±0.2ms 6.24±0.5ms ~1.12 quad_hexagon.QuadHexagon.time_open_grid

Benchmarks that have got worse:

Change Before [b11d0115] After [a0be1c16] Ratio Benchmark (Parameter)
+ 6.78±0.4ms 7.93±1ms 1.17 quad_hexagon.QuadHexagon.time_open_dataset

github-actions[bot] avatar Jul 17 '24 16:07 github-actions[bot]

UXDataset support

rytam2 avatar Nov 22 '24 21:11 rytam2