uxarray icon indicating copy to clipboard operation
uxarray copied to clipboard

DRAFT: Dask implementation of Node-Face Topological Aggregation

Open philipc2 opened this issue 1 year ago • 2 comments
trafficstars

Closes #XXX

Overview

Expected Usage

import uxarray as ux

grid_path = "/path/to/grid.nc"
data_path = "/path/to/data.nc"

uxds = ux.open_dataset(grid_path, data_path)

# this is how you use this function
some_output = uxds.some_function()

# this is another way to use this function
other_output = uxds.some_function(some_param = True)

PR Checklist

General

  • [ ] An issue is linked created and linked
  • [ ] Add appropriate labels
  • [ ] Filled out Overview and Expected Usage (if applicable) sections

Testing

  • [ ] Adequate tests are created if there is new functionality
  • [ ] Tests cover all possible logical paths in your function
  • [ ] Tests are not too basic (such as simply calling a function and nothing else)

Documentation

  • [ ] Docstrings have been added to all new functions
  • [ ] Docstrings have updated with any function changes
  • [ ] Internal functions have a preceding underscore (_) and have been added to docs/internal_api/index.rst
  • [ ] User functions have been added to docs/user_api/index.rst

Examples

  • [ ] Any new notebook examples added to docs/examples/ folder
  • [ ] Clear the output of all cells before committing
  • [ ] New notebook files added to docs/examples.rst toctree
  • [ ] New notebook files added to new entry in docs/gallery.yml with appropriate thumbnail photo in docs/_static/thumbnails/

philipc2 avatar Jul 18 '24 01:07 philipc2

@rytam2

We discussed this a little durring our meeting earlier today. Lets chat about this sometime this week and do some analysis of the Dask implementation.

philipc2 avatar Jul 18 '24 01:07 philipc2

This is a good contribution, we can add a test case, also pass chunk during loading, something along the lines:

ds = xr.open_dataset('large_dataset.nc', chunks={'time': 100, 'latitude': 50, 'longitude': 50})

Not very sure, but I think our **kwargs might be able to handle chunks in our open_dataset wrapper call.

rajeeja avatar Jul 23 '24 19:07 rajeeja