DataSets.jl icon indicating copy to clipboard operation
DataSets.jl copied to clipboard

Large dataset examples

Open c42f opened this issue 2 years ago • 0 comments

It'd be nice to have a practical list of examples of scientific data workflows as inspiration for DataSets.jl.

Here's some examples I came across on discourse:

  • Reading and chunk-based parallel processing of NetCDF files: https://discourse.julialang.org/t/help-me-beat-my-pythonist-friends-code-speeding-up-data-reading-with-simple-reduction-from-netcdf-file/76457/31
  • Downloading protein data and untarring it on the fly: https://discourse.julialang.org/t/how-to-plumb-together-download-uncompress-untar-without-writing-full-downloaded-file/76668/6

c42f avatar Feb 16 '22 00:02 c42f