glue

glue copied to clipboard

Is your feature request related to a problem? Please describe it: When using large distributed datasets, or data on slower storage devices it would be useful to be able to use precomputed statistics about the data from the metadata in the file.

This could include things like min / max / std / 99.9% etc. This would then be used by the image viewer etc rather than sampling the array to compute these things.

In addition it might be useful if these statistics could be specified per-axis or per dask chunk. This would allow axes like Stokes to have different dynamic ranges over the whole axis without having to do the computation (which could lead to a lot of dask chunks being loaded).

Describe the solution you'd like: An API for data factories to be able to specify these precomputed statistics.

Just a quick note that for more complex cases, overloading compute_statistic might be the easiest way

Am really interested in working on this issue.

Let me start with some AIA and HMI data as @Cadair has suggested first.

Found the keywords needed for the statistics of AIA and HMI data sets. Let me try to do the same for IRIS Level 2 raster and SJI data cubes to see if something similar can be found.