NeurodataWithoutBorders
[Feature]: Recommend compression and chunking options based on data type and shape
What would you like to see added to PyNWB?
Follow-up to @satra's comment: https://github.com/dandi/helpdesk/discussions/184#discussioncomment-13093548
more generally it may be nice if we created a little function in nwb perhaps that did a quick evaluation of compression options for the user. @rly had done a comparative evaluation project in the past @bendichter is writing a dandi blog post about this, i think. it would be nice to have a utility for doing so with strategic options for different data types and modalities.
This exists in NeuroConv: https://github.com/catalystneuro/neuroconv/blob/9b505833d90b412b7fc6a1526ce7b3ec43bfca7f/src/neuroconv/tools/nwb_helpers/_configuration_models/_base_dataset_io.py#L251
It would be nice to implement this in PyNWB. We should take into account both cloud benchmarking and local benchmarking results.
Is your feature request related to a problem?
Confusing for users to know how chunking and compression affect their data in particular
What solution would you like?
Create small function that runs a quick evaluation on impact of chunking and compression on a representative subset of the data
Do you have any interest in helping implement the feature?
Yes.
Code of Conduct
- [x] I agree to follow this project's Code of Conduct
- [x] Have you checked the Contributing document?
- [x] Have you ensured this change was not already requested?
