Decorator to mark experimental features

[oruebel]

It would be useful to have a consistent way in HDMF to mark certain features (classes, functions etc.) as experimental to let users know when a feature is still under development.

Proposed solution Create a decorator to allow us to mark classes and functions as experimental. This could be a stand-alone decorator and/or this could be a feature as part of docval. The decorator should be configurable and support as options:

• Issue warning when an experimental feature is used.
• Add a standard note to the docstring of the class/function indicating that the feature is experimental
• In addition to the developer configuring the warnings, the user should be able to set an option in HDMF to disable experimental feature warnings.
• Maybe we could even have an special import option where a user could disable/enable the use of experimental features, e.g., to make sure that experimental features are not accessible in production environments.

Is your feature request related to a problem? Please describe. A common problem is that we want to make experimental features available for testing but it needs to be clear to other users what the state of those features is. Experimental features also allow for breaking changes during development without requiring major version bumps on the API.

Checklist

• [ ] Have you ensured the feature or change was not already reported ?
• [ ] Have you included a brief and descriptive title?
• [ ] Have you included a clear description of the problem you are trying to solve?
• [ ] Have you included a minimal code snippet that reproduces the issue you are encountering?
• [ ] Have you checked our Contributing document?

[rly]

Initially I thought a decorator and warning would be a good idea, but I worry about over-engineering and over-complicating this. Having all experimental features / types within a separate experimental package and/or just marking certain features / types as experimental in the documentation might be sufficient.

Scikit-learn has an experimental module: https://github.com/scikit-learn/scikit-learn/tree/main/sklearn/experimental

""" The :mod:sklearn.experimental module provides importable modules that enable the use of experimental features or estimators. The features and estimators that are experimental aren't subject to deprecation cycles. Use them at your own risks! """

Numpy has experimental features too. This one is enabled with an environment var: https://numpy.org/devdocs/release/1.16.0-notes.html#numpy-functions-now-support-overrides-with-array-function

pandas has experimental features that they seem to document only by marking it as experimental in the docs: https://pandas.pydata.org/pandas-docs/version/1.0.0/whatsnew/v1.0.0.html#experimental-new-features https://pandas.pydata.org/docs/development/extending.html#extension-types

TensorFlow has an experimental module: https://www.tensorflow.org/api_docs/python/tf/experimental

[rly]

And the Pandas version policy (we should probably make one) explicitly describes how they handle experimental features: https://pandas.pydata.org/pandas-docs/version/1.0.0/development/policies.html#policies-version

[oruebel]

version policy

Agreed, we should make this an item in the versioning policy and seems like the first thing we should do now.

Having all experimental features / types within a separate experimental package and/or just marking certain features / types as experimental in the documentation might be sufficient

My current assessment of this is as follows:

1. Having an experimental feature in a distinct location (e.g., package) is ideal as it makes it explicit (already on import) that the feature is experimental. Also, once the feature should move to production it changes location (which can be both good and bad)
2. Having a decorator is helpful for convenience and because it has the potential to allow us to be more flexible in how we handle experimental features. However, I agree that the decorator is an optional, nice-to-have feature but not a requirement
3. Mentioning in the documentation that a feature is experimental is the minimum we should do