NiMARE icon indicating copy to clipboard operation
NiMARE copied to clipboard

Add documentation section on contributing new methods

Open tsalo opened this issue 2 years ago • 2 comments

Summary

One thing NiMARE is meant to be is a central location for meta-analytic algorithms. Folks who produce new methods (e.g., in graduate school) should be able to add those methods to NiMARE, but we of course have requirements regarding both code and method quality. We should come up with some instructions for contributing new methods and add them to the documentation.

I see this as a specific section within the contributing guidelines.

Additional details

Hopefully this will convince independent methodologists to add their methods to NiMARE.

Next steps

  1. Determine what we want the guidelines to be.
    • Maybe we could use a model like scikit-learn's extension model?
  2. Add the guidelines to the contributing documentation.

tsalo avatar Jun 04 '22 14:06 tsalo

Scikit-learn has a template for extensions, scikit-learn-contrib/project-template. The approach seems to involve creating estimators, for which there is both documentation and a check_estimator test. I didn't see any information on moving from a contrib project to scikit-learn though.

tsalo avatar Jun 10 '22 14:06 tsalo

I think we probably need a checklist of requirements for algorithms that should be merged, in addition to requirements for mergeable code.

Some possible requirements for mergeable algorithms:

  • Associated publication, preprint, or conference presentation.
  • Some statement regarding long-term support for the algorithm. I don't think we should require the original developer to support the implementation in NiMARE, but we should at least have a solid idea of whether they will or not.
  • Available test data for validating the NiMARE/Python implementation.
  • Meaningful default parameters. We don't want algorithms that folks have to tinker with until they get an interpretable result.

tsalo avatar Jun 15 '22 15:06 tsalo