pints
pints copied to clipboard
Create a PR template for introduction of new sampling methods into PINTS
Good description of how to do so here: https://blog.axosoft.com/enhancing-pull-request-descriptions-templates/
As part of this, I'm thinking that people need to specify hyperparameters for their method (in a file on PINTS somewhere) that are hand-picked for each of the problems we use for functional testing. I think we probably already have something like this for existing methods on the original functional testing repo...
It might also be worth having a "score sheet" for a method presented before it's merged: showing its performance on the functional tests. Whether we'd want such a score sheet in PINTS: I doubt it. But, we could have a separate repo with scores for each method.
We're thinking it should have:
- optimal hyperparameters for each functional test
- average (I think mean) score across each of the functional tests