triage
triage copied to clipboard
dssg
General Purpose Risk Modeling and Prediction Toolkit for Policy and Social Good Problems
@saleiro suggested this. His words were "@tweddielin has an amazing set of plots for crosstabs" ... and I trust him
Or maybe the model group _and_ the model: (Image from dirtyduck)  The rational for this request, is that the modeler will use this plot in the "model group selection/exploration"...
The methods used by ModelGroupEvaluator in the notebook are old, e.g. `plot_jaccard` is now `plot_jaccard_preds`
This table would map models with feature groups and loo/loi strategies so we would be able to quickly know that a given model is a loo of feature group x...
In `dirtyduck` the last prediction `as-of-date` is `2017-12-01`. The following table is a fragment of the final models from the inspections part of `dirtyduck`: ```| model_group_id | model_type | hyperparameters...
http://scikit-learn.org/stable/auto_examples/calibration/plot_calibration_curve.html
We should consider restructuring the [postmodeling tutorial notebook](https://github.com/dssg/triage/blob/master/src/triage/component/postmodeling/contrast/postmodeling_tutorial.ipynb) to provide a more natural user flow, for instance: - Moving the model group evaluation and performance metrics up front to potentially...
The [postmodeling tutorial](https://github.com/dssg/triage/blob/master/src/triage/component/postmodeling/contrast/postmodeling_tutorial.ipynb) uses `params.thresholds_iterator` in performing error analysis, however this parameter is not present in the [example postmodeling config file](https://github.com/dssg/triage/blob/master/example/config/postmodeling_config.yaml)
Postmodeling currently identifies feature groups simply by splitting on the first underscore, but some feature groups may use underscores in their name and start with the same characters, so we...
Check how much of your feature space has been imputed by triage. This would give a little bit more of context to the already available plot showing the sparsity of...
