rsmtool
rsmtool copied to clipboard
EducationalTestingService
A Python package to facilitate research on building and evaluating automated scoring models.
We allow for multiple subgroups but only one continuous co-variate ("length"). Sometimes one might want to test for multiple covariates (e.g. length and speech rater for spoken data). Would be...
@desilinguist suggested that the fact that we are comparing an experiment to itself takes away from the utility of the tutorial. He thinks we should actually compare different experiments.
[From JIRA] For feature description tables in RSMCompare, it would be useful to highlight non-zero differences.
[from JIRA] For historical reasons, the two tools use different terminology. If we keep maintaining RSMCompare, we should eventually reconcile the terminology.
[from JIRA] The human_score_column in RSMPredict is by default set to 'sc1' when processing the .json file and there is no way to distinguish later in the code whether the...
Currently, we are catching a number of potential warnings in our notebooks using `warnings.catch_warnings()`. However, some of these warnings may be helpful to the user. We are discarding them primarily...
Add some tests to make sure that `FileUpdater` behaves as expected.
Clean up the `intermediate_file_paths` notebooks, by (1) providing more descriptive titles for some files (e.g. `'confMatrix' -> 'Confusion Matrix'`), and (2) grouping files into table cells.
Right now, `grid_jobs=` in `train_skll_model()` but this could be really slow for large datasets.
We might consider changing `update_files.py` to separate new files and updated files and print those statistics as two different lists.
Metadata
Owner
Metadata
A Python package to facilitate research on building and evaluating automated scoring models.