rsmtool
rsmtool copied to clipboard
EducationalTestingService
A Python package to facilitate research on building and evaluating automated scoring models.
(1) RMSE between all sets of scores (2) Correlations between all sets of scores (3) Scatter plots between all pairs of scores
It might be useful to automatically generate a log file like SKLL does with all of the warnings etc. properly captured. I know it's possible to redirect the output to...
For certain purposes (e.g., when writing a book), it's much easier to show a single file (the config file) rather than both the config file and the features CSV file...
When the number of features is large, the `preprocessed_features` notebook hangs, and the report generation times out. The bottleneck seems to be the training set distribution graphs in this notebook....
It would be good to show a summary table for subgroup analyses: mean/std/min/max for different performance measures across all subgroups.
This can specify the prioritized list of channels inside as well as the name of the environment inside the file which makes the installation command much simpler than using just...
Right now, the confusion matrix and the score distribution plots rely entirely on the test set human scores. This means that if there are, say, 6s in the training data...
With those two versions, generating a report with the fairness sections included yields the following error: ```python ValueError Traceback (most recent call last) Cell In [58], line 110 107 with...
I was trying to run an `rsmxval` experiment with a handful of features and ran into the following issue (something I did not get when I ran a single `rsmtool`...
Metadata
Owner
Metadata
A Python package to facilitate research on building and evaluating automated scoring models.