rsmtool

rsmtool copied to clipboard

In some situations we might need to predict scores from coefficients stored in a data frame. This will be useful when we need to generate predictions for linear models where...

aloukina

When tests are run in STRICT mode, we check for any errors or warnings in the report. We currently have two functions: ``test_utils.check_report`` https://github.com/EducationalTestingService/rsmtool/blob/main/rsmtool/test_utils.py#L767 checks for both errors and warnings...

aloukina

We currently allow for different sign columns in the feature_subset_file to accommodate a legacy situation where we could have inputs with divergent feature signs. This generates a lot of complexity....

aloukina

We use scaled predictions in tutorial. Prediction scaling is becoming increasingly obsolete: it was used to maximize kappas that were computed on rounded values but in fact it increases MSE....

aloukina

Deprecate _modify_eval_columns_to_ensure_version_compatibilty in RSMCompare

2

The `Comparer` class in `comparer.py` currently contains a class `_modify_eval_columns_to_ensure_version_compatibilty` to allow running RSMCompare on outputs from RSMTool < 7.0. I think it's time to deprecate this.

aloukina

We need to explicitly add support for SKLL VotingLearners since right now RSMTool can only use `Learner` instances.

desilinguist

Pre commit

8

Introducing `pre-commit` into the workflow. I have tried to segregate my commits into those that add/change config files and those that apply changes to source files that had `flake8` issues,...

mulhod

Tests for notebooks

1

Even though our initial intent was not to have much code in the notebooks beyond reading in .csv and displaying tables, the code for plots etc. is becoming increasingly complex...

aloukina

Rewrite docstrings to match scikit-learn Style

1

We should try and match the way that scikit-learn writes their docstrings. [Here](https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/linear_model/_huber.py#L125) is an example. For example, there is a blank line between the parameters and the default values...

desilinguist

Currently SDM/DSM is computed in `utils.metrics` but the parameters for computation are passed through two more functions in `analyzer` leading to confusing and repetitive documentation and making it difficult to...

aloukina