rsmtool

rsmtool copied to clipboard

Don't update user's jupyter history when notebooks are executed

2

When notebooks are executed, all of the cells are added to the user's history each time they are executed. This clogs up the history and makes it hard to use...

mulhod

Now that [type hints](https://docs.python.org/3/library/typing.html) (and consequent static type checking using `mypy`) is becoming more common, we should add them to the RSMTool codebase and pipelines.

desilinguist

Consistent approach to warnings

5

In some parts of the code we use `logger.warning`while in other parts we used `warnings`. Is there any reason not use `warnings` everywhere?

aloukina

Investigate whether we can use any visualizers from Yellowbrick

1

The [yellowbrick](https://www.scikit-yb.org/en/latest/index.html) project has some pretty nice visualizers. May be some of them are useful for the report?

desilinguist

In the rsmeval output, under association statistics, there is “wtkappa” output. I just realized this is confusing for some users who are scanning for “QWK”, the standard abbreviation. Also there...

bwriordan

QWK generates runtime warning

1

There are edge cases where QWK function generates runtime warning: ```/opt/python/rsmtool/lib/python3.6/site-packages/rsmtool/utils/metrics.py:471: RuntimeWarning: invalid value encountered in double_scalars kappa = numerator / denominator /opt/python/rsmtool/lib/python3.6/site-packages/rsmtool/utils/metrics.py:471: RuntimeWarning: invalid value encountered in double_scalars kappa...

aloukina

Add suppress_warnings argument to various metrics

1

The functions for computing individual metrics raise warning in edge cases (for example, when standard deviation is zero). In some cases this can result in multiple warnings which all have...

aloukina

Currently config files/dictionary can only contain a path to training/test file. It would be nice to allow the user to pass a pandas Dataframe if they are using API with...

aloukina

Try solution for kernel RuntimeErrors

2

Based on [this comment](https://github.com/jupyter/jupyter_client/issues/154#issuecomment-580320664), we might want to try this to see if it fixes issues for the mysterious kernel errors we get from time to time.

desilinguist

Right now, we rely on a combination of an environment variable (`TESTDIR`) and the variable `rsmtool_test_dir` but this doesn't always work. We need to figure out a way how to...

desilinguist