profanity-check icon indicating copy to clipboard operation
profanity-check copied to clipboard

Added train_model.py and made the necessary modifications

Open menkotoglou opened this issue 4 years ago • 4 comments

  1. Reverse engineered training script
  2. Updated versions of scikit-learn and included it in setup.py and requirements.txt
  3. Retrained model with training data and updated version as in (2)
  4. Dropped support for Python3.5 (see Travis configuration), because it was no longer compatible with current scikit-learn version
  5. Fixed test as one test case is now below threshold

We tested these changes on a private dataset with the following results:

Before:

Predicted Actual | Not Profane(0) | Profane(1) Not Profane(0) | 703 | 14 Profane(1) | 93 | 39

Accuracy Score: 87.4%

After:

Predicted Actual | Not Profane(0) | Profane(1) Not Profane(0) | 697 | 20 Profane(1) | 87 | 45

Accuracy Score: 87.4%

Used in production as committed here: https://github.com/dimitrismistriotis/profanity-check

menkotoglou avatar Jun 30 '20 19:06 menkotoglou

@koti How do I use this build?

ieshaan12 avatar Sep 01 '20 06:09 ieshaan12

@koti How do I use this build?

By referencing the other repository. For pip + "requirements.txt", use the following instead of "profanity-check":

-e git+https://github.com/dimitrismistriotis/profanity-check.git#egg=profanity-check

Also check this issue here if @vzhou842 accepts it, you can bring back profanity-check.

dimitrismistriotis avatar Sep 01 '20 07:09 dimitrismistriotis

@dimitrismistriotis Thanks! I was wondering if we could implement a function which censors content like the profanity package?

ieshaan12 avatar Sep 01 '20 11:09 ieshaan12

@dimitrismistriotis Thanks! I was wondering if we could implement a function which censors content like the profanity package?

Censor is a very broad concept, also didn't get the "like the profanity package" part: profanity detects

dimitrismistriotis avatar Sep 15 '20 10:09 dimitrismistriotis