fingerprint-securedrop icon indicating copy to clipboard operation
fingerprint-securedrop copied to clipboard

Generate plot of precision/recall as a function of k

Open redshiftzero opened this issue 7 years ago • 2 comments

We want to know in a realistic scenario - i.e. one that incorporates the effect of the class imbalance - how effective these attacks are in terms of true and false positives. A really nice plot that would show this (right now the machine learning pipeline generates only an ROC curve) is a graph of precision and recall as a function of k, the percent of the ranked list flagged. Let's add this to evaluate.py.

Also: see Figure 5 in this paper to see a nice comparison between ROC curves and precision/recall graphs in the presence of different base rates.

redshiftzero avatar Oct 12 '16 00:10 redshiftzero