accuracy by tag

[celsofranssa]

Hello,

Is there an approach to measure the accuracy by TAG?

Thanks.

[celsofranssa]

@aneesh-joshi Sklearn classification-report can calculate accuracy each tag. Could you show how to put it in action over model_evaluation.py? Thanks.

[aneesh-joshi]

Hey @Ceceu Sorry for responding so late. I hope it's not too late.

Yes, I believe it can be done.

From the sklearn website:

>>> from sklearn.metrics import classification_report
>>> y_true = [0, 1, 2, 2, 2]
>>> y_pred = [0, 0, 2, 2, 1]
>>> target_names = ['class 0', 'class 1', 'class 2']
>>> print(classification_report(y_true, y_pred, target_names=target_names))

So, from the repo, you just need y_pred and y_true. y_true will be the actual tags and y_pred will be the predicted tags. You can do it on a sentence by sentence basis.

If you figure something out, you can make a PR with a separate file.