vowpal_wabbit icon indicating copy to clipboard operation
vowpal_wabbit copied to clipboard

VW python wrapper: is it possible to get (raw) scores for multiclass?

Open oleg131 opened this issue 2 years ago • 1 comments

Version: 8.10.1 Implementation: Python wrapper (vowpalwabbit.pyvw)

Description: For the binary case I have been using get_partial_prediction method to get raw score of the classifier:

vw_text = '|U some text'
example = classifier.parse(vw_text)
classifier.predict(example)
score = example.get_partial_prediction()

However, I'm not sure how to replicate this for the multiclass case. Assuming that the model was trained using --oaa 3 (for data with 3 classes), how can I get the scores for each class, or at least for the winning class?

E.g. when I pass some data as above, I get

example.get_multiclass_prediction()
> 1 
# correct label
example.get_partial_prediction()
> 0.0
# probably meaningless

If this is supported in a later version but not in v8.10.1, I can consider upgrading too.

oleg131 avatar Apr 28 '22 20:04 oleg131

Generally speaking the concept of "raw" scores is a command line thing and is implemented for a small subset of reductions. I do not believe this is supported.

jackgerrits avatar Jul 01 '22 15:07 jackgerrits