Prediction score column is empty

[mjuvilla]

Describe the bug Similar to what this https://github.com/HumanSignal/label-studio/issues/2263 issue spoke about. Once I get the predictions using the model API I can check the results in label studio but the "Prediction Score" column is empty. In the aforementioned issue they speak about an option to show the prediction scores but as far as I can tell this option does not exist anymore.

Expected behavior I expected to be able to see the score of the prediction so I can sort and filter the tasks accordingly.

Screenshots

Just in case I checked the source of the tasks and they indeed contain the prediction and the score:

Environment (please complete the following information):

• OS: Ubuntu 22.04.4 LTS
• Label Studio Version: 1.12.0.post0

Additional context: I'm using this as the base for my ML backend but didn't do any significant changes aside from pointing to my own fine-tuned models. https://github.com/HumanSignal/label-studio-ml-backend/tree/master/label_studio_ml/examples/bert_classifier Also, I've never been able to train my model using the label-studio api since I always get a server error (not sure if this is significant to the main issue).

[sajarin]

Hey @mjuvilla I know it's been a while, thanks for making this issue. Are you still facing prediction score columns being empty, and on the latest version?

[code4days]

@sajarin I'm on the latest version and I'm experiencing the same issue, when I create predictions with scores via the api, the prediction score column is not populated, this is preventing me from using the filters.

Edit: For me it seems like it has to do with the model_version; I'm setting the model_version in the prediction but in the code here

...
else:
        model_version = first_task.project.model_version
        if model_version is None:
            return queryset.annotate(predictions_score=Avg('predictions__score'))
        else:
            return queryset.annotate(
                predictions_score=Avg('predictions__score', filter=Q(predictions__model_version=model_version))
            )

its reading the model_version from project which is an empty string in my case and not None, which invokes the else statement.

[djwessel]

I found that the only way to get the prediction scores to show up in the frontend is if you have predictions for ALL the datapoints. However, that may not even be necessary. Looking at the code for annotate_predictions_score, it looks like if the first task doesn't have associated predictions, the score will not be shown for any of the tasks.

[mjuvilla]

Hey @mjuvilla I know it's been a while, thanks for making this issue. Are you still facing prediction score columns being empty, and on the latest version?

Yes, turns out that in my case since I was prelabeling my dataset using the "Retrieve Predictions" option, then I had to go to Settings > Annotation > Prelabeling and select the Predictions I just made instead of the model. Then the Prediction Score column gets populated.