[EAGLE-4114] - return all the annotations and predictions of an evaluation

[phatvo9]

What

Return all the annotations and predictions of an evaluation. Only support for classifier models

How

code snippet

import os
from sklearn.metrics import accuracy_score
from sklearn.metrics import classification_report
import numpy as np
from clarifai.client.model import Model
from clarifai.client.dataset import Dataset

os.environ["CLARIFAI_PAT"] = "???"

model = Model(url="url/of/model/includes/version-id")
dataset = Dataset(dataset_id="dataset-id")
y, y_pred, clss, input_protos = model.get_raw_eval(dataset, return_format="array")
y = np.argmax(y, axis=1)
y_pred = np.argmax(y_pred, axis=1)
report = classification_report(y, y_pred, target_names=clss)
print(report)
acc = accuracy_score(y, y_pred)
print("acc ", acc)

example output

              precision    recall  f1-score   support

         cat       1.00      1.00      1.00        56
         dog       0.98      1.00      0.99        57
 non-cat-dog       1.00      0.99      0.99        82

    accuracy                           0.99       195
   macro avg       0.99      1.00      1.00       195
weighted avg       0.99      0.99      0.99       195

acc 0.9948717948717949

[ackizilkale]

Thanks a lot @phtvo! What is the issue with detector models? We need this feature more for detection.