evidently icon indicating copy to clipboard operation
evidently copied to clipboard

BinaryClassification with name != "default" crashes

Open emacha opened this issue 7 months ago • 1 comments

I don't know if this is a bug or not supposed to be implemented. But I'll ask the question here.

The premise is that I have a model that trains in more than one target. So I want to pass more than one BinaryClassification class to my data_definition. Given the signature is classification: List[Classification] | None, I'd assume this is intended as a feature.

If I don't define a unique name for BinaryClassification, it crashes on report.run because it's not unique.

>>>   raise ValueError("More than one classification with id {}".format(classification_id))
>>>ValueError: More than one classification with id default

However, if I do define a name, it doesn't work like my example below.

import pandas as pd
from evidently import Dataset, DataDefinition, Report, BinaryClassification
from evidently.metrics import Accuracy

data = pd.DataFrame({
    "id_col": [1, 2, 3],
    "actual_1": [10.0, 20.0, 30.0],
    "predicted_1": [11.0, 19.0, 29.0],
    "actual_2": [10.0, 20.0, 30.0],
    "predicted_2": [11.0, 19.0, 29.0],
})
classification_ = [BinaryClassification(target="actual_1", prediction_labels="predicted_1", name="regression_1"), BinaryClassification(target="actual_2", prediction_labels="predicted_2", name="regression_2")]
data_definition = DataDefinition(id_column="id_col", classification=classification_)
dataset = Dataset.from_pandas(data, data_definition=data_definition)
report = Report(metrics=[Accuracy(target="actual_1", prediction="predicted_1"),])
report.run(dataset)

>>>     raise KeyError(key) from err
>>> KeyError: None

I think I traced it to this function _default_input_data_generator, that hardcodes the name to default. So when running the report it tries to slice a DataFrame with none.

emacha avatar Jun 13 '25 11:06 emacha

Is there "BinaryClassification" library from evidently? i didn't saw any library matrices like this in last version!

sarahshehri avatar Jul 23 '25 16:07 sarahshehri