BinaryClassification with name != "default" crashes
I don't know if this is a bug or not supposed to be implemented. But I'll ask the question here.
The premise is that I have a model that trains in more than one target. So I want to pass more than one BinaryClassification class to my data_definition.
Given the signature is classification: List[Classification] | None, I'd assume this is intended as a feature.
If I don't define a unique name for BinaryClassification, it crashes on report.run because it's not unique.
>>> raise ValueError("More than one classification with id {}".format(classification_id))
>>>ValueError: More than one classification with id default
However, if I do define a name, it doesn't work like my example below.
import pandas as pd
from evidently import Dataset, DataDefinition, Report, BinaryClassification
from evidently.metrics import Accuracy
data = pd.DataFrame({
"id_col": [1, 2, 3],
"actual_1": [10.0, 20.0, 30.0],
"predicted_1": [11.0, 19.0, 29.0],
"actual_2": [10.0, 20.0, 30.0],
"predicted_2": [11.0, 19.0, 29.0],
})
classification_ = [BinaryClassification(target="actual_1", prediction_labels="predicted_1", name="regression_1"), BinaryClassification(target="actual_2", prediction_labels="predicted_2", name="regression_2")]
data_definition = DataDefinition(id_column="id_col", classification=classification_)
dataset = Dataset.from_pandas(data, data_definition=data_definition)
report = Report(metrics=[Accuracy(target="actual_1", prediction="predicted_1"),])
report.run(dataset)
>>> raise KeyError(key) from err
>>> KeyError: None
I think I traced it to this function _default_input_data_generator, that hardcodes the name to default. So when running the report it tries to slice a DataFrame with none.
Is there "BinaryClassification" library from evidently? i didn't saw any library matrices like this in last version!