evidently
evidently copied to clipboard
evidentlyai
TestColumnCorrelation(column_name="XYZ") Enhancement
As part of my reporting on model performance I've been using ColumnCorrelationsMetric(column_name='model_score') to keep an eye on changes between how the inputs are coming in and the model_score coming out. I was hoping to use a TestColumnCorrelation style call in my TestSuite to go along with the report form of the call. I can use TestCorrelationChanges(is_critical=False) broadly for the dataset, but I'd like to have a much more limited test to specifically are there changes in the corellations between this column I specifically care about and all of the other columns provided in the reference/current datasets.
Is there any plan to add Tests based on the already existing metric looking at changes between the reference and current datasets? Would be quite useful for highlighting changes between high value columns (like the model output) and the rest of the dataset instead of getting warnings for all correlation changes.
Thank you @ZWMiller - this indeed makes sense! We'll try to add this functionality in one of the next releases.
Hi @ZWMiller, we added the column_name parameter to the TestCorrelationChanges test - available in Evidently 0.4.5. Kudos to @0lgaF for the implementation!
If you pass this parameter, only the correlations between the selected column and other columns in the dataset will be considered.
To check correlation changes for several chosen columns, you should create several Tests in a suite with different column_name.
Let us know how it'll work for you!
Hey @elenasamuylova . I tested this today and it works as I expected, thanks! Only request - if there is a column_name provided, is it possible to update the title of the test outcome to say "Change in Correlation with column_name" . Right now, the title implies there's no change in any correlations.
Thanks!
Thanks for testing it out @ZWMiller!
cc @0lgaF - could you take at a look at how we update the wording in the test outcome?