[Error Analysis] Support for bringing in features that are not being used by the model but useful for error analysis

[nushib]

trafficstars

Suggested by Melanie Fernandez Pradier:

“Given a model trained on certain features, is there any way I can include additional features (not used in training) but that I want to monitor in the error analysis?"

This is currently possible by enriching the set of input features to the dashboard after the inference step. However, will need further support on the UI side to clearly mark features that are not being used by the model in the feature list as well as in the combo boxes of the heatmap. May also need a brief example in the notebooks and documentation.

[romanlutz]

This reminds me of sensitive features in fairness use cases! They often have to be excluded for legal reasons, but we still want to analyze the results to see if any groups would be treated differently, etc.