woodwork icon indicating copy to clipboard operation
woodwork copied to clipboard

Consider Ordinal handling of training and holdout data with different categories

Open tamargrey opened this issue 2 years ago • 0 comments

There are situations where we may instantiate an Ordinal logical type for a column based on training data, and then when we see the holdout data, there may be new categories that weren't present in the training data, causing the Ordinal validation to fail.

I'm wondering if it would be worth considering a workaround in the Ordinal validation for this sort of situation. Maybe a handling that just converts new values to be nans or allows the user to specify new values in the order. That said, maybe it should be on the Woodwork user to be aware of this and handling it themself.

This issue may end up coming up in EvalML as we add the OrdinalEncoder and further increase Ordinal usage across EvalML.

tamargrey avatar Dec 13 '22 19:12 tamargrey