woodwork
woodwork copied to clipboard
Consider Ordinal handling of training and holdout data with different categories
There are situations where we may instantiate an Ordinal logical type for a column based on training data, and then when we see the holdout data, there may be new categories that weren't present in the training data, causing the Ordinal validation to fail.
I'm wondering if it would be worth considering a workaround in the Ordinal validation for this sort of situation. Maybe a handling that just converts new values to be nans or allows the user to specify new values in the order. That said, maybe it should be on the Woodwork user to be aware of this and handling it themself.
This issue may end up coming up in EvalML as we add the OrdinalEncoder and further increase Ordinal usage across EvalML.