Custom validation set

[davidlkl]

Hi, I find this package very useful for EDA, however current it does not support using custom validation set in fit(). In XGBoost / Lightgbm, there is a parameter eval_set to allow for that.

In my use case, I am doing a 5-fold group-stratified CV to train different base models and then stacking them up. Therefore I would need to fit the model by custom validation set.

Similar issue: https://github.com/interpretml/interpret/issues/177

[interpret-ml]

Hi @davidlkl ,

Thanks for bringing this up! Your use case makes perfect sense, and supporting custom validation sets is on our backlog though we don't have an exact timeline for when we'll have it implemented.

The eval_set parameter seems like a nice way of introducing this to the API. Because the other issue you referenced is closed, we'll leave this one open to track progress on this.

Thanks, -InterpretML Team

[sarim-zafar]

Hello, this request has been around for almost 2 years now. Has any progress been made on this front? Is there a dirty way around to achieve this given it has not been released yet?

Also, I am adding links to similar posts (the ones I could find) requesting the same feature going back to 2019 so that the mod can consolidate all of them in one place.

https://github.com/interpretml/interpret/issues/31 https://github.com/interpretml/interpret/issues/208 https://github.com/interpretml/interpret/issues/177

[paulbkoch]

It remains in our backlog for future implementation. We'd welcome a PR that implements it.

Closing this issue to consolidate custom validation sets into issue #31 (thanks for listing the duplicates).