Model validation with feature store should be supported.

[mingyu89]

Mode validation uses mlflow.evaluate. Currently it doesn't support evaluating models registered with databricks feature store.

[aminebizid]

Any news about this limitation please?

[jackroseman]

I am also wondering when this feature is coming. Thanks!

[skaliy]

Any updates? :)

[arpitjasa-db]

Hi we're looking into adding support for this, the tricky part is we need to update underlying feature store and MLflow components to get this to work, but will hopefully have an update for you all soon!

[david-straub]

Any updates?

[arpitjasa-db]

@david-straub we are currently reviewing a PR to add this functionality, but note this workaround will be slightly limited in that we cannot enable baseline comparison, as we need to provide a model_uri of a pyfunc model (so this means it's not possible to compare a new FS model with a baseline FS model)

[arpitjasa-db]

We just merged https://github.com/databricks/mlops-stacks/pull/165 thanks to @aliazzzdat! This workaround should resolve most use cases for this issue, but if there's any other problems, please open another issue thank you!