RAIVisionInsights to accept torch Datasets

[microsoft-sampsa]

Is your feature request related to a problem? Please describe.

In torch we have (standard) Dataset and Dataloader classes which work fine, however, in the multiclass image classification example in here, the only input RAI accepts, are panda tables with a filename column.

Ideally, one could use the same Datasets (which eventually uses stuff from azure datastore) as one was using to train and test the model in the first place, without the need to create yet another differently-formatted dataset scheme: a directory with files + pandas table with the 0/1 column format, etc.

Describe the solution you'd like responsibleai_vision.RAIVisionInsights accepting torch Dataset

Describe alternatives you've considered n/a

Additional context

If you follow the links in here: https://github.com/microsoft/responsible-ai-toolbox/tree/main#responsible-ai-dashboard-customization "Try the tool: model debugging of a fridge image classification model" that link is broken

A bonus question

I'd imagine you know the answer to this: as of today, azure ml does not support RAIVisionInsights?

Emphasis on that small appearing toast that says "only support models with scikit-learn flavor"

EDIT: maybe it's just me, but you could really emphasize all over the place that this thing only works with "fastai" API

EDIT 2: I mean - really? How about printing the traceback when catching the Exception ..?

File /anaconda/envs/mlflow-env/lib/python3.10/site-packages/responsibleai_vision/rai_vision_insights/rai_vision_insights.py:408, in RAIVisionInsights._validate_rai_insights_input_parameters(self, model, test, target_column, task_type, classes, serializer, maximum_rows_for_test)
    406     model.predict(test_img)
    407 except Exception:
--> 408     raise UserConfigValidationException(
    409         'The model passed cannot be used for'
    410         ' getting predictions via predict()'
    411     )

UserConfigValidationException: The model passed cannot be used for getting predictions via predict()

[jamesbchao]

Hello, thanks for reaching out with this suggestion. I will send out a PR in the next couple days with code that allows users to use standard pytorch models and datasets with the vision dashboard. Thanks!