Custom model format breaks Models UI

[ghost]

Currently with, if a user specifies a model format with a custom name in the inference service manifest and deploys it, the Models UI on kubeflow will become disfunctional and not showing any model deployed under the same namespace. This has been mentioned in #46 but there has been no update.

For example,

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: dummy-model
  namespace: dummy-namespace
spec:
  predictor:
    model:
      image: dummy-image
      modelFormat:
        name: my-model
      name: 'kserve-container'

with a ClusterServingRuntime

kind: ClusterServingRuntime
metadata:
  name: kserve-my-model
spec:
  containers:
    image: dummy-image
    name: kserve-container
  supportedModelFormats:
  - autoSelect: true
    name: my-model

This is because PredictorType is currently hardcoded as an Enum (See here), but getPredictorType function will return the name of the model format as an Enum directly instead of checking if it's a member of the Enum first.. TypeScript does not raise an error when by default when this happens, and will return undefined, which breaks all the downstream tasks, and the logs from the web-app will not show anything related to this.

Suggestion: Check if the model format is a member of PredictorType Enum and returns PredictorType.Custom if it's not (similarly to what we do with old predictor formats before Kserve 0.7). I believe #47 has already attempted at fixing this problem but I can also make a PR for this.

Question: How should we make PredictorType more flexible for custom model types? Say a user has custom model types: "A", "B" and "C". Even with the above suggestion, the Models UI will still show custom for all three.

[Sayed-Imran]

The same is happening for the sample mlflow modelFormat.

The model is getting deployed, and I am even able to inference the model, still it is not showing in the UI

[supertetelman]

I am encountering this same issue when I launch my custom defined ServingRuntime and InferenceService (https://github.com/supertetelman/nim-kserve/blob/main/nim-models/llama-2-7b_1-a100_24.01.yaml) running on build version Kubeflow v1.8.0

When I deploy this InferenceService I need to check the developer console to get the endpoint URL and things work. But nothing is displayed in the UI.

[capoolebugchat]

Raising this issue again. Everything works great but this undermines the project, as models logging for debugs (without the need to read the Pod's logs or any additional services) is quite a crucial part for small ML teams.