Istio enabled Seldon core: Support for URI re-write customization in VirtualService created by the controller.

[kpriyank20]

In case of ambassador, the deployed model endpoints can be easily customized with URI path re-write option with the below annotation - seldon.io/ambassador-config

Example SeldonDeployment: apiVersion: machinelearning.seldon.io/v1alpha2 kind: SeldonDeployment metadata: creationTimestamp: '2021-07-06T13:15:24Z' ................ ................ ........ spec: annotations: ....... project-name: test_project seldon.io/ambassador-config: | --- apiVersion: ambassador/v2 kind: Mapping name: seldon_testmodel_default_rest_mapping1 prefix: /xyz/test_project/testmodel/ rewrite: / service: testmodel-default.svc:9000 timeout_ms: 300000 --- ............

However, if Istio ingress is enabled in seldon core operator, the related virtual service is auto generated by the controller and I could not find any documentation on seldon.io, on how we can customize the model endpoints in the virtual service to achieve similar flexibility as offered in case of ambassador.

Kindly share the relevant documentation if this is possible ? If not, is there any road map to extend customized URI re-write feature to istio virtual service creation as well ?

[ukclivecox]

No you are right we have limited customization functionality at present: https://docs.seldon.io/projects/seldon-core/en/latest/ingress/istio.html#istio-configuration-annotation-reference

Can you describe your use case?

[kpriyank20]

The use case we have is - we are using seldon core operator to manage ML models deployment on RedHat OpenShift cluster. While doing so we are using Istio ingress to expose the MLOps endpoints to external systems, so requirement here is to have some custom URI prefix path instead of the default prefix, in the auto generated Istio virtual service as shown below -

apiVersion: networking.istio.io/v1alpha3 kind: VirtualService ................. ................ - match: - uri: prefix: /seldon/<namespace_name>/<deployment_name>/ rewrite: uri: / ................

[ukclivecox]

If we had a single vanilla SVC endpoint for all models then we could allow users to add any istio resources as needed as the spec for these would be of a predictable nature. This is something we are investigating for v2 APIs in core. This would allow you to create any customized istio resources as needed when deploying a model.

[kpriyank20]

Thanks for the update @cliveseldon , is there any release timeline targeted for making the customized istio resources feature available ? Or does the recent releases say 1.11.x has any support on these lines ?

[ukclivecox]

Not exactly - 1.12 will hopefully have https://github.com/SeldonIO/seldon-core/pull/3609 which may help We do have functionality like this for Ambassador. We'd be open to a PR if you want to look at it or propose one.

[github-actions[bot]]

This issue is stale because it has been open 10 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.