Seldon + Istio: Custom URI re-write in VirtualService configuration

[senthilrch]

I have deployed seldon-core with Istio enabled. i.e. Istio Ingress gateway is used to route the model invoking request to seldon. Model invocation works fine if I hit the endpoint:-

https://<hostname>/seldon/<namespace>/<deployment>/api/v1.0/predictions

I need the ability to invoke models without the trailing path "/api/v1.0/predictions" i.e. like this:-

https://<hostname>/seldon/<namespace>/<deployment>/

The virtual service created by seldon controller is as follows:-

spec:
  gateways:
  - mxe-senthil/gateway-mxe-csr
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /seldon/<namepsace>/<deployment>/
    rewrite:
      uri: /
    route:
    - destination:
        host: single-stateless-manual-main
        port:
          number: 8000
        subset: main

Request to allow user to configure re-write rules when deploying seldon, so that it creates a virtual service supporting custom uri re-write:-

spec:
  gateways:
  - mxe-senthil/gateway-mxe-csr
  hosts:
  - '*'
  http:
  - match:
    - uri:
        exact: /model-endpoints/single-stateless-manual-main
    rewrite:
      uri: /api/v1.0/predictions
    route:
    - destination:
        host: single-stateless-manual-main
        port:
          number: 8000
        subset: main

[ukclivecox]

At present /api/v1.0/predictions is expected in the service orchestrator and python wrapper as the v1 protocol path for predictions so even if we did allow this it would need changes on all configurations to accept the new paths which is harder to achieve as for python wrapped files these are static images.

We would need to investigate allowing configurable paths.

[ukclivecox]

Can you give some background on your request so we can factor into future API updates. We are looking for what things to improve upon in v2 APIs.

[senthilrch]

We have an existing system that uses Nginx ingress controller -> Ambassador -> Model Pod to invoke the model. This system has exposed the URL endpoint /model-endpoints/single-stateless-manual-main to invoke the model. The URL re-write to support this, is performed in the Nginx ingress controller.

We are looking to remove Nginx ingress controller and replace Ambassador with Istio Ingress Gateway and still expose the URL endpoint /model-endpoints/single-stateless-manual-main to invoke the model, to maintain backward compatibility. In this setup the only place we could do the rewrite is the Istio Virtual Service, so we need the ability to configure seldon controller to create Virtual Service with custom URL rewrite.

[brightsparc]

I've proposed a #4299 as a simple way to support adding URL rewrite features for istio into the deployment spec.

[ukclivecox]

In v2 we are more agnostic to service meshes and they can be deployed alongside your v2 components. Please test in v2 and reopen if this does not work.