seldon-core icon indicating copy to clipboard operation
seldon-core copied to clipboard

support use model-initializer with "no-engine" when modelUri set

Open cnfatal opened this issue 2 years ago • 1 comments

For some historical reason, I need serving some custom protocol models(image). I'm not able to Building a new inference server for these models, so I have to serve them directly with the annotation seldon.io/no-engine: "true".

Then I cannot use storageInitializer even though the modelUri set, the init container not injected.

As an alt, I set the init container in componentSpecs by hand, but it's not what I want.

apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: sklearn-iris
spec:
  predictors:
    - name: default
      replicas: 1
      annotations:
        seldon.io/no-engine: "true"
      graph:
        name: classifier
        type: MODEL
        # modelUri: s3://sklearn/iris
      componentSpecs:
        - spec:
            containers:
              - name: classifier
                image: some-model-image:1.13
                securityContext:
                  privileged: true
                ports:
                  - containerPort: 8080
                    protocol: TCP
                    name: http
                volumeMounts:
                  - mountPath: /mnt/models
                    name: classifier-provision-location
                    readOnly: true
            volumes:
              - name: classifier-provision-location
                emptyDir: {}
            initContainers:
              - name: classifier-model-initializer
                image: seldonio/rclone-storage-initializer:1.15.0-dev
                imagePullPolicy: IfNotPresent
                args:
                  - "s3://sklearn/iris"
                  - "/mnt/models"
                volumeMounts:
                  - mountPath: /mnt/models
                    name: classifier-provision-location
                envFrom:
                  - secretRef:
                      name: seldon-init-container-secret

I'm glad to implement this feature if needed

cnfatal avatar Aug 02 '22 10:08 cnfatal

There are some cases:

Case 0:

I have several images of embedded models that implement sekldon's Custom inference servers, But there are too many images to write each of them into the seldon-config configmap.

apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: sklearn-iris
spec:
  predictors:
    - name: default
      replicas: 1
      graph:
        name: classifier
        # implementation: null
        modelUri: gs://seldon-models/v1.15.0-dev/sklearn/iris
      componentSpecs:
        - spec:
            containers:
              - name: classifier
                image: model-server-allinone:latest

Case 1(my situation):

In Case 0, some images not implement Custom inference servers,so I must serve them directly with seldon.io/no-engine: "true". It's simple like a web app.

apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: sklearn-iris
spec:
  predictors:
    - name: default
      replicas: 1
      annotations:
         seldon.io/no-engine: "true"
      graph:
        name: classifier
        modelUri: gs://seldon-models/v1.15.0-dev/sklearn/iris
        endpoint:
          type: REST
          httpPort: 80
      componentSpecs:
        - spec:
            containers:
              - name: classifier
                image: nginx:1.13
                securityContext:
                  privileged: true
                ports:
                  - containerPort: 80
                    protocol: TCP
                    name: http

In both cases I need to use rclone to download model files from some storage server. But in these cases the initialize container is not injection when modelUri is set.

cnfatal avatar Aug 08 '22 05:08 cnfatal

Closing also please try this in v2

ukclivecox avatar Dec 19 '22 10:12 ukclivecox