seldon-core
seldon-core copied to clipboard
support use model-initializer with "no-engine" when modelUri set
For some historical reason, I need serving some custom protocol models(image). I'm not able to Building a new inference server for these models, so I have to serve them directly with the annotation seldon.io/no-engine: "true"
.
Then I cannot use storageInitializer even though the modelUri
set, the init container not injected.
As an alt, I set the init container in componentSpecs
by hand, but it's not what I want.
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
name: sklearn-iris
spec:
predictors:
- name: default
replicas: 1
annotations:
seldon.io/no-engine: "true"
graph:
name: classifier
type: MODEL
# modelUri: s3://sklearn/iris
componentSpecs:
- spec:
containers:
- name: classifier
image: some-model-image:1.13
securityContext:
privileged: true
ports:
- containerPort: 8080
protocol: TCP
name: http
volumeMounts:
- mountPath: /mnt/models
name: classifier-provision-location
readOnly: true
volumes:
- name: classifier-provision-location
emptyDir: {}
initContainers:
- name: classifier-model-initializer
image: seldonio/rclone-storage-initializer:1.15.0-dev
imagePullPolicy: IfNotPresent
args:
- "s3://sklearn/iris"
- "/mnt/models"
volumeMounts:
- mountPath: /mnt/models
name: classifier-provision-location
envFrom:
- secretRef:
name: seldon-init-container-secret
I'm glad to implement this feature if needed
There are some cases:
Case 0:
I have several images of embedded models that implement sekldon's Custom inference servers, But there are too many images to write each of them into the seldon-config configmap.
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
name: sklearn-iris
spec:
predictors:
- name: default
replicas: 1
graph:
name: classifier
# implementation: null
modelUri: gs://seldon-models/v1.15.0-dev/sklearn/iris
componentSpecs:
- spec:
containers:
- name: classifier
image: model-server-allinone:latest
Case 1(my situation):
In Case 0, some images not implement Custom inference servers,so I must serve them directly with seldon.io/no-engine: "true"
. It's simple like a web app.
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
name: sklearn-iris
spec:
predictors:
- name: default
replicas: 1
annotations:
seldon.io/no-engine: "true"
graph:
name: classifier
modelUri: gs://seldon-models/v1.15.0-dev/sklearn/iris
endpoint:
type: REST
httpPort: 80
componentSpecs:
- spec:
containers:
- name: classifier
image: nginx:1.13
securityContext:
privileged: true
ports:
- containerPort: 80
protocol: TCP
name: http
In both cases I need to use rclone to download model files from some storage server. But in these cases the initialize container is not injection when modelUri is set.
Closing also please try this in v2