seldon-core icon indicating copy to clipboard operation
seldon-core copied to clipboard

Configure Server volumeClaimTemplate accessModes setting in Seldon-Core V2

Open GuyAttia opened this issue 11 months ago • 0 comments

Describe the bug

When trying to set an MLServer replicas > 1 in Seldon-Core V2, I'm getting an error related to the PVC:

Multi-Attach error for volume "pvc-*****************" Volume is already exclusively attached to one node and can't be attached to another.

As I understand it's due to the fact that the pods are on different nodes and the definition of the volumeClaimTemplates for mlserver-models is defined as ReadWriteOnce by default. When I manually changing the definition to ReadWriteMany in the ServerConfig the error is gone.

To reproduce

Deploy a Server resource (I used the mlserver serverConfig in this case) with more than 1 replica. ** You should deploy it on a multi-node cluster

Expected behavior

Following this Slack thread, I understand I can configure the helm chart or create a custom ServerConfig. But, I think it will be valuable to have an option to configure it per Server resource

GuyAttia avatar Sep 11 '23 09:09 GuyAttia