cluster-operator icon indicating copy to clipboard operation
cluster-operator copied to clipboard

Problem with AKS storage

Open michalalter opened this issue 5 years ago • 3 comments

Hi all,

Am I missing smth or I don't see any capabilities to install storageos (even 2.x version) on dedicated pvcs attached to the nodes in clouds Kubernetes distrubutions? For example this possibility has rook - could be provisioned and use pvc mounts from osd pods The relevant part of rook:

        spec:
          resources:
            requests:
              storage: 128Gi
          storageClassName: managed-premium
          volumeMode: Block
          accessModes:
            - ReadWriteOnce

The scenario is that in clouds like Azure - cloud creates AKS nodes without possibility to manipulate local storage On the other hand, I could do some workarounds like initcontainers invoking /var/lib/storageos space covered by azure-disk but I'm expecting storageos to do that.. I'm expecting the possibility to point /var/lib/storageos into PVC volume

Regards, Michal

michalalter avatar Jun 27 '20 21:06 michalalter

Hello Michal,

If I didn't misunderstand your question, no you can't install StorageOS on dedicated PVCs. StorageOS runs as daemonset pods on all the nodes in your cluster (you can also run storageos on a subset of nodes only as well if your use case requires it). For example following this guide here https://docs.storageos.com/docs/install/kubernetes/ but first making sure all the Prerequisites are met (see docs; https://docs.storageos.com/docs/prerequisites/).

We do have a roadmap item for StorageOS to provision storage directly from cloud providers but unfortunately there is not a defined date for this yet.

Please let us know if you got more questions.

Thank you, Stella

senglezou avatar Jun 29 '20 12:06 senglezou

Hi Stella, Thank you for your answer. Let me clarify a little bit more. StorageOS by default is using /var/lib/storaeos directory for the storage and the rest of stuff. In vanilia Kubernetes I can do OS preparation very easily, I have access to nodes, so I can add a drive and mount (lvm or direct) to this dir. In Cloud Kubernetes distributions, the access to the native OS is a little bit more difficult. I could do ssh (in my case I'm talking about AKS in Azure) access and change smth on the nodes, but I need to take into account that after upgrades (Kubernetes or vms) it'll be not a persistent change. So in my case this is blocking a decision about using StorageOS in AKS.

Regards, Michal

michalalter avatar Jun 29 '20 15:06 michalalter

Hello Michael,

I understand the difficulty you are facing in preparing your environment in AKS but unfortunately at the moment we don't offer a solution for this out of the box. We do have a roadmap item to add support in the future but we don't have a set released date unfortunately. Please also note that Azure has rolling updates on VMs for Kubernetes version changes, however an in-place upgrades are needed when storing data.

Regards, Stella

senglezou avatar Jul 07 '20 13:07 senglezou