openebs-docs
openebs-docs copied to clipboard
Update the steps for cstor volume resize.
From OpenEBS 1.2 onwards how to perform resize of cStor Volume
Step1: Find the cstorvolume of pvc which needs to perform PVC resize(kubect get cstorvolume -n <openebs_namespace> -l openebs.io/persistent-volume=<pv_name>).
Step2: Update the cstorvolume spec.Capacity with the new size(Wait till the status.capacity of cstorvolume is updated to the latest size).
Step3: https://github.com/openebs/openebs-docs/blob/day_2_ops/docs/cstor_volume_resize.md#rescan-iscsi-on-node-where-application-is-running (As mentioned in the above link following manual steps are required to rescan of iscsi session, resizing filesystem, verify whether size got reflected inside the application pod and at last updating PV capacity)
Below are the detailed steps to perform cStor volume resize and If OpenEBS version >= 1.2 then follow the below steps. Click here to perform resize if OpenEBS version is < OpenEBS1.2
Step1: Find the cstorvolume of corresponding PV
kubectl get cstorvolume -n openebs -l openebs.io/persistent-volume=pvc-15f6b83a-4ce7-471c-9fdd-64d092ddf105(pv name)
NAME STATUS AGE CAPACITY
pvc-15f6b83a-4ce7-471c-9fdd-64d092ddf105 Healthy 105m 5Gi
Step2: Edit spec.capacity in the cStorVolume
kubectl edit cstorvolume pvc-15f6b83a-4ce7-471c-9fdd-64d092ddf105 -n openebs
Step3: Wait still the status.capacity
was updated to latest capacity (or) events on cstorvolume will say resize was done
From Status.Capacity
kubectl get cstorvolume pvc-15f6b83a-4ce7-471c-9fdd-64d092ddf105 -n openebs
NAME STATUS AGE CAPACITY
pvc-15f6b83a-4ce7-471c-9fdd-64d092ddf105 Healthy 109m 10Gi
Events
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Updated 2s (x3 over 111m) pvc-15f6b83a-4ce7-471c-9fdd-64d092ddf105-target-777c9c8c7cgbd94, 127.0.0.1 Updated resize conditions
Normal Updated 1s pvc-15f6b83a-4ce7-471c-9fdd-64d092ddf105-target-777c9c8c7cgbd94, 127.0.0.1 successfully resized volume from 5Gi to 10Gi
Resizing cStorvolume was done with above steps. To reflect this size in application perform below steps
Rescan iscsi on node where application is running
Find correct node
$ kubectl -n <APP_NAMESPACE> get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
percona-test-percona-db78ff4d8-qztph 1/1 Running 0 55m 10.244.3.7 k8s-openebs-demo-node-0 <none> <none>
Connect to node (ssh, gcloud compute ssh, etc) and run the following commands
Follow your standard procedures for connecting to your node. See instructions from your cloud provider regarding SSH for more information.
Rescan iscsi
$ sudo iscsiadm -m node -R
Rescanning session [sid: 2, target: iqn.2016-09.com.openebs.cstor:pvc-15f6b83a-4ce7-471c-9fdd-64d092ddf105, portal: 10.103.75.24,3260]
Verify size using lsblk and get device name
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 88.4M 1 loop /snap/core/6964
loop1 7:1 0 58.9M 1 loop /snap/google-cloud-sdk/84
loop2 7:2 0 59.1M 1 loop /snap/google-cloud-sdk/85
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 19.9G 0 part /
├─sda14 8:14 0 4M 0 part
└─sda15 8:15 0 106M 0 part /boot/efi
sdb 8:16 0 50G 0 disk
├─sdb1 8:17 0 50G 0 part
└─sdb9 8:25 0 8M 0 part
sdc 8:32 0 50G 0 disk
├─sdc1 8:33 0 50G 0 part
└─sdc9 8:41 0 8M 0 part
sdd 8:48 0 10G 0 disk /var/lib/kubelet/pods/ce5888b6-87b8-11e9-b062-42010a800014/volumes/kubernetes.io~iscsi/pvc-15f6b83a-4ce7-471c-9fdd-64d092ddf105
Resize the filesystem
The device name was listed in the last command. That will be used here for the resize.
$ sudo resize2fs /dev/<DEVICE_NAME>
resize2fs 1.44.1 (24-Mar-2018)
Filesystem at /dev/sdd is mounted on /var/lib/kubelet/plugins/kubernetes.io/iscsi/iface-default/10.103.75.24:3260-iqn.2016-09.com.openebs.cstor:pvc-15f6b83a-4ce7-471c-9fdd-64d092ddf105-lun-0; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 3
The filesystem on /dev/sdd is now 2621440 (4k) blocks long.
Verify size in application pod
$ kubectl -n <APP_NAMESPACE> exec -it <APP_POD> -- df -h
Filesystem Size Used Avail Use% Mounted on
overlay 20G 3.6G 16G 19% /
tmpfs 64M 0 64M 0% /dev
tmpfs 1.8G 0 1.8G 0% /sys/fs/cgroup
/dev/sda1 20G 3.6G 16G 19% /etc/hosts
shm 64M 0 64M 0% /dev/shm
/dev/sdd 10G 258M 20G 2% /var/lib/mysql
tmpfs 1.8G 12K 1.8G 1% /run/secrets/kubernetes.io/serviceaccount
tmpfs 1.8G 0 1.8G 0% /proc/acpi
tmpfs 1.8G 0 1.8G 0% /proc/scsi
tmpfs 1.8G 0 1.8G 0% /sys/firmware
Edit PV
Get PV
$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-15f6b83a-4ce7-471c-9fdd-64d092ddf105 5Gi RWO Delete Bound default/percona-test-percona openebs-cstor-test 46m
Patch PV
Now edit spec.capacity.storage
in the PV
NOTE: PVC size will remain the same as the old size(but that will not be a problem for application) but the application can consume the expanded size.
Document has been updated via #755