VMware vSphere Container Storage Plug-in 3

3.0 2.0

Expanding a Volume with vSphere Container Storage Plug-in

vSphere Container Storage Plug-in
 supports volume expansion for block volumes that are created dynamically or statically.
Kubernetes supports offline and online modes of volume expansion. When the PVC is used by a pod and is mounted on a node, the volume expansion operation is categorized as online. In all other cases, it is an offline expansion. For information about vSphere versions that support volume expansion, see Supported Kubernetes Functionality.
Expanding volume is not supported when volume snapshot is present, and when a Node VM snapshot is present with the volume attached to it.
The following considerations apply when you use volume expansion:
Feature Gate
Expand CSI Volumes feature is enabled by default since it was promoted to beta in Kubernetes 1.16. For Kubernetes releases before 1.16, enable Expand CSI Volumes feature gate to support volume expansion in
vSphere Container Storage Plug-in
.
Sidecar Container
An external resizer sidecar container implements the logic of watching the Kubernetes API for PVC edits, issuing the ControllerExpandVolume RPC call against a CSI endpoint, and updating the
PersistentVolume
 object to reflect the new size. This container is already deployed as part of the vsphere-csi-controller pod.
Volume Resize Limitations
  • Kubernetes does not support expansion of a volume backed up by Statefulset.
  • vSphere Container Storage Plug-in
     does not support expansion of a volume backed up by
    vSAN
     file service.

Expand a Volume in Online Mode

vSphere Container Storage Plug-in
 supports volume expansion for block volumes in online mode.
  1. Create a PVC and a pod to use this PVC.
    1. Create a new StorageClass or edit the existing StorageClass to set
      allowVolumeExpansion
       to true.
       
      kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: example-block-sc
provisioner: csi.vsphere.vmware.com
allowVolumeExpansion: true
    2. Create or edit a PVC that references the StorageClass.
      Ensure that the PVC is in bound state. If you are using a statically provisioned PVC, ensure that the PVC and the PV specifications have the
      storageClassName
       parameter pointing to the StorageClass with the
      allowVolumeExpansion
       set to true.
    3. Deploy the PVC and create a pod to use this PVC.
       
      $ kubectl get pvc,pv,pod
NAME                                      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS           AGE
persistentvolumeclaim/example-block-pvc   Bound    pvc-84c89bf9-8455-4633-a8c8-cd623e155dbd   1Gi        RWO            example-block-sc       8m5s

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                       STORAGECLASS           REASON   AGE
persistentvolume/pvc-84c89bf9-8455-4633-a8c8-cd623e155dbd   1Gi        RWO            Delete           Bound    default/example-block-pvc   example-block-sc                7m59s

NAME                    READY   STATUS    RESTARTS   AGE
pod/example-block-pod   1/1     Running   0          7m1s
  2. Expand the PVC in online mode.
    1. Patch the PVC to increase the storage size, in this example, to
      2Gi
      .
       
      $ kubectl patch pvc example-block-pvc -p '{"spec": {"resources": {"requests": {"storage": "2Gi"}}}}'
persistentvolumeclaim/example-block-pvc patched
      This action triggers an expansion in the volume associated with the PVC in vSphere Cloud Native Storage.
    2. Describe the PVC.
      The output looks similar to the following. The PVC shows the increase in size after the volume underneath is expanded.
      $ kubectl describe pvc example-block-pvc
Name:          example-block-pvc
Namespace:     default
StorageClass:  example-block-sc
Status:        Bound
Volume:        pvc-84c89bf9-8455-4633-a8c8-cd623e155dbd
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed: yes
               pv.kubernetes.io/bound-by-controller: yes
               volume.beta.kubernetes.io/storage-provisioner: csi.vsphere.vmware.com
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      2Gi
Access Modes:  RWO
VolumeMode:    Filesystem
Mounted By:    example-block-pod
Events:
  Type     Reason                      Age   From                                                                                                Message
  ----     ------                      ----  ----                                                                                                -------
  Normal   ExternalProvisioning        19m   persistentvolume-controller                                                                         waiting for a volume to be created, either by external provisioner "csi.vsphere.vmware.com" or manually created by system administrator
  Normal   Provisioning                19m   csi.vsphere.vmware.com_vsphere-csi-controller-5d8c5c7d6-9r9kv_7adc4efc-10a6-4615-b90b-790032cc4569  External provisioner is provisioning volume for claim "default/example-block-pvc"
  Normal   ProvisioningSucceeded       19m   csi.vsphere.vmware.com_vsphere-csi-controller-5d8c5c7d6-9r9kv_7adc4efc-10a6-4615-b90b-790032cc4569  Successfully provisioned volume pvc-84c89bf9-8455-4633-a8c8-cd623e155dbd
  Warning  ExternalExpanding           75s   volume_expand                                                                                       Ignoring the PVC: didn't find a plugin capable of expanding the volume; waiting for an external controller to process this PVC.
  Normal   Resizing                    75s   external-resizer csi.vsphere.vmware.com                                                             External resizer is resizing volume pvc-84c89bf9-8455-4633-a8c8-cd623e155dbd
  Normal   FileSystemResizeRequired    69s   external-resizer csi.vsphere.vmware.com                                                             Require file system resize of volume on node
  Normal   FileSystemResizeSuccessful  6s    kubelet, k8s-node-072                                                                               MountVolume.NodeExpandVolume succeeded for volume "pvc-84c89bf9-8455-4633-a8c8-cd623e155dbd"
      The PVC goes through events
      Resizing
       to
      FileSystemResizeRequired
       to
      FileSystemResizeSuccessful
      .
    3. Display the PV to verify the expanded size.
       
      $ kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                       STORAGECLASS           REASON   AGE
pvc-84c89bf9-8455-4633-a8c8-cd623e155dbd   2Gi        RWO            Delete           Bound    default/example-block-pvc   example-block-sc                25m

Expand a Volume in Offline Mode

vSphere Container Storage Plug-in
 supports volume expansion for block volumes in offline mode.
  1. Create a PVC.
    1. Create a new StorageClass or edit the existing StorageClass to set
      allowVolumeExpansion
       to true.
       
      kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: example-block-sc
provisioner: csi.vsphere.vmware.com
allowVolumeExpansion: true
    2. Create or edit a PVC that references the StorageClass.
      Ensure that the PVC is in bound state. If you are using a statically provisioned PVC, ensure that the PVC and the PV specifications have the
      storageClassName
       parameter pointing to the StorageClass with the
      allowVolumeExpansion
       set to true.
    3. Deploy the PVC.
       
      $ kubectl get pvc,pv
NAME                                      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS           AGE
persistentvolumeclaim/example-block-pvc   Bound    pvc-9e9a325d-ee1c-11e9-a223-005056ad1fc1   1Gi        RWO            example-block-sc       5m5s

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                       STORAGECLASS           REASON   AGE
persistentvolume/pvc-9e9a325d-ee1c-11e9-a223-005056ad1fc1   1Gi        RWO            Delete           Bound    default/example-block-pvc   example-block-sc                5m18s
  2. Expand the PVC in offline mode.
    1. Patch the PVC to increase the storage size, in this example, to
      2Gi
      .
       
      $ kubectl patch pvc example-block-pvc -p '{"spec": {"resources": {"requests": {"storage": "2Gi"}}}}'
persistentvolumeclaim/example-block-pvc patched
      This action triggers an expansion in the volume associated with the PVC. The capacity of the corresponding PV object changes. However, the capacity of the PVC does not change until the PVC is used by a pod and mounted on a node.
      $ kubectl get pv
NAME                                       CAPACITY ACCESS MODES RECLAIM POLICY STATUS   CLAIM                       STORAGECLASS           REASON AGE
pvc-9e9a325d-ee1c-11e9-a223-005056ad1fc1   2Gi           RWO         Delete     Bound    default/example-block-pvc   example-block-sc              6m44s

$ kubectl get pvc
NAME                STATUS VOLUME                                     CAPACITY ACCESS MODES   STORAGECLASS       AGE
example-block-pvc   Bound  pvc-9e9a325d-ee1c-11e9-a223-005056ad1fc1   1Gi           RWO       example-block-sc   6m57s
      You can also see a
      FilesystemResizePending
       condition applied on the PVC when you describe it.
    2. Create a pod to use the PVC.
       
      apiVersion: v1
kind: Pod
metadata:
  name: example-block-pod
spec:
  containers:
  - name: test-container
    image: gcr.io/google_containers/busybox:1.24
    command: ["/bin/sh", "-c", "echo 'hello' > /mnt/volume1/index.html  && chmod o+rX /mnt /mnt/volume1/index.html && while true ; do sleep 2 ; done"]
    volumeMounts:
    - name: test-volume
      mountPath: /mnt/volume1
  restartPolicy: Never
  volumes:
  - name: test-volume
    persistentVolumeClaim:
      claimName: example-block-pvc
      $ kubectl create -f example-pod.yaml
pod/example-block-pod created
      The Kubelet on the node triggers the filesystem expansion on the volume when the PVC is attached to the pod.
      $ kubectl get pvc
NAME                STATUS VOLUME                                    CAPACITY ACCESS MODES STORAGECLASS     AGE
example-block-pvc   Bound  pvc-24114458-9753-428e-9c90-9f568cb25788   2Gi         RWO      example-block-sc 2m12s

$ kubectl get pv
NAME                                       CAPACITY ACCESS MODES RECLAIM POLICY STATUS   CLAIM                     STORAGECLASS           REASON AGE
pvc-24114458-9753-428e-9c90-9f568cb25788   2Gi           RWO        Delete      Bound    default/example-block-pvc example-block-sc              2m3s
      $ kubectl get pod
NAME               READY STATUS  RESTARTS AGE
example-block-pod   1/1  Running 0        65s
      The capacity of PVC is modified and the
      FilesystemResizePending
       condition is removed from the PVC. Offline volume expansion is complete.

Content feedback and comments