Running a stateful container with raw block volume configurations

Use this section as a sample of how to run a stateful container with a raw block volume configuration.

Before you begin

Before starting the procedure, be sure to see all information detailed in Running a stateful container with file system configurations.

Procedure

Open a command-line terminal.

Create an array secret.

Important: Be sure that the username and password match the same username and password used on the storage system.

$> cat demo-secret-svc-array.yaml
kind: Secret
apiVersion: v1
metadata:
  name: svc-array
  namespace: csi-ns
type: Opaque
stringData:
   management_address: <ADDRESS-1, ADDRESS-2> # Array management addresses
   username: <USERNAME>                   # Array username
data:
   password: <PASSWORD base64>            # replace with valid password
  
$> kubectl create -f demo-secret-svc-array.yaml
secret/svc-array created

Create a storage class.

Note: The SpaceEfficiency values for Spectrum Virtualize Family are: thick, thin, compressed, or deduplicated. These values are not case specific.

For DS8000® Family systems, the default value is standard, but can be set to thin, if required. These values are not case specific. For more information, see Creating storage classes.

This parameter is not applicable for IBM FlashSystem® A9000 and A9000R systems. These systems always include deduplication and compression.

$> cat demo-storageclass-gold-svc.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: gold
provisioner: block.csi.ibm.com
parameters:
  SpaceEfficiency: deduplicated
  pool: gold

  csi.storage.k8s.io/provisioner-secret-name: svc-array
  csi.storage.k8s.io/provisioner-secret-namespace: csi-ns
  csi.storage.k8s.io/controller-publish-secret-name: svc-array
  csi.storage.k8s.io/controller-publish-secret-namespace: csi-ns

  csi.storage.k8s.io/fstype: xfs   # Optional. values ext4\xfs. The default is ext4.
  volume_name_prefix: demo         # Optional.

$> kubectl create -f demo-storageclass-gold-svc.yaml
storageclass.storage.k8s.io/gold created

Create a PVC demo-pvc-raw-block.yaml with the size of 1 Gb.

Note: For more information about creating a PVC yaml file, see Creating a PersistentVolumeClaim (PVC).

https://github.com/IBM/ibm-block-csi-driver/blob/develop/deploy/kubernetes/examples/demo-pvc-raw-block.yaml$> cat demo-pvc-raw-block.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: demo-pvc-raw-block
spec:
  volumeMode: Block
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: gold

$> kubectl apply -f demo-pvc-raw-block.yaml
persistentvolumeclaim/demo-pvc-raw-block created

Display the existing PVC and the created persistent volume (PV).

Note: For more information about creating a PVC yaml file, see Creating a PersistentVolumeClaim (PVC).

$> kubectl get pv,pvc
NAME                                                        CAPACITY   ACCESS MODES
persistentvolume/pvc-828ce909-6eb2-11ea-abc8-005056a49b44   1Gi        RWO

RECLAIM POLICY   STATUS   CLAIM              STORAGECLASS   REASON   AGE
Delete           Bound    default/demo-pvc-raw-block   gold                    109m

NAME                             STATUS   VOLUME                                     CAPACITY   
persistentvolumeclaim/demo-pvc-raw-block   Bound    pvc-828ce909-6eb2-11ea-abc8-005056a49b44   1Gi

ACCESS MODES   STORAGECLASS   AGE
RWO            gold           78s

kubectl describe persistentvolume/pvc-828ce909-6eb2-11ea-abc8-005056a49b44
Name:            pvc-828ce909-6eb2-11ea-abc8-005056a49b44
Labels:          <none>
Annotations:     pv.kubernetes.io/provisioned-by: block.csi.ibm.com
Finalizers:      [kubernetes.io/pv-protection external-attacher/block-csi-ibm-com]
StorageClass:    gold
Status:          Bound
Claim:           default/demo-pvc-raw-block
Reclaim Policy:  Delete
Access Modes:    RWO
VolumeMode:      Block
Capacity:        1Gi
Node Affinity:   <none>
Message:
Source:
    Type:              CSI (a Container Storage Interface (CSI) volume source)
    Driver:            block.csi.ibm.com
    VolumeHandle:      SVC:60050760718106998000000000000543
    ReadOnly:          false
    VolumeAttributes:      array_address=baremetal10-cluster.xiv.ibm.com
                           pool_name=csi_svcPool
                           storage.kubernetes.io/csiProvisionerIdentity=1585146948772-8081-block.csi.ibm.com
                           storage_type=SVC
                           volume_name=demo_pvc-828ce909-6eb2-11ea-abc8-005056a49b44
Events:                <none>

Create a StatefulSet, using the demo-statefulset-raw-block.yaml.

Note: For more information about creating a StatefulSet, see Creating a StatefulSet.

$> kubectl create -f demo-statefulset-raw-block.yaml
statefulset/demo-statefulset-raw-block created

$> cat demo-statefulset-raw-block.yaml

kind: StatefulSet
apiVersion: apps/v1
metadata:
  name: demo-statefulset-raw-block
spec:
  selector:
    matchLabels:
      app: demo-statefulset
  serviceName: demo-statefulset
  replicas: 1
  template:
    metadata:
      labels:
        app: demo-statefulset
    spec:
      containers:
      - name: container-demo
        image: registry.access.redhat.com/ubi8/ubi:latest
        command: [ "/bin/sh", "-c", "--" ]
        args: [ "while true; do sleep 30; done;" ]
        volumeDevices:
          - name: demo-volume
            devicePath: "/dev/block"
      volumes:
      - name: demo-volume
        persistentVolumeClaim:
          claimName: demo-pvc-raw-block

#      nodeSelector:
#        kubernetes.io/hostname: HOSTNAME

Check the newly created pod.

Display the newly created pod (make sure the pod status is Running).

$> kubectl get pod demo-statefulset-raw-block-0
NAME                 READY   STATUS    RESTARTS   AGE
demo-statefulset-raw-block-0   1/1     Running   0          43s

Write data to the persistent volume of the pod.

The PV should be mounted inside the pod at /dev.

$> kubectl exec podraw67b -- bash -c " echo "test_block" | dd conv=unblock of=/dev/block"
0+1 records in
0+1 records out
11 bytes copied, 9.3576e-05 s, 118 kB/s
$> kubectl exec podraw67b -- bash -c "od -An -c -N 10 /dev/block"
t e s t _ b l o c k

Delete StatefulSet and then recreate, in order to validate data (test_block in /dev/block) remains in the persistent volume.

Delete the StatefulSet.

$> kubectl delete statefulset/demo-statefulset-raw-block
statefulset/demo-statefulset-raw-block deleted

Wait until the pod is deleted. Once deleted the '"demo-statefulset-raw-block" not found' is returned.


$> kubectl get statefulset/demo-statefulset-raw-block
Error from server (NotFound): statefulsets.apps <StatefulSet name> not found

Recreate the StatefulSet and verify that the content written to /dev/block exists.

$> kubectl create -f demo-statefulset-raw-block.yml
statefulset/demo-statefulset-raw-block created

$> kubectl exec podraw67b -- bash -c "od -An -c -N 10 /dev/block"
t e s t _ b l o c k

Delete StatefulSet and the PVC.

$> kubectl delete statefulset/demo-statefulset-raw-block
statefulset/demo-statefulset-raw-block deleted

$> kubectl get statefulset/demo-statefulset-raw-block
No resources found.

$> kubectl delete pvc/demo-pvc-raw-block
persistentvolumeclaim/demo-pvc-raw-block deleted

$> kubectl get pv,pvc
No resources found.