Troubleshooting installation and operation of IBM Storage Enabler for Containers

This section explains how to solve problems that may arise during installation and operation of the IBM Storage Enabler for Containers.

Table 1. Troubleshooting for IBM Storage Enabler for Containers
Description Corrective action
Failure to mount a storage volume to a Kubernetes node If the FlexVolume fails to locate a WWPN within multipath devices, verify your multipathing configuration and connectivity to a storage system.
IBM Storage Enabler for Containers database fails to achieve the Running state after the configured timeout expires
  • Check the kubectl logs for the FlexVolume pod on a node where the database was scheduled to run. Verify that the mount and rescan operations were successful. Another reason might be that the Docker image pulling is taking too much time, preventing the deployment to become active.
  • Check the kubectl logs for the FlexVolume pod that runs on the master node. Verify that the API was attached successfully.
  • Check the Kubernetes scheduling. Verify that it does not exceed the timeout configured in the installation script.
IBM Storage Enabler for Containers database persists in the Creating status In addition, the Volume has not been added to the list of VolumesInUse in the node's volume status message is stored in /var/log/message file on the node, where the database is deployed. To resolve this, move kube-controller-manager.yaml out and into /etc/kubernetes/manifests/ to be recreated the control-manager pod:
mv /etc/kubernetes/manifests/kube-controller-manager.yaml  /tmp
sleep 5
mv /tmp/kube-controller-manager.yaml /etc/kubernetes/manifests/
sleep 15
#check the control-manager pod is running.
$> kubectl get pod -n kube-system  | grep controller-manager
# Verify it is in Running state.
Persistent volume remains in the Delete state, failing to release Review the Provisioner log ($> kubectl logs -n ubiquity deploy/ubiquity-k8s-provisioner) to identify the reason for deletion failure. Use the $ kubectl delete command to delete the volume. Then, contact the storage administrator to remove the persistent volume on the storage system itself.
Communication link between IBM Storage Dynamic Provisioner and other solution elements fails due to Provisioner token expiration IBM Storage Dynamic Provisioner uses a token that in some environments has an expiration time, for example twelve hours. To keep the link alive for an unlimited time, you can use a service-account token without expiration time. You can replace the current token with the service-account token, as follows: 
$> TOKEN=$(kubectl get secret --namespace default $(kubectl get secret
--namespace default | grep service-account | awk '{print $1}') -o yaml | 
grep token: | awk '{print $2}' | base64 -d)

$> kubectl config set-credentials <mycluster.user> --token=${TOKEN}