Checking the log files

The IBM® Spectrum Control Base Edition maintains log files that record different types of events.

You can find the following log files in the /var/log/sc/ directory:
  • events.log – Records Spectrum Control Base events according to their type: Info, Error, or Warning. The event logging is compatible with the Rsyslog application, an open source utility for forwarding log messages over IP networks (for more information, see the Rsyslog website). The following example shows different events that might be recorded:
    IBMSC-0001, INFO, "User {user_name} has logged in."
IBMSC-0002, INFO, "User {user_name} has logged out."  
IBMSC-0003, WARNING, "User {user_name} login attempt failed."
IBMSC-0004, INFO, "IBM Spectrum Control local user account {user_name} was created."
IBMSC-0005, INFO, "IBM Spectrum Control local user account {user_name} was deleted."
IBMSC-0006, INFO, "IBM Spectrum Control local user account {user_name} password was reset."
IBMSC-0007, INFO, "Storage credentials were set for user {user_name}."
IBMSC-0008, ERROR, "Storage credentials for user {user_name} were disabled. Reason: {reason}."
IBMSC-0009, INFO, "Storage array identified as {identifier} with IP address {ip_address} has been added."
IBMSC-0010, WARNING, "Storage array {identifier} was removed."
IBMSC-0011, WARNING, "Storage array {identifier} was modified. Its new IP address is {ip_address}."
IBMSC-0014, INFO, "vCenter server with IP address {ip_address} was added by user {user_name}."
IBMSC-0015, WARNING, "vCenter server with IP address {ip_address} was removed."
IBMSC-0016, WARNING, "Credentials for vCenter server with IP address {ip_address} were updated by user {user_name}."
IBMSC-0017, INFO, "Storage pool {pool_name} on storage array {identifier} was attached to vCenter server with IP address
{ip_address}."
IBMSC-0018, WARNING, "Storage pool {pool_name} on storage array {identifier} was detached from vCenter server with IP address
{ip_address}."
IBMSC-0019, ERROR, "Failed to connect to storage array {identifier}. Reason: {reason}."
IBMSC-0020, INFO, "Information retrieval from storage array {identifier} was completed."
IBMSC-0021, ERROR, "Failed to retrieve information from storage array {identifier}. Reason: {reason}."
IBMSC-0022, INFO, "LDAP authentication was enabled."
IBMSC-0023, INFO, "LDAP authentication was disabled."
IBMSC-0024, INFO, "Completed the vSphere Web Client extension task {task_name} with the following parameters: {parameter_list}."
IBMSC-0025, ERROR, "Failed to complete the vSphere Web Client extension task {task_name} with the following parameters
{parameter_list}.
Reason: {reason}."
  • hsgsvr.log – Records events regarding monitoring and operations on storage systems and volumes.
  • vasa1.log – Records events regarding the communication between Spectrum Control Base and the connected vCenter servers that utilize VASA 1.0 functions.
  • vasa2.log – Records events regarding the communication between Spectrum Control Base and the connected vCenter servers that utilize VASA 2.0 functions. In addition, several events related to the VASA 1.0 activity may be recorded in the vasa2.log file as well.
  • vwc.log – Records events regarding the communication between Spectrum Control Base and the vSphere Web Client Server on which the IBM Storage Enhancements are installed.
  • celery.log – Records events regarding the Celery services on the Linux host on which Spectrum Control Base is installed.
  • django.log – Records events regarding the Django service on the Linux host on which Spectrum Control Base is installed.
  • vco.log – Records events regarding operation of the vSphere Orchestrator, complementing information stored in the hsgsvr.log file. The vco.log file can be accessed via the Log tab of the vCO plug-in interface. If the Inventory folder within the 'IBM Storage' context is empty and the following message is stored in the vco.log:
    [SCRepository] com.sun.jersey.api.client.ClientHandlerException: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No name matching sc8.ps.xiv.ibm.com found    ,
    replace the default Spectrum Control Base SSL certificate and key files as described in Managing server certificates.
  • vcops.log – Records events regarding the communication between Spectrum Control Base and the connected vROps servers.
  • traffic.log – Records XCLI events. This log file is reserved for debug purposes.
  • To collect and display logs, related to the different components of IBM Storage Enabler for Containers, use the following Kubernetes commands:
    • Log collection – ./ubiquity_cli.sh -a collect_logs. The logs are kept in a folder, named as ./ubiquity_collect_logs_MM-DD-YYYY-h:m:s. The folder is placed in the directory, from which the log collection command was run.
    • IBM Storage Enabler for Containers – $> kubectl logs -n ubiquity deploy/ubiquity.
    • IBM Storage Enabler for Containers database – $> kubectl logs -n ubiquity deploy/ubiquity-db.
    • IBM Storage Kubernetes Dynamic Provisioner – $> kubectl logs -n ubiquity deploy/ubiquity-k8s-provisioner.
    • IBM Storage Kubernetes FlexVolume for a pod – $> kubectl logs -n ubiquity pod ubiquity-k8s-flex<pod_ID>. In addition, events for all pods on a specific Kubernetes node are recorded in the ubiquity-k8s-flex.log file. You can view this file in the following directory: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ibm~ubiquity-k8s-flex.
    • Controller-manager:
      • Static pod – kubectl get pods -n kube-system to display the master pod name. Then, kubectl logs -n kube-system pod_name to check the logs.
      • Non-static pod – journalctl to display the system journal. Then, search for the lines that have controller-manager entries.

You can retrieve and save the current Spectrum Control Base log files in a compressed TAR archive file by using the Collect Log option on the Settings menu of the Spectrum Control Base GUI. This option allows you to save different instances of the log files at different times.

Figure 1. Controller GUI – Collect Log option
This image shows the Collect Log option on the Settings menu.

Additional information can be retrieved from other resources. For example, the Failed to establish connection <hostname> message stored in the /var/log/vvold.log file on an ESXi host indicates a loss of IP connectivity between the Spectrum Control Base server and the ESXi host that uses a VVol. To resolve this issue, re-establish the connection with all ESXi host, consuming VVols via the Spectrum Control Base web login port.

Also, vSphere components generate assorted logs that contain additional information about activities in vSphere environment. This information might help you resolve technical issues. The log files locations are detailed in the following table.

Table 1. VMware log file locations
VMware component Operating system GUI
Windows Appliance
vSphere Web Client C:\ProgramData\VMware\vCenterServer\logs\vsphere-client\logs /var/log/vmware/vsphere-client/logs –
vRo 6.x:
  • <instal_dir>\VMware\Orchestrator\app-server\logs, <instal_dir>\VMware\Orchestrator\apps
  • <instal_dir>\VMware\Orchestrator\configuration\logs
 6.x, 7.x: /var/log/vco
  • 6.x: Logs > Generate log report
  • 7.x: Control center > Export Logs
vROps – 6.0.1:
  • storage/vcops/log/adapters/VinAdapter/VinAdapter_<id>.log
  • /storage/vcops/log/collector.log
  • 6.0.1: Administration > Support > Logs
  • 6.5: Administration > Support Bundles > + > Create log bundle > Download the bundle