Removing and replacing a midplane assembly

A trained service provider can use this procedure to replace a faulty control enclosure midplane with a new one received from stock. Ensure that your control enclosure midplane assembly is replaced only by a trained service provider.

About this task

DANGER
When working on or around the system, observe the following precautions:

Electrical voltage and current from power, telephone, and communication cables are hazardous. To avoid a shock hazard:

  • If IBM® supplied a power cord(s), connect power to this unit only with the IBM provided power cord. Do not use the IBM provided power cord for any other product.
  • Do not open or service any power supply assembly.
  • Do not connect or disconnect any cables or perform installation, maintenance, or reconfiguration of this product during an electrical storm.
  • The product might be equipped with multiple power cords. To remove all hazardous voltages, disconnect all power cords.
  • Connect all power cords to a properly wired and grounded electrical outlet. Ensure that the outlet supplies proper voltage and phase rotation according to the system rating plate.
  • Connect any equipment that will be attached to this product to properly wired outlets.
  • When possible, use one hand only to connect or disconnect signal cables.
  • Never turn on any equipment when there is evidence of fire, water, or structural damage.
  • Disconnect the attached power cords, telecommunications systems, networks, and modems before you open the device covers, unless instructed otherwise in the installation and configuration procedures.
  • Connect and disconnect cables as described in the following procedures when installing, moving, or opening covers on this product or attached devices.
    To disconnect:
    1. Turn off everything (unless instructed otherwise).
    2. Remove the power cords from the outlets.
    3. Remove the signal cables from the connectors.
    4. Remove all cables from the devices.
    To connect:
    1. Turn off everything (unless instructed otherwise).
    2. Attach all cables to the devices.
    3. Attach the signal cables to the connectors.
    4. Attach the power cords to the outlets.
    5. Turn on the devices.
  • Sharp edges, corners and joints might be present in and around the system. Use care when handling equipment to avoid cuts, scrapes and pinching. (D005)
Attention:
  • The control enclosure midplane assembly must be replaced only by a trained service provider. Complete this procedure only if instructed to do so by a service action or the IBM Support center.
  • If you have a single control enclosure, this procedure requires that you shut down your system to replace the control enclosure midplane assembly. If you have more than one control enclosure, part of the system can continue to run. However, you lose access to the volumes that are on the affected I/O group. You also lose access to any volumes that are in other I/O groups that depend on the drives that are in the affected I/O group. If the system is still doing I/O requests in all the I/O groups, schedule the replacement during a maintenance period or other time when the I/O can be stopped.
  • When you replace hardware components in the back of the enclosure, ensure that you do not inadvertently disturb or remove cables that you are not instructed to remove.
  • Ensure that you are aware of procedures for handling static-sensitive devices before you remove the enclosure.

Procedure

  1. Log in to the service assistant on one of the node canisters in the control enclosure. In the Node Details panel, select the Location tab. Note the following values for the enclosure in Table 1. This information might be needed later.
    Table 1. Recording control enclosure information
    Item Value
    WWNN 1  
    WWNN 2  
    Machine type and model  
    Serial number  
  2. If the control enclosure is still active, stop host I/O, FlashCopy®, Global Mirror, or Metro Mirror activities on all the volumes that depend on the control enclosure.

    This step applies to all I/O group volumes that are managed by this control enclosure. It also applies to any volumes in other I/O groups that depend on the drives in the affected I/O group.

  3. Complete Procedure: Powering off a node canister for the node canister that requires the midplane assembly replacement.

Removing the midplane assembly

  1. Disconnect both power cables from the rear of the control enclosure.
  2. Write down the port and cable connections before you disconnect all cables from the ports rear of the control enclosure. Use the following tables to record the information for each node canister.
    Notes:
    1. Depending on your system configuration, some PCIe adapter ports and Ethernet ports might not be connected.
    2. The orientation, slot numbers, and port numbers of node canister 1 (top canister) differ from node canister 2 (bottom canister).

    #fs9100_rplc_cntrl_enc_midplane__node1 summarizes the possible PCIe slot and port configuration of node canister 1.

    Table 2. Summary of node canister 1 PCIe slots and port connections
    PCIe slot 2 PCIe slot 1
    Port 4 Port 3 Port 2 Port 1 Port 4 Port 3 Port 2 Port 1
                   
                   
    Table 3 summarizes the Ethernet ports and possible connections for node canister 1.
    Table 3. Summary of node canister 1 Ethernet ports
    Port 4 Port 3 Port 2 Port 1
           
           

    #fs9100_rplc_cntrl_enc_midplane__node2 summarizes the possible PCIe slot and port number configuration of node canister 2 (the bottom node canister).

    Table 4 summarizes the Ethernet ports and possible connections for node canister 2.
    Table 4. Summary of node canister 2 Ethernet ports
    Port 1 Port 2 Port 3 Port 4
           
           
  3. Carefully remove each drive or drive blank and label it with the drive slot from which it was removed, as described in Removing and replacing a drive or Removing and replacing a drive blank.
  4. Remove the two power supplies from the control enclosure, as described in Removing and replacing a power supply unit.
  5. Remove the two power interposers from the control enclosure, as described in Removing and replacing a power interposer.
  6. Remove the node canisters from the control enclosure, as described in Reseating a node canister in the control enclosure. Label them to indicate what canister came from each canister slot.
  7. Remove the end caps from the control enclosure, as described in Removing and replacing the enclosure end caps.
  8. Remove the screws from the front of the control enclosure to free the enclosure from the rack.
    Figure 1 shows the location of the screws. This figure shows two black M5 screws and two silver screws in the rack. However, some racks might use four black M5 screws.
    Figure 1. Location of the screws on the front of the enclosure
    Location of the screws on the front of the enclosure
  9. Carefully slide the control enclosure from the rack.
  10. Place the control enclosure on a work surface, so that the underside of the enclosure faces upward, and the enclosure front is facing toward you.
  11. Remove the nine screws from under the front of the enclosure (see Figure 2).
    Figure 2. Location of the screws on the underside of the control enclosure
    Location of the screws on the bottom of the enclosure
  12. Turn the control enclosure over so that the top of the enclosure is facing upward, and the front is facing towards you.
  13. Remove the five screws from the top of the enclosure.
  14. Remove the four screws on the right side that secure the midplane assembly to the enclosure.
  15. Remove the four screws on the left side that secure the midplane assembly to the enclosure.
  16. Remove the midplane assembly from the chassis by rotating up the midplane assembly to about 45°. Then, withdraw the midplane assembly from the front of the enclosure.
    Figure 3 shows the front of the enclosure with the midplane assembly removed; the screw locations are also shown.
    Figure 3. Midplane assembly removed from the control enclosure
    Control enclosure after the midplane assembly was removed.

Replacing the midplane assembly

  1. Unpack the replacement midplane assembly. Locate the pack of replacement screws that comes in the package. The package includes 24 screws, but only 22 screws (shown in Figure 4) are used to secure the enclosure chassis.
    Figure 4. Midplane assembly screws
    Control enclosure after the midplane assembly was removed.
  2. Hold the midplane assembly the correct way up; ensure the drive bays face toward you.
  3. Insert the midplane assembly into the front of the enclosure at 45° angle. Then, rotate it down into the front of the enclosure so that the screw holes in the midplane assembly align with the screw holes in the enclosure.
  4. Secure the midplane assembly to the enclosure chassis on the right and left sides by using four screws each side to replace the screws that were removed in steps 17 and 18.
  5. Secure the five screws that were removed in step 16 to the top of the enclosure chassis.
  6. Turn over the bottom of the enclosure to face upward; then, insert the nine screws on the bottom of the enclosure that were removed in step 14.
  7. Reinstall the control enclosure in the rack cabinet, securing it with the screws that were removed at step 11.
  8. Reinstall the end caps at the front of the enclosure, as described in Removing and replacing the enclosure end caps.
  9. Reinstall the drives or drive blanks that were removed in step 6. Ensure that each drive is inserted into the same slot from which it was removed.
  10. Reinstall the node canisters into the same canister slots from which they were removed, as described in Reseating a node canister in the control enclosure.
  11. Reinstall the power interposers, as described in Removing and replacing a power interposer.
  12. Reinstall the two power supplies, as described in Removing and replacing a power supply unit.
  13. Reconnect the data cables to the same connectors at the rear of the control enclosure. Use the data that you collected in step 5.
  14. Reconnect power to the control enclosure.

    The node canisters restart. The yellow fault LED indicators are on because the new enclosure was not set with the identity of the old enclosure. The node canisters log node error 504 to report that they are in the wrong location. In the system Event log, the error code is 1192.

  15. Use an ssh client to log in as the superuser to the service IP address of either node canister. Then, access the service assistant CLI.
    Note: If you replaced a node canister, log in to the node canister that was not replaced.
  16. Enter the satask chvpd -replacemidplane command.

    If the command is successful, the node canister modifies the VPD data. You are disconnected from the CLI as both node canisters reboot.

    If the satask chvpd -replacemidplane command fails, complete the following steps:

    1. Connect to the Service Assistant Tool of the node canister. Go to http://service_ip/service or connect to the technician port of the node canister.

      If you replaced a node canister at the same time as the midplane assembly, connect to the node canister that was not replaced.

      If you cannot access the service assistant, see Problem: Cannot connect to the service assistant.

    2. In the Service Assistant tool, use the Configure Enclosure panel.
    3. Select to use the node copy of the data that you recorded in step 1 (WWNN 1, WWNN 2, machine type and model, and serial number).

      If the node copy for any of the values is not available, select the Specify option for each missing value. Then, enter the value that you recorded at step 1 .

    4. After you specify all values in the Enclosure Information panel, click Modify.

      The node canisters restart. When the restart finishes, the system comes online with both node canisters online.

      Important:
      • Do not reset the system ID.
      • After the replacement midplane is modified, it cannot be used as a replacement part for a different enclosure. Do not return a modified midplane assembly to FRU stock.
  17. Wait for the green status LED of both node canisters to light and indicate that the node canisters are active in the system.

Note: Customers can complete the following tasks after the new midplane is installed and the control enclosure is operating again.

  1. Use the management GUI to check the status of all volumes and physical storage to ensure that the system is operating normally.
  2. Go to Monitoring > Events to check the Event Log for other events or errors.
  3. Restart the host application and any FlashCopy, Global Mirror, or Metro Mirror activities that were stopped.