Connections for control enclosures

Control enclosures have specific connection requirements. Care must be taken to note the orientation of each node canister in the control enclosure.

The control enclosure contains two node canisters. A label on the control enclosure identifies each node canister and power supply unit (PSU). As Figure 1 shows, node canister 1 is on top and node canister 2 is on the bottom. Because the node canisters are inverted, the location of the ports and the numbering of the ports are oriented differently on each node canister. It is important to remember this orientation when you are installing adapters and cables.

Orientation label — Figure 1. Label showing the orientation of the node canisters and PSUs

For example, Figure 2 shows the top node canister. On this canister, the numbering of the PCIe slots and ports goes from right to left. PCIe adapter slot 1 contains a 4-port 16 Gbps Fibre Channel adapter. PCIe slot 2 contains a 2-port 25 Gbps iWarp Ethernet adapter. PCIe slot 3 contains a 4-port 12 Gbps SAS adapter. The onboard Ethernet and USB ports are also shown.

Figure 2. Orientation of ports on node canister 1

Figure 3 shows the bottom node canister. This node canister has the same type and number of adapters installed. However, on the bottom canister, the numbering of the PCI slots and ports goes from left to right.

Figure 3. Orientation of ports on node canister 2

Four 10 Gbps Ethernet ports on each node canister provide system management connections and iSCSI host connectivity. The onboard 10 Gbps Ethernet ports use RJ-45 connections, not optical connections. The 10 Gbps ports operate at 1 Gbps when connected to a 1 Gbps switch. A separate technician port provides access to initialization and service assistant functions. Table 1 describes each port.

Table 1. Summary of onboard Ethernet ports
Onboard Ethernet Port	Speed	Function
1	10 Gbps	Management IP, Service IP, Host I/O
2	10 Gbps	Secondary Management IP, Host I/O
3	10 Gbps	Host I/O
4	10 Gbps	Host I/O
T	1 Gbps	Technician Port - DHCP/DNS for direct attach service management

Make the following connections for control enclosures:

Each control enclosure requires two Ethernet cables to connect it to an Ethernet switch. One cable connects to port 1 of the top node canister, and the other cable connects to port 1 of the bottom node canister. For 10 Gbps ports, the minimum link speed is 1 Gbps. Both Internet Protocol Version 4 (IPv4) and Internet Protocol Version 6 (IPv6) are supported.
Note: For increased redundancy, a second Ethernet connection is supported for each node canister.
To ensure management IP failover, Ethernet port 1 on each node canister must be connected to the same set of subnets. If used, Ethernet port 2 on each node canister must also be connected to the same set of subnets. However, the subnets for Ethernet port 1 do not have to be the same as Ethernet port 2.
With more than one control enclosure in your system, have at least two connections between each node canister in the system unless they are in the same enclosure. These connections can be through:
- One or more Fibre Channel switches
- One or more Ethernet switches that support RDMA (ROCe or iWARP), as described in Configuration details for using RDMA-capable Ethernet ports for node-to-node communications
- Direct connections between Fibre Channel ports on the node canisters, as described in Planning for a direct-attached configuration
To have remote copy relationships with other systems, see Planning for high availability.

Each FlashSystem 7200 node canister has three PCIe interface slots to support adapters. Table 2 provides an overview of the adapters.

The 4-port SAS adapters support expansion enclosures. However, only two ports (1 and 3) are used for SAS connections. In total, the FlashSystem 7200 control enclosures can have up to 20 chain-linked expansion enclosures, with 10 U of expansion enclosures connected per node canister SAS port.

Table 2. Summary of supported adapters
Protocol	Ports	Quantity supported
32 Gbps Fibre Channel	4	0-3
16 Gbps Fibre Channel	4
25 Gbps Ethernet (RoCE)	2
25 Gbps Ethernet (iWARP)	2
12 Gbps SAS Expansion	4, but only two are active for SAS expansion chains.
Notes: Each node canister within the control enclosure (I/O group) must be configured with the same number and type of host interface adapters. For SAS expansion enclosure connections, the SAS interface adapter must be installed in PCIe slot 3.

Each node canister has four onboard 10 Gbps Ethernet ports. A node canister can also support up to three 2-port 25 Gbps Ethernet adapters.

Table 3 shows the fabric types that can be used for communicating between hosts, nodes, and RAID storage systems. These fabric types can be used concurrently.

Table 3. Communications types
Communications type	Host to node	Node to storage system	Node to node
Fibre Channel SAN	Yes	Yes	Yes
iSCSI 10 Gbps Ethernet 25 Gbps Ethernet	Yes	Yes	No
iSER 25 Gbps Ethernet	Yes	No	No
RDMA over Ethernet (ROCe or iWARP)	No	No	Yes

Each of the following adapter features includes standard SFP transceivers:

32 Gbps Fibre Channel adapter
16 Gbps Fibre Channel adapter
25 Gbps iWarp adapter
25 Gbps RoCE adapter

The 2-port 25 Gbps Ethernet adapter for iWARP and the 2-port 25 Gbps Ethernet adapter for RoCE each support iSER host attachment and node to node communication that uses RDMA over Ethernet. However, RoCE and iWARP are not cross-compatible; therefore, it is important to use the adapter that matches the RDMA implementation on your SAN.