Home, Cost-Based Optimizer, Benchmarks, Server Systems, Systems Architecture, Processors, Storage,
  Storage Overview, System View of Storage, SQL Server View of Storage, File Layout,
  PCI-E, SAS, FC, HDD, SSD Technology, RAID Controllers, Direct-Attach,
  SAN, Dell MD3200, EMC AX4, CX4, VNX, V-Max, HP P2000, EVA, P9000/VSP, Hitachi AMS
  SSD products: SATA SSDs, PCI-E SSDs , Fusion iO , other SSD

EMC CLARiiON CX4

Some details of the EMC CLARiiON CX4 line is show below. Each Clariion system is comprised of two Service Processors (SP). The SP is simply an Intel Core 2 architecture server system.

 CX4 120CX4 240CX4 480CX4 960
Processor
per SP
1 dual-core 1.2GHz1 dual-core 1.6GHz1 dual-core 2.2GHz2 quad-core 2.33GHz
Memory per SP3GB4GB8GB16GB
Max Cache600MB1.264GB/s4.5GB10.764GB
Front-end FC ports (Base/Max)4/84/128/168/24
Back-end FC ports (Base/Max) 2/24/48/88/16

The Clariion CX4 line came out in 2008. I do have some criticism on the choice of processors for each model. First, the Intel Processor price list does not even show a 1.2GHz model in the Xeon 5100 or 3000 series. This means EMC asked Intel for a special crippled version of the Core 2 processor. The Intel Xeon processors start at 1.6GHz for a dual-core with a price of $167. The quad-core X3220 2.4GHz has price of only $198, so why in the world does EMC use the 1.2GHz dual-core at the low-end? Sure, basic storage server operations does not require a huge amount of compute cycles, but all the fancy features (that really should not be used in a critical SQL system) the SAN vendors advocate do use CPU-cycles. So when the features are used, performance tanks on the crippled CPU used in the expensive SAN storage system.

Now what we really want at the mid-range 480 level is having two processor sockets populated, as this will let the system use the full memory bandwidth of the Intel 5000 (or 5400) chipset, with 4 FB-DIMM memory channels. Yes, the 960 does have two quad-core processors, but I am inclined to think that the 960 (SP pair) with up to 16 back-end FC port might be over-reaching for the capability of the Intel 5000P chipset. If the CX4 960 in fact uses the 5400 chipset, then this might be a good configuration. But I have seen no documentation that the 960 can drive 5.6GB/sec. The quad-core E5405 2.00GHz processor is a mere $209 each, and the E5410 2.33GHz used in the high-end 960 model is $256 each. In late 2008, the dual-core E5205 1.86GHz was the same price as the quad-core E5405 2.0GHz. The Dell PowerEdge 2900 with 2 E5405 quad-core processors and 16GB was $2300.

This is less than the cost of each of the quad-port FC adapters, of which there are two in each SP of the 480. Consider also the cost of the 480 and 960 base systems, and that the 16GB memory in each 960 SP has a cost of around $800 each. Why not just fill the 16 DIMM sockets allowed by the 5000P chipset with 4GB DIMMs at about $3200 for 64GB per SP, unless it is because a large cache on a storage controller is really not that useful?

If it seems that I am being highly critical of the EMC Clariion line, let me say now that the other mid-range SAN storage system use even more pathetic processors. So, the Clariion CX4 is probably the best of the mid-range systems.

Below is another summary of the Clariion CX features. The number of FC ports is total, front and back, even though it is more useful to list the two separately.

SAS

Below is a rearview of the CX4-960. Each SP has a CPU complex, IO carrier, and annex IO carrier. Each IO carrier can accommodate two IO modules, and each FC IO module has 2 FC ports on the front-end (host-side) and 2 on the back-end. There are other IO module options as well.

SAS

Another view of the CX4-960 with IO modules annotated.

SAS

Below is a representation of the CX4-960 SP. Presumably the operating system (Windows XP) changed from 32-bit to 64-bit between the CX3 and CX4 generations.

SAS

Below is the older CX3-80. The x8 CMI is (Coherent Memory Interface?, some document refer to CMI as Clariion Messaging Interface. Below, it was originally Configuration Manager Interface.) to maintain write cache on both SP, in case of failure. Another (older?) EMC document said this was over FC, which would result is serious write throughput limitations. The x8 designations suggest this is PCI-E. Perhaps there is a 3rd-party PCI-E bridge chips for the purpose of connecting two separate systems, but not for ordinary IO expansion?

Some documents describe CMI traffic going over FC, probably in the 1Gb and 2Gb period. Newer documents refer to CMI with the x4 or x8 qualifier, suggesting that traffic goes directly over the PCI-E bus? The CLARiiON CX4 series DC Power Systems whitepaper states: A PCI Express x4 CMI channel between the SPs is used for communication and messaging between the SPs and to mirror data that is written to the storage system's write cache for the CX4-480.

The paper Introduction to the EMC CLARiiON CX4 Series Featuring UltraFlex Technology say the high-end CX4-960 has a PCI Express x8 CMI channel. Also the previous generation CX3-80 is shown below with the x8 CMI qualifier.

SAS

The following excerpts from Steve Todd weblog on CLARiiON Write Cache Design:
1) Mirror, The first decision we made was to mirror any write request to CLARiiON's peer storage processor. This second processor was not only redundant, but it was also active. Once the new application data had been written into local memory and safely sent to the peer, we could notify the application that the data was safely stored.
2) The Vault, We decided to dedicate private space on the first five disk drives in the CLARiiON as an "emergency de-stage" area for the cache. The first four drives would be a direct mapping of the cache memory onto the disks, and the fifth drive would contain RAID-3 style parity information. So if and when a power-failure occured, it was only necessary to hold up the storage processors (and the vault disks) long enough to safely store the customer's data.
CMI, Well, it required significant modification to a piece of software known as "CMI". This acronym has undergone identity name change over the years but it originally stood for "Configuration Manager Interface". It was a messaging system between the two storage processors to swap information about the configuration, and to request permission to perform new configuration operations. This messaging system was modified to support data transfer from one physical location to another.

Below, a CX4 architecture diagram.

SAS

There is an EMC slidedeck, CLARiiON Top 10 Best Practice Hints for Performance at the end of this Swedish? slidedeck EMC CLARiiON Family.

Also see EMC CLARiiON Best Practices for Performance and Availability: Release 29.0 Firmware Update and more recent firmware updates as available.