Systems Architecture - Parent
Historical, AMD Opteron, Intel QPI, Dell PowerEdge, HP ProLiant, IBM x Series, NUMA, Sandy Bridge,

I need to consolidate 2 separate items on this topic

System Architecture Sandy Bridge EN & EP (Updated)

The first Sandy Bridge processors that came out in 2011 were the dual and quad core models without QPI. Some models have integrated graphics and others do not. There are 16 PCI-E gen 2 lanes plus DMI 2, which is essentially a PCI-E gen 2 x4 port with additional capabilities to support legacy functions. The server variant is the Xeon E3 series.

In early 2012, the Sandy Bridge EN and EP processors came out as the Xeon E5-2400 and Xeon E5-2600 series respectively. Both are offered with 2, 4, 6 or 8 cores. The EN has 3 memory channels, 1 QPI, 24 PCI-E gen 3 lanes and DMI 2. The EP has 4 memory channels, 2 QPI, 40 PCI-E gen 3 lanes and DMI 2. Each memory channels support 3 DIMMs even though the diagrams below only show 2. The DMI 2 can function as PCI-E gen 2 x4 when not used to connect to the a C600 chipset, which supports various low bandwidth IO devices.

The Xeon E5-2400 (EN) supports 2-socket systems at moderately lower cost structure the E5-2600 (EP), and a more powerful single socket system than the Xeon E3 series.


The Xeon E5-2600 (Sandy Bridge EP) fits in above the Xeon 5600 (Westmere-EP) but somewhat below the EX in not supporting glue-less 8-way systems, fewer cores (8 versus 10 in Westmere-EX). The E5-2600 series version of Sandy Bridge EP only supports 2-way systems.


In 2012 Q3, the Xeon E5-4600 series version of Sandy Bridge EP came out for 4-way systems. It appears that all the signals for both th 2600 and 4600 series are the same. It is quite reasonable to expect that 4-way system validation takes longer to complete than 2-way system validation. Perhaps the 4600 required a later stepping?


The current Sandy-Bridge EP systems, both E5-2600 and E5 4600 are configured with 3 DIMMs per memory channel ath most, for 12 DIMMs per socket where the Westmere EX systems employed a memory expander on each memory channel to support 4 DIMMs per channel and 16 DIMMs total.

The 2-way system below shows memory connected directly to the processor. In this case the memory channel signalling rate would follow the current Nehalem/Westmere-EP pattern of 1333MHz with 1 DIMM per channel, 1066 for 2 and 800 for 3.

Presumably the SMB, which expands a single memory channel into 2 could support 4 DIMMs at 1066MHz similar to the Nehalem/Westmere-EX systems, Sandy Bridge EP could suceeds both the Westmere-EP platforms with 2 IOH, and the 2-socket high-memory EX systems. There is no particular reason either system could be configured with or without the SMB. With 8 DIMMs per socket and 16GB DIMMs, this is 128GB per socket, or 256GB per socket with 16 DIMMs each.

In 2011, there was a price premium for a 16GB DIMM over 2 8GB DIMMs, but this closed over the next year as the 32GB DIMM becomes the premium part.

As of 2013-13, the price of memory from Crucial is $210 for 16GB registered ECC DIMM, $1349 for 32GB. With consideration for the cost structure of other parts of the system, storage and software, the practical strategy continues to be: fill the DIMM sockets with the largest economical DIMM, currently 16GB.

Intel Sandy Bridge based systems with QPI 1.1 (Original)

Sandy Bridge is a new micro-architecture suceeding the Nehalem architecture, but sharing the same 32nm process as Westmere, per Intel's Tick-Tock strategy. There are several versions of Sandy-Bridge, including two dual-core versions differing in the on-die graphics having 6 or 12 execution units. There is also a quad-core version without on-die graphics. These three versions are currently available for single socket systems. Later (2012 Q1 perhaps) there will be the Sandy Bridge EN for entry 2-socket systems and Sandy Bridge EP for high-end 2-socket and 4 socket systems. There will not be an EX version capable of support glue-less 8-way until the Ivy Bridge generation(?)

The only Sandy Bridge products currently available are the dual and quad core models without QPI. Some models have integrated graphics and others do not. There are 16 PCI-E gen 2 lanes plus DMI 2, which is essentially a PCI-E gen 2 x4 port with additional capabilities to support legacy functions. The server variant is the Xeon E3 series.

Coming in early 2012(?) are the EN and EP versions, both with up to 8 cores per socket. The EN has 3 memory channels, 1 QPI, 24 PCI-E (gen 3?) lanes and DMI 2. The EP has 4 memory channels, 2 QPI, 40 PCI-E and DMI 2. It might the intent for EN to support 2-socket systems at moderately lower cost structure than Nehalem/Westmere-EP, in not having the IOH?

The Sandy Bridge EP fits in above the current EP but somewhat below the EX in not supporting glue-less 8-way systems, fewer cores (8 versus 10 in Westmere-EX) , and 8 DIMMs per socket versus 16?. The 2-way system below shows memory connected directly to the processor. In this case the memory channel signalling rate would follow the current Nehalem/Westmere-EP pattern of 1333MHz with 1 DIMM per channel, 1066 for 2 and 800 for 3.

Presumably the SMB, which expands a single memory channel into 2 could support 4 DIMMs at 1066MHz similar to the Nehalem/Westmere-EX systems, Sandy Bridge EP could suceeds both the Westmere-EP platforms with 2 IOH, and the 2-socket high-memory EX systems. There is no particular reason either system could be configured with or without the SMB. With 8 DIMMs per socket and 16GB DIMMs, this is 128GB per socket, or 256GB per socket with 16 DIMMs each. There is currently a price premium for a 16GB DIMM over 2 8GB DIMMs, but this should close over the next year as the 32GB DIMM becomes the premium part.

Below shows a 4-way without memory expanded, and 2 DIMMs per channel, even though actual systems have 3 DIMMs per channel.

Below show a 4-way with memory expander. It does not appear that any vendors are offering such a configuration.

DDR3 note: 8 internal banks. 1066MHz DDR has 8 x 133MHz internal banks? Two transfers per cycle of quadrupled clock. Transfer rate is memory clock x 4 (bus multiplier) x 2 (data rate). 8-burst deep prefetch buffer.

Wikipedia provides the information below:

  1. 32 kB data + 32 kB instruction L1 cache (3 clocks)
  2. and 256 kB L2 cache (8 clocks) per core
  3. Shared L3 cache includes the processor graphics (LGA 1155)
  4. 64-byte cache line size
  5. Two load/store operations per cycle for each memory channel

Decoded micro-op cache and enlarged, optimized branch predictor
Improved performance for transcendental mathematics, AES encryption, and SHA-1 hashing
256-bit/cycle ring bus interconnect between cores, graphics, cache and System Agent Domain
Advanced Vector Extensions (AVX) 256-bit instruction set with wider vectors, new extensible syntax, and rich functionality
Intel Quick Sync Video, hardware support for video encoding and decoding
Up to 8 physical cores and 16 logical cores through Hyper-threading, in selected models.

Ivy Bridge is the codename for the yet-to-be released 22 nm die shrink of Sandy Bridge. Ivy Bridge processors will be backwards-compatible with the Sandy Bridge platform. Ivy Bridge processors are expected in March-April 2012.
Expected Ivy Bridge feature improvements from Sandy Bridge:
Intel's tri-gate transistor technology, which will significantly reduce power consumption.
PCI Express 3.0 support.
Next Generation Intel HD Graphics with DirectX 11, OpenGL 3.1, and OpenCL 1.1 support.
The built-in GPU is believed to have up to 16 execution units (EUs), compared to Sandy Bridge's maximum of 12.
Next Generation Intel Quick Sync Video.
The new random number generator and the RdRand instruction, which is codenamed Bull Mountain.
While Ivy Bridge will be compatible with the Cougar Point chipset motherboards associated with Sandy Bridge, Intel will also release a new 7-series Panther Point chipset with Ivy Bridge. This chipset will have integrated USB 3.0.

SocketCoresGraphicsTransistor countDie size
LGA 11554?995 Million216 mm2
26 EUs504 Million131 mm2
212 EUs624 Million149 mm2

The figure below shows a functional representation on the Sany-Bridge EN/EP die layout (not necessarily the actual layout).

Below is a comparison of common and differentiating features for 2-scoket EN and EP.

More details of the Sandy Bridge EN/EP PCH, codenamed Patsburg. Note the 4 x 6Gbps SAS ports, along with 2 6Gbps and 4 3Gbps SATA ports, PCI-E lanes and other.

In the figure below, note that 2 QPI links connect the sockets of an EP platform.