My analyst firm has been researching the nascent ARM-based server market and its technologies before it was cool to do so. ARM-based servers are not shipping in meaningful volumes today, but show promise. Two key points we are always making to folks we advise are to first, don’t make the mistake thinking ARM-based server SoC’s are like smartphone’s just because they use the same instruction set and two, think beyond the SoC – server architecture is a really big deal. This is certainly the case with new features in Applied Micro Circuit’s X-Gene2 server architecture combined with X-Weave. If you are looking for a deep dive, you can download our paper here. Let me start with some background first.
Datacenter 101
To understand the benefits APM’s technologies can bring to the datacenter, you need to understand a bit about classic datacenter architecture, but you don’t need to be an expert. Today’s classic datacenter is comprised of multiple rows of racks filled with processors and their memory, networking, storage, cooling, and power.
Each rack will have multiple modules for compute, a networking module at the top of the rack, sometimes a networking module in the middle of the rack, andsometimes storage in the rack.
The networking switch at the top of a rack (known oddly enough as a “top of rack” or TOR switch) talks to a networking module at the end of a row of racks (“end of row” or EOR switch) and then that module talks to a core network switch.
Classic datacenter drawback- expensive shared memory
There are drawbacks to this classic datacenter architecture, particularly if you choose to hang on to a TOR-based networking topology in the future. For example, if your analytics application needs access to a lot of shared memory, you may need to buy a very expensive 4P or 8P shared memory compute node. Also, many types of applications could be better served with really large amount of memory, which today require greater than 8P configurations, so that’s could be an expensive limiter, particularly with emerging classes of Big Data analytics.
Another way to gain greater access to shared memory inside a rack is to take a page out of HPC (high performance computing) and invest in the most expensive networking gear available, like Infiniband, so your compute modules inside the rack can talk to each other.
Classic datacenter drawback- increased compute density means more networking
Rack compute density, or packing more compute into the same amount of space and power, is fast becoming an important factor in datacenters. In highly compute-dense environments that want to keep a top of rack networking module, to support shared memory and limit the amount of trips to the end of row and core networking modules, datacenters must add more in-rack networking modules, which gets very expensive while increasing network complexity.
More in-rack networking is not only more expensive and complex, but also makes the rack a whole lot harder to cool – networking switches burn power just like processors, and increasing the number of networking cables actually starts to block airflow through racks to reduce cooling efficiencies.
AppliedMicro X-Gene2 SoC and X-Weave Architecture
AppliedMicro has developed a way for datacenters with specific workloads to have their cake and eat it, too. For datacenters who want to keep their current network topology (top of rack network device talking to an end of row network device talking to the core network switch), AppliedMicro created the combination of the X-Gene2 SoC combined with X-Weave Architecture.
X-Weave architecture can enable affordable, compute-dense racks without the need to add in-rack networking, plus it leverages classic TOR-based network architectures, resulting in less cables and a simplified networking management scheme.
The X-Gene2 SoC is designed to enable shared memory across the in-rack compute nodes, meaning a single application can work on incredibly large workloads without the prohibitive cost of Infiniband networking.
This combination can provide very large benefits to analytics and Big Data using algorithms like MapReduce, caching with memcached, distributed storage with Ceph and Swift, and even some classes of HPC applications.
Wrapping Up
AppliedMicro’s X-Gene1 SoC is already differentiated in that it is has the first custom, ARM-based server core to ship and in that it will be a more “brawny” core than any of its predecessors and many SoCs after it. Most of X-Gene’s initial competitors have cores considered “wimpy” from companies like Cavium , or based on off the shelf ARM designs.
X-Gene2 combined with X-Weave goes even further as it extends value outside of the compute module and across the entire rack. The combination has the potential to increase compute density while lowering TCO in certain cloud workloads, eliminating in-rack networking, simplifying network management, and reducing cable count. This combination is a differentiator and am sure OEMs and ODMs will be looking closely at progress.
If you are looking for a technical, deep dive, you can download our paper here.