Enterprise IT can learn from the hyperscalers and cloud service providers (CSPs) that deploy infrastructure at a massive scale. These organizations that monetize IT assets have done the complex modeling and math that result in an environment designed for the best performance at the lowest cost.
To feed these complex models, organizations that deploy servers at scale tend to measure performance and power at the rack level. Meaning, how much work can a rack of servers perform and at what power envelope. And for cloud-native workloads that tend to be lightweight and consistent in performance, work is measured in cores.
With the above said, every major CSP has deployed Arm-based servers to support general-purpose customer workloads. And given the design and performance of Ampere's Altra Max CPU, it's no surprise that it is the CPU of choice of every CSP that hasn't designed its own Arm chip.
In the following few sections, I will look at the evolution of the datacenter processor market and where Ampere delivers a compelling solution that delivers on performance, cost, and sustainability. I’ll also dig into the rack-level measurement of power and performance and why I think is the right metric.
The CPU market – a brief history
The enterprise datacenter has long been an x86-dominated space, where two major players have driven an innovation cycle for software developers and IT professionals supporting the needs of businesses. Mission and business-critical applications developed in a three-tiered model were deployed at scale across enterprise datacenters. As compute requirements grew, IT organizations added more servers and storage.
The above operating model served businesses quite well for a very long time. And when virtualization technology began to find adoption in the enterprise, operational efficiencies were further realized as deploying and managing virtual servers enabled faster time to operation.
As cloud computing evolved from nascency to a critical element of the enterprise datacenter, cloud-native architectures became a preferred method of designing and deploying applications for performance, mobility, and security reasons. And cloud-native was reserved for more than just new application development; many organizations are refactoring and rearchitecting those traditionally designed mission and business-critical applications as part of digital transformation efforts.
Cloud-native workloads are designed to be lightweight and mobile. The concept is that an application is broken down into many different microservices that only run when necessary and can be shared with other applications. And these services move from server to server, from on-prem to off-prem, and from cloud to cloud.
I don't write this to try and educate a reader who already understands the concept of cloud-native, but rather to map these needs to the evolution of CPUs in the cloud. These many lightweight services continuously spinning up and spinning down benefit from a server powered by a CPU with many single-threaded cores to support these performance characteristics. And this, in essence, is the Arm CPU. When AWS first introduced its homegrown Graviton CPU, it found quick success in supporting these general-purpose cloud workloads exploding across businesses of all sizes and types. And to an analyst (and industry), the success of Graviton proved the viability of Arm in the cloud (and hyperscale) datacenter as it drove a level of compute unmatched density, along with a reduced power envelope that resulted in real and measurable cost savings.
While AWS worked toward the in-house development of Graviton, a few ex-Intel executives and semiconductor veterans were quietly working on designing and delivering a commercial Arm CPU that could support everybody else. Enter Ampere Computing. Based on Arm's Neoverse Architecture, Altra Max is a CPU that marries the performance, reliability, and consistency of x86 with the power savings associated with Arm.
Ampere positions Altra Max as the first cloud-native CPU in the market. I like this positioning as it accurately describes where I see the CPU playing – with CSPs and hyperscale environments. These are organizations that deploy hundreds of thousands of servers and tend to measure performance and power at the rack level.
Ampere's positioning has resonated with the CSPs. Following AWS’s success with Graviton, the remaining major cloud providers have adopted Altra Max and offered Arm-based instances as an economical alternative to support cloud-native workloads. Oracle (an investor in Ampere) launched its Oracle Ampere A1 Compute instance first, with availability in all regions. Microsoft Azure followed suit with three instance types – the Dpsv5, DPlsv5, and Epsv5 series (very creative naming, might I add). And Google Cloud Platform (GCP) brought up the rear with the launch of the Tau T2A instances.
There are two ways to look at this adoption of Ampere in the cloud. The first is that AWS put competitive pressure on the market to launch an Arm-based instance, and Ampere had the best story. The second is that the other CSPs saw the popularity of Graviton and the internal cost savings achieved by deploying Arm at scale, and Ampere delivered a competitive (and compelling) commercial offering. Knowing the CSP market, I'm putting all my money into the latter as there is a tangible and measurable advantage regarding cost and manageability. And these costs are passed on to the customer base.
Digging deeper into the Ampere efficiency at the rack story
Ampere’s value proposition is about compute density and power efficiency – per rack. Meaning, how many compute cores can fit in a 42-rack unit (RU) form factor, and how much power does that datacenter rack require? When looking at performance and power across the datacenter – especially for a workload that has consistent performance like cloud native – this rack level measurement is the best measurement. It allows a datacenter architect to quickly extrapolate compute capacity relative to needs at scale. It also enables a very quick and accurate measure of power consumption and power efficiency.
Rack-level measurement of both performance and power is also important as it speaks to virtually every datacenter architect's challenge – how to drive maximum efficiency. If I, as a datacenter architect, can support my cloud-native workloads in 3,300 racks instead of 10,000 racks – that would be substantial savings in datacenter floor space (and power, management, etc.). The first obvious benefit is the reduced cost of 1/3 of the space required for my server farm. The second obvious benefit is the reduced power bill I pay every month – especially significant in today's soaring energy costs. And the third is the less obvious (but substantial) cost savings achieved through deploying, monitoring, and managing fewer servers.
These savings are significant. And this is precisely what Ampere is claiming.
It is important to note that the above compares Ampere to an N-1 configuration of its competition. AMD's 4th Gen processor supports up to 96 cores (soon to be 128 cores) per CPU. Likewise, Intel’s 4th Gen Xeon processor supports up to 56 cores. With this said, the 3rd generation processors are seen in the market.
I will also note that Moor Insights & Strategy did not perform the testing nor perform a detailed methodological analysis, but the testing passes our first pass analysis.
Ampere’s claims are impressive. Almost too good to be true. But, as I mentioned, the numbers add up. And the company is putting a lot of noise behind its campaign to point out these advantages to the marketplace.
Let’s not forget about sustainability
The other area where Ampere is making noise is the obvious sustainability slant. With carbon footprint reduction being a top priority for virtually every enterprise (and indeed every CSP) – a 2.8x reduction in power consumption adds up at scale.
There are two ways to look at sustainability – the idealistic "save mother earth" view and the pragmatic "cost savings" view. Each is legitimate and important to a company. And again, Ampere is making very bold claims. A cloud datacenter supporting cloud-native workloads with Ampere can go a long way to achieving sustainability goals. Further, the cost savings can, in power alone, be astronomical.
Closing thoughts - is this all too good to be true?
Great performance combined with reduced carbon footprint and significant cost savings sounds too good to be true. And I'm sure that the x86 players will have a response to the counter Ampere's claims. Isn't that the way these campaigns work?
There is merit to Ampere's claims. We no longer live in a datacenter world where x86 is the only option, and deploying Arm is no longer met with a "yeah, but" mindset, particularly in the cloud. The ecosystem no longer strongly distinguishes between the architectures, and in fact, the cloud-native software ecosystem has enthusiastically supported Arm as a supporting platform. And yes, power a the rack is the right metric.
Ampere has carved out its place in the datacenter, and the company has undoubtedly proven to be the leading chipmaker to deliver for the cloud. And this competitive pressure is good for all—especially the customer.