Since launching Neoverse in October of 2018, the Arm architecture has gained a foothold in every major cloud service provider (CSP) around the globe. From AWS to Alibaba cloud, CSPs are not only deploying Arm-based instances, and they are expanding services as customer demand grows.
Further, the strength of Arm's model of delivering architecture and IP that can be leveraged is seen through the implementation of chips in a few different ways. First is through chip vendors like NVIDIA (Grace) and Ampere delivering its Altra and Altra Max CPU used by CSPs and OEMs. The second is through the custom SoC development of CSPs like AWS (Graviton) and Alibaba Cloud (Yitian) for specific needs. Server vendors like Fujitsu design another to drive absolute HPC performance with the A64FX chip. And finally, Neoverse is being used for accelerators and other network functions by Intel (Mt Evans), AMD (Pensando), Marvell, NVIDIA and many others.
It is fair to say that Arm has momentum in the datacenter infrastructure market. And its reach is expanding from the cloud to the enterprise datacenter.
Driving on this momentum, the company recently announced Neoverse V2 – an updated IP portfolio designed to achieve greater performance, efficiency, and security gains. In the following few paragraphs, I'll unpack Neoverse V2 and what this means for enterprise IT.
A quick primer on Neoverse
Before getting into Arm’s announcement, let’s start with a refresher on Neoverse. If you think about the workloads and functions that populate the datacenter, you can carve these into three categories:
- Performance-hungry workloads where every microsecond of latency matters, and every bit of compute resource is required. High-performance computing (HPC), artificial intelligence/machine learning (AI/ML), and deep analytics – are examples where an organization will consume every watt of power to drive the best performance.
- Scale-out workloads are distributed across many nodes, thus requiring fewer computational resources per node. Container-based cloud-native applications comprised of hundreds of services that spin up and down constantly are the classic example of a scale-out workload running in the enterprise.
- Networking functions that move data north, south, east, and west – and secure that data require a lot of throughput but far less computational power.
Each category has unique characteristics that translate into silicon (CPU) designs that should be somewhat bespoke. Where one use case requires powerful cores tied to lots of memory and cache, another use case is better served with lower performing cores but a lot of throughput to move packets of data.
In the past, tech companies would address these needs by taking a CPU and simply defeaturing capabilities. Same silicon. Same architecture. Just some things "fused off and a different name. But this doesn't entirely solve the needs of the customer looking to achieve optimal performance and power efficiency.
Arm figured this out some time ago and began the work of creating somewhat bespoke architectures under the Neoverse brand. In Neoverse, three distinct designs were created to meet these needs:
Customers can engage with Arm in two different ways in adopting Neoverse. The first is an IP license, whereby a customer would take a design "as is" and develop CPUs based on this design. This enables a customer to quickly turn around a CPU to meet its (or market) needs.
If you apply the 80/20 rule (or maybe 90/10 rule) to CPUs – think of V-Series, N-Series, and E-Series as being highly tuned for each class of performance. Some customers want to eek that extra 10 or 20% optimization, and for these customers, Arm offers its customers an architectural license. This allows customers to make microarchitectural changes, finely tuning for a specific workload or environment. This is often what is found in the market as ecosystem partners like Ampere try to create more differentiation with AmpereOne, or a company like Fujitsu looks to make the highest performing HPC installation.
This approach by Arm has proven to be successful in expanding its footprint in the market. You can see in the above graphic that Arm's ecosystem has created silicon that leads in various deployments and use cases.
If you're thinking, "well, all of this 'customization' is creating compatibility problems," think again. Because of Arm's SystemReady program, software developed for the Arm architecture just runs. Arm has set standards around base architecture and base boot that assure compatibility up the stack – from OS to applications.
So, Neoverse has opened up a few avenues for Arm in the datacenter. First, it has enabled its silicon ecosystem partners to bring products to market, allowing a performance per watt value that can lead to significant savings for CSPs and hyperscalers. Secondly, it enables its silicon partners to build very finely tuned CPUs for specific workloads, environments, or even customers.
Wonder how successful Neoverse has been? Consider this – 48 of AWS' top 50 customers are utilizing Graviton-based instances. I would classify this as a “win.”
What was announced
In its announcement, Arm unveiled the second generation of its Neoverse N and E series IP, aptly named V2 and E2 (N2 was previously launched).
My focus on Arm's announcement was more around what the company was doing with V2, as this plays squarely in the enterprise compute space (NVIDIA Grace is based on V2).
V2 is what one would think – Arm doubling down on the performance for a range of workloads that power the modern enterprise. And it does this through better-performing cores, more cache, and instructions that accelerate the performance of these workloads that run in the cloud and the datacenter.
Arm is pretty confident as it pertains to V2 performance. The company claims that V2 integer performance will best that of its x86 competitors (so much for Arm only being a power efficiency play). Moor Insights & Strategy Chief Analyst, Patrick Moorhead thought that this supremacy messaging was very aggressive for Arm and a first. Combining integer performance with a large private and system cache and extensions like bfloat16 and MatMul should make this a performance beast. It will be interesting to see how the performance compares to Ampere’s AmpereOne SoC, sampling today, with its custom cores and other IPs.
As you can see, Arm has a relatively aggressive Neoverse roadmap. This is a little less aggressive than what the company was presenting early in the life of Neoverse when it was considering a (close to) annual cadence of IP. This pacing enables ecosystem partners to stand up solutions in the market for a little longer.
I do like the approach the company is taking with Neoverse. As the challenger to x86, it has to provide enough of a differentiated experience to win market share. That means delivering almost gaudy performance numbers and overall value that almost forces enterprise organizations to at least evaluate these cloud instances.
Does Arm threaten x86 in the datacenter?
Anybody that has read my Arm ramblings knows that I've been bullish on what the company is doing and the long-term prospects of the Arm-powered datacenter. I'm no less bullish today than when Neoverse was first announced. Any market where choice is given leads to diversification of suppliers. In the case of the datacenter market, the adoption of cloud and open source have made choice possible (choice beyond two x86 suppliers, that is).
The most significant indicator that Arm will succeed in the on-prem enterprise datacenter comes from HPE. Its announcement of the ProLiant RL300 at Discover (based on Ampere) tells me that enterprise customers are asking for Arm-based servers. Can Dell be right around the corner? I wouldn’t be surprised.
Let's be clear – I am not saying Arm will replace x86 in the datacenter. I am simply saying that the x86 players should make room for Arm at the dinner table.
Can we finally say that Arm in the datacenter is not a trend? When the company launched Neoverse in 2018, much work had to be done to make Neoverse a legitimate player in the market. The silicon, hardware, and software ecosystems had to be expanded; standards around interoperability had to be hardened and enforced, and customers had to start using Arm to accelerate the innovation engine. It is fair to say Arm has achieved success.
Now that Neoverse V2 is fully announced and we will start to see silicon from NVIDIA and others, I expect Arm's momentum to gain steam and the enterprise to increasingly see Arm-based platforms as ideal for a number of their workloads – in the cloud or on-prem. Hmmm… maybe HPE is on to something.
Note: this analysis contains contributions from Patrick Moorhead, Moor Insights & Strategy Chief Analyst.