This morning at this year's Six Five Summit 2021, the head of Intel's DPG, Navin Shenoy, announced its intention to create the “Infrastructure Processing Unit," or the "IPU." He spoke after Intel CEO Pat Gelsinger kicked off the five-day thought leadership and strategy event presented by my company Moor Insights & Strategy and Futurum Research. Shenoy's keynote title was very spicy, entitled, "Why You Have Been Thinking About the Future of the Data Center All Wrong." Provocative? Very. And it is great to see Intel stepping it up and getting more aggressive.
So, what is an IPU? An IPU is designed to offload the main CPU in the datacenter and edge, which gives more predictable and efficient application performance and enables improved virtualization capabilities that CSPs and carriers are seeking. IPUs has become a big industry discussion point as the datacenter, and the edge evolves.
Shenoy put an exclamation mark on "change" with one fact, "more than one-half of all of the data in the world has been created in just the past two years." He added that "probably less than 10% of the world's data has been processed to create any meaningful value or insight". Sure, we have heard similar things before, but this is still mind-blowing when you internalize it for the future, and indicates change and the massive datacenter and edge growth in the future. I find the edge fascinating as it's closer to data creation and provides lower latency to act and respond to that data more efficiently.
On the other hand, pooling resources at the edge is more of a challenge. Micro-services are a way to have your cake and eat it too, as you can run them at the most efficient place but keep a sense of management through orchestration. That orchestration, as I will dig into later, creates challenges that an IPU can help solve.
Many players have jumped into the IPU water as we saw NVIDIA acquire Mellanox and announce its "DPU" and appliance pioneer Marvell and even AWS's Nitro with its Annapurna acquisition. Azure has been in the game with FPGA-configurable Intel SmartNICs accelerating both networking and storage since 2015. Intel is no stranger to the predecessor of the IPU, the SmartNIC, as it is the market share leader.
But what makes Intel’s IPU different? Intel knows the data center and says it’s the only IPU built in collaboration with hyperscale cloud partners. This would mean that Intel would be able to actively innovate and deliver a product that is already addressing real-world problems. Technically, if you equate its SmartNIC as IPU, Intel is already the IPU volume leader in the IPU market with Xeon-D, FPGA and Ethernet components.
So why does the world need another processor? I alluded to it above, but Shenoy cited something at the Six Five Summit event that even shocked me with how large it was: at Google and Facebook, microservices communication overhead is 22% to 80% of CPU cycles. This means between 20% to 78% of that powerful and expensive Xeons are available for applications. That's stunning. I thought this was a single-digit, not a double-digit, performance hit. Offload capabilities alone are a smart reason to leverage an IPU, but it also enables more capabilities. As we have seen with AWS's Nitro layer, an IPU can better virtualize by securely accelerating storage, networking, and security functions. This enables improved multi-tenant capabilities and easier addition and management of new processing types like the array of Intel XPUs and CPUs. With this layer, you don't need to have a dedicated virtualization layer for every CPU, GPU, ML Training ASIC, ML Inference ASIC, FPGA, FPGA SoC, and whatever the industry cooks up next.
Let me end by calling out one element of positioning that grabbed my attention. Shenoy placed the IPU at the same level of importance as the CPU and XPU, which makes this a huge announcement and, to me, shows how important this is. This is an indication of the new Intel, folks.
Intel says in the future, it will roll out more FPGA and ASIC-based IPUs and to expect more details in the run up to its "Intel InnovatiON" event in October. I will be covering those as they roll out.
Disclosure: The Six Five Summit is a vendor-sponsored event held by my company, Moor Insights & Strategy and Futurum Research.