There’s been a lot of news lately on the AI chip front, so I wanted to share a short synopsis of what has been happening for anyone who may be distracted by the holidays. I will flesh these stories out further before my annual Cambrian AI Explosion blog in January here on Forbes.
Let’s start with the big news. Amazon AWS (AMZN) made two significant AI announcements on December 1st at the annual AWS re:Invent conference. First, Andy Jassy, AWS head, announced that the cloud leader would offer Intel’s Gaudi training chip in the elastic cloud. The AWS deployment is the first traction we have seen for Gaudi, which Intel received in its $2B acquisition of Habana Labs last year. This is long-awaited good news for Intel.
Second, and more surprisingly, Jassy announced that AWS would launch a chip it developed in-house for training, called “Trainium,” in the 2nd half of 2021. AWS claims Trainium will be the fastest chip for AI available in the cloud. Supporting Trainium, Gaudi and NVIDIA GPUs is a smart move, and it is consistent with AWS’s strategy of offering customers a variety of technologies to meet their specific needs. Picking just one winner would be foolish. Trainium appears to be a fitting bookend to Inferentia, AWS’s successful proprietary inference chip that was announced and deployed last year.
Qualcomm (QCOM) introduced the latest generation of its popular Snapdragon processor, the model 888, at its annual Snapdragon Summit event on held on December 1st. The new mobile chip sports Qualcomm’s 6th generation AI Engine, which is now a fused processing unit for AI that cranks out 26 TOPS of Int-8 performance. This engine streamlines the previous 3-domain device to support scalar, tensor and vector operations. Additionally, the engine includes 16-times more on-chip memory to handle larger models.
Now, let’s move on to the startups.
- SimpleMachines announced a chip for low power inference. The startup’s chip looks attractive based on the limited specs available: 35 8-bit TOPS at only 4 watts. That’s near the top of the heap! The company touts the invention of its “Composable Behavior Execution,” which enables the chip to manipulate and understand program properties such as data size and program size to optimize storage and execution. But it’s not the only startup angling for the inference market.
- Mythic, an Austin, TX startup, takes the analog processing route for inference. The company recently announced it has begun shipping samples to select customers. Mythic claims to deliver fifty times the density of a digital alternative (let’s assume that was a T4, ok?) at 1/20th the cost. That would be groundbreaking. The Mythic Analog Compute Engine looks to solve embedded and edge inference applications for 4- and 8-bit inference jobs. More details should be forthcoming, so watch this space.
- Imagination, a Chinese-owned and UK-based provider of GPU and other IP for cell phones, announced its AI Series 4 multicore NNA chip IP. A spokesperson said the company targets the Automotive market, which makes sense as there is no 800-pound gorilla to displace. The highly scalable IP, which SoC designers can license and embed in its silicon, looks surprisingly robust. At 30 TOPS per watt, to my knowledge, it is the most power-efficient DNN processor around. I hope to dive deeper into this exciting development soon.
The Cambrian AI Explosion is finally underway, with new chips for inference and training process from established chip companies, cloud service providers and startups. But this game is only in the second inning, and I expect more announcements in the coming year. As usual, I will summarize the developments we observed this year, and speculate on what to expect in 2021, in my annual Cambrian AI Explosion column on Forbes in January. Stay safe, and have a happy holiday season!