What Does Google’s Tensor SoC Mean For Google’s Hardware And Software Efforts?

Google recently announced its Tensor SoC for its upcoming Pixel 6 flagship phones. GOOGLE

This past week, Google announced its upcoming Tensor SoC, supported by a blog from Google Hardware lead Rick Osterloh detailing how the SoC will impact Google’s Pixel 6 and Pixel 6 Pro flagship phones. Interestingly, though Osterloh’s blog mentions AI four times, it never once mentions 5G—the primary selling point for phones these days. This is likely because Google’s biggest strength is in software, and more specifically, AI.

Google’s AI hardware experience

If Google’s announcement of its new Tensor smartphone SoC sounded familiar to you at all, it’s because Google’s Tensor Processor Units, known as TPUs, are already in use in the cloud for AI inference purposes. Google’s TPU v4 for the cloud put up some impressive preliminary MLPerf training results against Nvidia’s A100, which is considered the gold standard for training. So, naturally, Google’s decision to name its new SoC (codenamed Whitechapel) Tensor, means Google is aiming for heavy ML tasks, primarily on the inference side. One thing to consider is that inference and training are very different types of machine learning functions and generally require different specific AI hardware. When you consider that most consumer products are built to do inference and not training, you wouldn’t necessarily expect TPUs to function the same as the Tensor SoC. Not all hardware that can do training is as good at inference and vice versa. For the Google Pixel, ML performance per watt is far more important than absolute performance since we’re talking about smartphone with a limited battery and a huge display.

The critical thing to consider about the Tensor SoC is that it will target AI performance first and foremost. Google is best in class when it comes to enriching the user experience with AI-accelerated features. Google Assistant is at the core of that experience, combining its complex voice recognition and context algorithms with its adept search capabilities. When you consider that Google’s translation, photography and dictation features also depend on ML acceleration, it’s understandable why Google would want to beef up that performance. Suppose Google can lead on AI features and use cases and accelerate them faster than competitors. In that case, there is very little chance that its competitors will catch up, making its user experience unattainable outside of its ecosystem.

Current and future silicon partners

Google has yet to announce the SoC’s foundry partner, which will either be TSMC or Samsung, or its modem partner, either MediaTek, Qualcomm or Samsung. It is unlikely that Google would drop Qualcomm’s SoC and stick with its modem, but we’ll leave it in the list for now. It’s worth remembering that if Google ditches Qualcomm, it risks the Pixel not having the fastest modem in town (especially considering Apple currently uses Qualcomm in its entire 5G iPhone 12 lineup). Many rumors suggest that Samsung is both the foundry partner and the modem provider, which would make sense because Samsung is the only company that could provide Google both at the same time.

According to  Qualcomm, the Snapdragon 888 has an AI performance of 26 trillion operations per second (TOPS). I would expect that Google’s Tensor SoC would need to surpass that if the company really wants to justify spending hundreds of millions of dollars extra every year to build a custom SoC for its Pixel line. That said, if Google builds a powerful enough SoC, it could foreseeably use these same SoCs in its tablets and Chromebooks. That could offer an interesting opportunity for Google to solve one of the biggest problems that plagues custom SoC designs: scale. Without scale, building a custom SoC is a losing proposition, and if a design team isn’t competitive with what’s already available in the market today, it comes off as a loss-leading exercise. Samsung and Apple both do it today because their businesses bring them scale. Huawei did the same until the US Government made it virtually impossible. In short, there’s an established track record of this being done by smartphone manufacturers, but generally ones with much higher volumes than Google.

Google Pixel phones. GOOGLE

Wrapping up

All that said, I do believe that Google has opportunities to improve the overall user experience on the Google Pixel line by fully integrating the SoC, OS and device under one roof. Apple has shown that the whole-system approach to power, performance, and user experience pays dividends—it outperforms the rest of the industry in nearly every benchmark. However, Apple simply does not have the amount of data or AI capabilities that Google does. I understand Google’s desire to take control of the whole device, but I worry that the company may hamper it’s competitiveness by choosing the wrong modem partner or make some other move (like not having enough RAM) that undercuts what it is trying to do with the Pixel 6.