Qualcomm Doubles The Smart In Smartphones

For the last decade, a “smart phone” was defined by its ability to run applications locally on the device. These days, however, “smart” means features powered by AI, with intelligent applications such as enhanced computational photography, voice processing and gaming with AR features. It appears that Qualcomm’s next generation mobile processors are about to take a major step forward in “smart,” creating next year’s must-have 5G consumer platforms. 

Last week Qualcomm launched the next generation Snapdragon platform for 5G mobile and AI computing, including the flagship Snapdragon 865, the midrange 765 products and a slew of modules and software to enable custom OEM development programs. While there is too much to unpack here in one blog, here’s a few highlights.

What’s new?

A better question might be “what is not new?” These new chips advance practically every technology in mobile, from 5G wireless to a 5th generation AI engine. While the performance gains are relatively modest, they add up. Here’s a snapshot of the key features of the SoC:

  1. The new X52 5G global modem comes integrated on the 765, and is a separate module on the 865. Both come with mmWave for high bandwidth and sub 6Ghz for greater connectivity. mmWave is the holy grail of 5G, but it will take some time for carriers to build out the infrastructure. 
  2. The application CPU was upgraded to four ARM Cortex A77 and four A55s.
  3. The Adreno 650 GPU was upgraded to double AI performance with support for mixed precision math (16- and 32-bit floating point).
  4. The Hexagon 698 tensor engine quadruples performance over the previous generation, supports mixed precision math (8- and 16-bit integer) and includes up to 50% lossless compression for deep learning networks, a key feature as networks continue to expand rapidly.
Figure 1: The new flagship Snapdragon 865 5G puts tremendous AI processing in the palm of your hand, with a rich tools and library stack for software developers. 
QUALCOMM

Net all this out, and the AI engine, which includes the Adreno GPU and the Hexagon accelerator, delivers 15 TOPS, or twice the aggregate performance of the previous 855 model. So, what does one do with all this performance? In general, as much inference processing as possible should be done at the edge. This enables more intelligence to take place on the device, minimizing latency and reducing the dependence on cloud computing resources. Here are some examples:

Qualcomm demonstrated real-time translation live on stage and it worked flawlessly. I suspect many such advanced use cases will require a hybrid approach, accessing cloud AI services transparently to handle some of the more intensive computations. This sort of multi-model query is at the forefront of the latest research in AI and is one of the key reasons a multi-engine approach on the SoC is a good idea. Even in a single network, the developer can tap into scalar, vector and tensor processors to accelerate processing while minimizing power consumption.

Figure 2: The Snapdragon SOC offers scalar, vector, and tensor processing for AI across the various processors for pooling, convolutions, and fully connected networks. 
QUALCOMM

Keep in mind that the complexity and size of neural networks doubles every 3.5 months, so even with all this processing power in your hand, your device will continue to rely on cloud or edge cloud computing resources. Qualcomm intends to take advantage of this trend by adding tightly-integrated edge cloud intelligence close to the device. It will launch a Cloud AI 100 server accelerator sometime in 2020, which will likely be a scaled-up version of the AI engine found in the Snapdragon 865. 

Conclusions

If you are like me, your smartphone can appear pretty dumb when it tries to transcribe spoken commands into actions or text. Qualcomm believes that better local AI processing will be key to solving this endemic 1st world problem. From what I observed at the Tech Summit, Qualcomm and dozens of partners are gearing up to bring useful intelligence to a phone near you. Furthermore, it will bring this intelligence to the datacenters that supplement the mobile devices with massive processing power. It is all about to become really interesting.