MWC: Qualcomm Uses Zeroth Cognitive Computing Platform To Reflect Human-Like Thinking And Actions

Cognitive computing and machine learning are some of the hottest topics in all of high tech. Many companies in the semiconductor industry have been looking for ways to improve how computers interact with us and more importantly, understand our behavior and needs. Because without understanding humans or our behavior, computing devices will never truly become as effective at helping us as they could be. So, naturally, many companies are chasing different solutions that try to improve how computers interact with us and better understand our world.

Companies like IBM are working on solving the big data problem by incorporating cognitive computing into better understanding humans’ questions in order to supply them with the right information. NVIDIA is harnessing their experience and knowledge with GPUs and parallel computing to enable running neural networks that can help improve computer vision and object recognition in order to make transportation safer. Others like Qualcomm are focusing on setting the stage for a new level of intelligence and personalization for mobile devices, and expanding this into other areas, such as automotive, robotics and wearables. Qualcomm’s efforts in this space are being accelerated with their introduction of the Zeroth platform, which is the focus of this column.

Improving how computers understand us

Right now, most of the cognitive computing requires vast amounts of high performance computing, which is generally cloud-based. Nvidia’s DriveX is arguably mobile in the fact that it’s embedded into a car, but once you leave the car it is no longer with you. However, there is no denying that a lot of cognitive computing in the mobile space has a lot to do with computer vision and visually recognizing objects and using them to provide context to the device to help it provide more relevant and accurate data to the user. Some of that occurs at the device end and eventually gets sent to the cloud for computing and recognition and then sent back to the device. This scenario is one that has too much latency in mobile environments, doesn’t actively and continually understand users’ interactions and above all, requires a constant internet connection and added power consumption.

(Credit: Qualcomm)

(Credit: Qualcomm)

Qualcomm Zeroth platform brings cognitive computing with the user

Qualcomm’s Zeroth Platform benefits from new hardware and software innovations that are at the heart of Qualcomm’s heterogeneous computing technology and leading edge connectivity within a highly integrated SoC. The Zeroth platform provides the foundation for more intuitive experiences and natural interactions through the addition of on-device intelligence designed for a range of key mobile experiences and cognitive capabilities. The next generation of Snapdragon SOCs should enable more seamless movement of data between different processing engines to enable low latency & efficient processing. As workloads move between different processing engines, Qualcomm’s platform seeks to minimize the power consumption of these workloads while maximizing the performance by choosing the optimal engine for each task.

Qualcomm using visual and auditory cognitive computing to improve experiences

Some of the cognitive computing enabled by the Zeroth platform will have to do with visual use cases, showing the inescapable need to see what’s going on around you to understand your surroundings. Some visual capabilities that Qualcomm is building into their Zeroth platform include visual perception, with the phone recognizing the environment around you so it can capture things that matter the most to you.

Think about the phone knowing you’re at a football game and therefore knows to zoom in on the player’s faces because it recognizes a football helmet. Through on-device deep learning, computer vision and cognitive camera technologies, devices can recognize objects, read handwriting, identify people and understand the overall scene and its context. This opens new possibilities like enabling your phone camera to autonomously adjusts its settings based on its understanding of the nature of the scene—imagine the different settings used when snapping shots at a football game versus a sunny beach versus a child’s birthday party. The phone could start aggregating knowledge it has gained by observing user behavior over time to personalize your pictures by changing exposures on the fly based on lighting and skin tones, facial recognition and other qualities that users may find valuable over time.

There are also additional capabilities that incorporate scene understanding, which may also incorporate other ‘senses’ like sound, which may give the device a better understanding of exactly where it might be. That could prove to be useful for giving the device improved context of surroundings and possibly adjusting things like volume when something loud drives by. Imagine your smartphone knowing you are driving in a car and optimizing the microphone and audio to compensate precisely for the situation. This shouldn’t require the user to change anything and if the device understands the users’ preferences through machine learning, then it can know how to accomplish such tasks automatically and adapt to the environment without user input.

There are other applications like handwriting recognition where the Qualcomm Zeroth platform can help recognize users’ handwriting and without the use of a special stylus. Imagine being able to take a picture of a blackboard, a whiteboard or a page of handwritten notes and the device recognizing what someone wrote down like you and I are able to. Many have tried to do this, but didn’t work well in part to the fact there just wasn’t enough processing power.

On-device intelligence saves power, improves performance, improves privacy

The key with Qualcomm’s cognitive computing approach is to push the intelligence as close as possible to the edge. This is because the performance and experience can be the best when the computing is closest to the user. Now that smartphone has extremely capable mobile processors and there is no reason to leave that processing power untapped or flatline compute capability. By taking advantage of a highly optimized heterogeneous computing architecture, the performance required for on-device cognitive capabilities can be achieved within the power and thermal constraints of mobile devices, and in fact, as a whole, will probably save power when you consider how much computing won’t be done in the cloud all the time. There will still be a need for the cloud to continually improve the intelligence and update the capabilities of these platforms like Qualcomm’s Zeroth, but they won’t be used as constantly as some cognitive cloud solutions today.

Differentiating with mobile

Qualcomm’s approach to cognitive computing is a unique one in the sense that they are harnessing a lot of their already existing technologies and combining them together with new hardware and software innovations in a way that enables better machine understanding of user needs. As it is right now, they are pretty much alone in the field in terms of enabling mostly standalone cognitive computing that is primarily done on the smartphone. By bringing that capability to the device directly, Qualcomm could enable the best possible cognitive computing experiences with minimal power consumption, which is absolutely necessary in today’s mobile world. Qualcomm is making this a reality by optimizing the Zeroth platform for premium mobile devices based on their next-generation premium tier SoC, the Qualcomm Snapdragon 820 processor.

Wrapping up…

Cognitive computing is going to be what lifts the entire industry up from competing on Gigahertz and number of cores in the coming years and companies will do battle to provide more unique and personal computing experiences. This will enable device manufacturers to continue to differentiate around user experience but in brand new ways and consequently open new market opportunities. Ultimately, the companies that are successful in developing cognitive technologies will do so by improving their customers’ experiences and enhancing their daily lives through simple yet impactful application of intelligence to things that matter most. Because realistically, a lot of the processing capability on our smartphones and tablets is generally sitting idle most of the time and we could be putting a lot of it to good use. That untapped compute capability could be helping us every day, making cognitive computing a constant companion and we wouldn’t even know it.

Qualcomm’s approach is a good one and one that I think they can have a lot of success with given their IP, investment capability and platform delivery track record.