Cognitive computing and machine learning are some of the hottest topics in all of high tech. Many companies in the semiconductor industry have been looking for ways to improve how computers interact with us and more importantly, understand our behavior and needs. Because without understanding humans or our behavior, computing devices will never truly become as effective at helping us as they could be. So, naturally, many companies are chasing different solutions that try to improve how computers interact with us and better understand our world.
Companies like IBM are working on solving the big data problem by incorporating cognitive computing into better understanding humans’ questions in order to supply them with the right information. NVIDIA is harnessing their experience and knowledge with GPUs and parallel computing to enable running neural networks that can help improve computer vision and object recognition in order to make transportation safer. Others like Qualcomm are focusing on setting the stage for a new level of intelligence and personalization for mobile devices, and expanding this into other areas, such as automotive, robotics and wearables. Qualcomm’s efforts in this space are being accelerated with their introduction of the Zeroth platform, which is the focus of this column.
Improving how computers understand us
Right now, most of the cognitive computing requires vast amounts of high performance computing, which is generally cloud-based. Nvidia’s DriveX is arguably mobile in the fact that it’s embedded into a car, but once you leave the car it is no longer with you. However, there is no denying that a lot of cognitive computing in the mobile space has a lot to do with computer vision and visually recognizing objects and using them to provide context to the device to help it provide more relevant and accurate data to the user. Some of that occurs at the device end and eventually gets sent to the cloud for computing and recognition and then sent back to the device. This scenario is one that has too much latency in mobile environments, doesn’t actively and continually understand users’ interactions and above all, requires a constant internet connection and added power consumption.
Lots of companies already have the existing silicon, tools and some APIs to make cognitive computing happen, as we’re seeing with NVIDIA and Qualcomm, but it seems that a lot of the cognitive computing capabilities for the real world are accomplished through GPUs. And unsurprisingly, some of the most powerful mobile GPUs available today come from NVIDIA and Qualcomm. But there are other aspects of cognitive computing that require more than just GPUs with simulated neural networks. An example of this is mobile, where you need to have an optimized and balanced heterogeneous computing architecture, which Qualcomm provides as part of their Qualcomm Zeroth platform. This platform takes advantage of “the right engine for the right task” and is optimized for mobile environments.
Qualcomm Zeroth platform brings cognitive computing with the user
Qualcomm’s Zeroth Platform benefits from new hardware and software innovations that are at the heart of Qualcomm’s heterogeneous computing technology and leading edge connectivity within a highly integrated SoC. The Zeroth platform provides the foundation for more intuitive experiences and natural interactions through the addition of on-device intelligence designed for a range of key mobile experiences and cognitive capabilities. The next generation of Snapdragon SOCs should enable more seamless movement of data between different processing engines to enable low latency & efficient processing. As workloads move between different processing engines, Qualcomm’s platform seeks to minimize the power consumption of these workloads while maximizing the performance by choosing the optimal engine for each task.
Qualcomm using visual and auditory cognitive computing to improve experiences
Some of the cognitive computing enabled by the Zeroth platform will have to do with visual use cases, showing the inescapable need to see what’s going on around you to understand your surroundings. Some visual capabilities that Qualcomm is building into their Zeroth platform include visual perception, with the phone recognizing the environment around you so it can capture things that matter the most to you.
Think about the phone knowing you’re at a football game and therefore knows to zoom in on the player’s faces because it recognizes a football helmet. Through on-device deep learning, computer vision and cognitive camera technologies, devices can recognize objects, read handwriting, identify people and understand the overall scene and its context. This opens new possibilities like enabling your phone camera to autonomously adjusts its settings based on its understanding of the nature of the scene—imagine the different settings used when snapping shots at a football game versus a sunny beach versus a child’s birthday party. The phone could start aggregating knowledge it has gained by observing user behavior over time to personalize your pictures by changing exposures on the fly based on lighting and skin tones, facial recognition and other qualities that users may find valuable over time.
There are also additional capabilities that incorporate scene understanding, which may also incorporate other ‘senses’ like sound, which may give the device a better understanding of exactly where it might be. That could prove to be useful for giving the device improved context of surroundings and possibly adjusting things like volume when something loud drives by. Imagine your smartphone knowing you are driving in a car and optimizing the microphone and audio to compensate precisely for the situation. This shouldn’t require the user to change anything and if the device understands the users’ preferences through machine learning, then it can know how to accomplish such tasks automatically and adapt to the environment without user input.
There are other applications like handwriting recognition where the Qualcomm Zeroth platform can help recognize users’ handwriting and without the use of a special stylus. Imagine being able to take a picture of a blackboard, a whiteboard or a page of handwritten notes and the device recognizing what someone wrote down like you and I are able to. Many have tried to do this, but didn’t work well in part to the fact there just wasn’t enough processing power.
On-device intelligence saves power, improves performance, improves privacy
There is simply no denying that on-device cognitive computing with something like Qualcomm’s Zeroth platform makes more sense than constantly sending data to the cloud. Three great reasons jump to mind. First, there’s a much better responsiveness from a device-based solution. Waiting for information to travel to and from the cloud is just not an option for certain situations. On-device processing removes the latency of connecting to the cloud. Imagine driving a car on a highway at night with a moose crossing the road. You would want to be notified as soon as possible. Second, on-device processing improves security and privacy by letting users control their own data. You only share what you want and know when things get sent up to the network. Third, having to constantly contact the cloud will result in even more latency and data traffic through the networks that are already overloaded in many cases. This will also not work very well for users that have data caps, which are fairly common in developed markets. This will also mean added power consumption by constantly keeping the modem connection open. That doesn’t mean that you won’t need the cloud, it just means you’re more in control of both the data charges and security when using on-device cognitive services.
The key with Qualcomm’s cognitive computing approach is to push the intelligence as close as possible to the edge. This is because the performance and experience can be the best when the computing is closest to the user. Now that smartphone has extremely capable mobile processors and there is no reason to leave that processing power untapped or flatline compute capability. By taking advantage of a highly optimized heterogeneous computing architecture, the performance required for on-device cognitive capabilities can be achieved within the power and thermal constraints of mobile devices, and in fact, as a whole, will probably save power when you consider how much computing won’t be done in the cloud all the time. There will still be a need for the cloud to continually improve the intelligence and update the capabilities of these platforms like Qualcomm’s Zeroth, but they won’t be used as constantly as some cognitive cloud solutions today.
Differentiating with mobile
Qualcomm’s approach to cognitive computing is a unique one in the sense that they are harnessing a lot of their already existing technologies and combining them together with new hardware and software innovations in a way that enables better machine understanding of user needs. As it is right now, they are pretty much alone in the field in terms of enabling mostly standalone cognitive computing that is primarily done on the smartphone. By bringing that capability to the device directly, Qualcomm could enable the best possible cognitive computing experiences with minimal power consumption, which is absolutely necessary in today’s mobile world. Qualcomm is making this a reality by optimizing the Zeroth platform for premium mobile devices based on their next-generation premium tier SoC, the Qualcomm Snapdragon 820 processor.
Cognitive computing is going to be what lifts the entire industry up from competing on Gigahertz and number of cores in the coming years and companies will do battle to provide more unique and personal computing experiences. This will enable device manufacturers to continue to differentiate around user experience but in brand new ways and consequently open new market opportunities. Ultimately, the companies that are successful in developing cognitive technologies will do so by improving their customers’ experiences and enhancing their daily lives through simple yet impactful application of intelligence to things that matter most. Because realistically, a lot of the processing capability on our smartphones and tablets is generally sitting idle most of the time and we could be putting a lot of it to good use. That untapped compute capability could be helping us every day, making cognitive computing a constant companion and we wouldn’t even know it.
Qualcomm’s approach is a good one and one that I think they can have a lot of success with given their IP, investment capability and platform delivery track record.