Rearview mirror image with the Mali C71 versus uncited competitor
ARM Holdings, a wholly-owned subsidiary of Softbank Group Corp, announced today the launch of the Mali-C71 image signal processor (ISP), the first product in their new Mali camera family, designed specifically for the Advanced Driver Assistance Systems (ADAS) inside of automobiles. The automobile industry is in transition, led by silicon, not steel, hence we see Tesla more valuable than General Motors. For the uninitiated, ISPs are responsible for processing raw pixel data for display, encoding, and furthering computer vision processing. ISPs are in nearly every electronic device with a camera including self-driving cars like Tesla. ISPs are typically inside an SoC (system on a chip) but not necessarily inside of the camera module which contains the image sensor and the lens. The large ADAS SoC will likely be centralized and process images from the entire car.
Many the new technological advances we’ve been seeing in the industry’s march towards autonomous driving such as distracted driving, pedestrian protection systems, and mirror replacement require powerful image processing capabilities inside the vehicle—capabilities ARM says Mali-C71 will deliver uniquely for that market. It’s worth noting that this is ARM’s first imaging offering since their acquisition of imaging and embedded computer vision company Apical, back in May 2016. Mali-C71 is our first glimpse at the fruits of that deal. So far, I’m encouraged by what I see and strategically, this demonstrates ARH Holdings IP expansion to more vertical workloads.
Safety with more cameras in higher definition
Higher-end vehicles in the not-so-distant future will need many more built-in cameras—possibly over 10 in a single vehicle. 10 cameras may sound nuts but when you consider you would need one on each of the four corners, two on the sides, two for the mirrors, two on top of the vehicle looking front and back and one for the driver, it starts to add up easily. We’re talking a whole lot of complex
cameras, with the ability to analyze and process unusual lighting and weather conditions. You want to be able detect all elements of the scene you’re capturing—from a pedestrian standing in the shade, to a puppy in a thunderstorm. ARM Mali-C71 offers ultra-wide dynamic range up to an impressive 24 “stops” which, in certain cases, is more detailed than what the human eye can even perceive. The best DSLR cameras around can only achieve around 15 stops, so this really is an insane level of clarity and certainly higher than any smartphone today.
Benefits of UWD dynamic range
After the Mali-C71 removes noise and processes the multiple exposures from a camera, an ultra-WDR (wide dynamic range) frame is generated and sent to a display or a computer vision engine like a CPU, GPU, FPGA or ASIC. Next-generation ISPs must be able to generate data for display, while simultaneously processing the same data for use by a computer vision engine so that quicker decisions can be made. ARM says the Mali-C71’s ability to provide these two differently processed outputs from a single image is a key strength of the ISP.
All of this must be up to the stringent (and necessarily so) standards of automotive safety. Not exactly a walk in the park, but ARM says that Mali-C71 was designed and built “from the ground up” for ADAS System on Chips (SoCs)—with this challenge taken into account. The Mali C71 boasts low-latency and error detection, and has over 300 dedicated fault detection circuits to ensure compliance with the highest ASIL safety and reliability standards.
I think for ARM’s first ISP offering, Mali-C71 is impressive and I like ARM Holding’s IP growth trajectory. This is a hot area and it will be important for ARM to make continuous improvements over time to this ISP. I can also see a home for the C71 or a derivative of it in other autonomous devices like drones and robots.
The functions of ISPs are becoming increasingly centralized within an ADAS SoC; ARM claims that the Mali-C71 is the first ISP to successfully implement parallel virtual processing outputs and built-in functional safety features within a single SoC. If it works as claimed, I believe Mali-C71 has the potential to really put ARM ahead of the pack in the next-generation automotive computing industry. Let’s face it, cars aren’t going to get less complex—they need to have the right architecture in place to meet rising demands. I’ll continue to follow with interest as ARM expands their Mali portfolio, will do some competitive analysis on the C71 with Qualcomm, NVIDIA, NXP and Intel, but the C71 seems to be a good starting point.