Image Signal Processor Like Arm’s Is Critical To ADAS Success

By Patrick Moorhead - March 14, 2022
Image Signal Processor Like Arm’s Is Critical To ADAS Success ARM

As the automotive industry is climbing up the ladder of intelligent driving, from advanced driver assist systems (ADAS) to autonomous driving (AD), the solutions that are making these innovative feats possible are improving. I follow the disruptive technology that goes into the next generation of vehicles very closely. Qualcomm announced its new ADAS offering in its Digital Chassis platform and Intel Mobileye is moving closer towards robo taxis with its new EyeQ Ultra SoC.

This week Arm adds its new automotive image signal processor for ADAS to its automotive portfolio, the Mali-C78AE ISP. While Arm is dominant in smartphones and IoT, its automotive IP is more competitive. Its “AE” series of automotive processor IP is critical to the growth and competitiveness of the automotive industry. In my most recent blog on Arm, I even noted that automotive is one of the biggest areas of growth for Arm. 

Safety first

The “AE” series of Arm’s processor IP meets ISO 26262 safety standards that are very critical to electric and electronic systems in a vehicle. ISO 26262 has four levels of safety hazard referred to as the Automotive Safety Integrity Level (ASIL). There are levels A, B, C, and D with ASIL D being the highest degree of automotive hazard, meeting the strictest safety requirements. For example, of Arm’s automotive processor IP, Its Cortex-A78AE CPU supports ISO 26262 ASIL B and D safety requirements. Arm’s Mali-C78AE ISP meets ISO 26262 ASIL B functional safety requirements which means it meets the requirements for automotive use.

If we take a step back and see Arm’s ISP as the replacement or semi-replacement for ADAS and AD systems, apart from the brains that make the decision the eyes are the most important component of a driver. That is one reason why paraplegic drives can adapt to driving but unfortunately the blind cannot. Automotive ISPs are going to replace the driver's eyes and will play one of the most important roles in achieving full autonomy. The human eye, although the complex is only two “cameras” to the driving component. For redundant ADAS and AD systems, there LiDAR and radar and the traditional camera will play into the robustness of the self-driving system. 

On one hand, processors are going to play the most prominent role in making decisions with powerful computations. What makes these decisions easier to make is the redundancy or abundance of information that comes from the ISP of the sensors. One of the reasons that Qualcomm is increasingly successful in the automotive space is because of its similarities to mobile and just as Arms is dominant in mobile so is Qualcomm. I believe Arm is positioned to be successful in the automotive industry and providing processor IP with the safety requirements for automotive is a key to its success. Arm says that the value of the automotive camera market is expected to grow greater than 19% in the next five years. I expect the ISP of an automotive system to play a larger role in the development of ADAS and AD systems and should be critical to the execution of L3+ ADAS in the near future.

I am not surprised that Mobileye is jumping on Arm’s new ISP for ADAS. Mobileye puts a focus on redundancy with ADAS sensors and the Mali-C78AE should allow Mobileye to meet the functional safety requirements. Arm says the Mali-C78AE has over 380 fault detection circuits that can detect sensor and hardware faults of connected cameras. I like how this fault detection feature accounts for safety in diagnostic terms and not just in driving terms. Going back to the human driver analogy, it is very noticeable when a driver’s eyes are not working properly. Not only for the driver but also for other drivers, especially other drivers that cut you off on the highway. The Mali-C78AE has this detection built in to notice when a single camera frame is incorrectly processed.

Powerfully quick

Arm says the Mali-C78AE is capable of processing data from up to four real-time cameras or 16 virtual cameras. As I have mentioned previously in this blog, the more information comes into the driving system, the easier and more accurate the decision is for an intelligent driving system. 

Arm says the Mali-C78AE employs advanced noise reduction technology and dynamic range management to ensure each frame is clear and properly exposed by adjusting overly dark or bright areas of a frame. This ability to change the exposure of a frame to present clear and accurate information to the driving system is paramount to the decision-making process. If a frame of data is even slightly misinformed or too slow the decision-making process could make critical mistakes that compromise the safety of the vehicle and passengers. For the Mali-C78AE to be able to adjust the frame data within the roughly 150-millisecond window dramatically reduces the chance of the Cortex-C78AE of making a mistake.

Showing the difference between the Mali-C78AE with and without a change in exposure. AARM

The Mali-C78AE is able to handle this quick processing for up to four high-resolution high-frame-rate cameras in real-time. Its ability to have more than one quality camera is not only good for redundancy but also cost and power of the system by reducing the number of ADAS functions.

Wrapping up

Just as the human eye plays an important role in driving a car, the vision system of an ADAS and self-driving vehicle plays an important role. Arm’s focus on safety, redundancy, and power with its latest Mali-C78AE ISP brings  all of those requirements to the table. The more accurate information the sensors can bring to the decision-making process, the easier it is for the processors to make the right choice. 

Seeing as Mobileye is the first to jump on Arm’s latest ISP, I believe the Mali-C78AE could play as important of a role as any other processing unit in an ADAS and AD system. The smartphone SoC and the automotive SoC are very similar with the exception of safety requirements and extra performance. However, because of the need for safety requirements, I believe ISPs are going to play a much bigger role in the development of automotive SoCs than in smartphone SoCs.

Note: Moor Insights & Strategy co-op Jacob Freyman contributed to this article.

+ posts
Patrick founded the firm based on his real-world world technology experiences with the understanding of what he wasn’t getting from analysts and consultants. Ten years later, Patrick is ranked #1 among technology industry analysts in terms of “power” (ARInsights)  in “press citations” (Apollo Research). Moorhead is a contributor at Forbes and frequently appears on CNBC. He is a broad-based analyst covering a wide variety of topics including the cloud, enterprise SaaS, collaboration, client computing, and semiconductors. He has 30 years of experience including 15 years of executive experience at high tech companies (NCR, AT&T, Compaq, now HP, and AMD) leading strategy, product management, product marketing, and corporate marketing, including three industry board appointments.