ADAS and IMS solutions for mass production of tens of thousands of smart cars

“Based on the FPGA platform, Zibo Technology continues to mass-produce highly reliable and cost-effective advanced driver assistance systems (ADAS), intelligent cockpit systems (IMS) and other in-vehicle visual perception intelligent driving solutions.

“

Based on the FPGA platform, Zibo Technology continues to mass-produce highly reliable and cost-effective advanced driver assistance systems (ADAS), intelligent cockpit systems (IMS) and other in-vehicle visual perception intelligent driving solutions.

Shenzhen Self-Technology Co., Ltd. is a world-leading provider of intelligent driving solutions and equipment, and a pioneer enterprise in China that develops computer vision and sensor fusion technology based on FPGA hardware platform. , with deep talent and technical reserves, as well as a complete product development tool chain. At the same time, the company is also one of the drafting units of China’s “Driver Attention Monitoring System Performance Requirements and Test Methods” (called DMS in the industry) national standard, Bosch China software partner, and Toyota Tsusho and Xilinx certification. one of the partners.

Figure 1C The core technology of Self-Technology

Project Overview

The development of artificial intelligence, 5G and Internet of Vehicles technologies is redefining the safety and human-computer interaction experience of future cars, and the iteration and deployment of various intelligent driving technologies are also accelerating.

“Intelligent driving, ADAS first”. ADAS and IMS, as the first two important areas of intelligent driving, promote the implementation of intelligent driving from the two directions of “driving automation” and “cockpit intelligence”.

Self-Technology is one of the earliest companies in China to contact FPGA, and it is also one of Xilinx’s earliest certified partners in the automotive market in China. With the help of Xilinx’s automotive-grade Zynq®-7000 SoC series adaptive computing platform, Autobahn has continued to mass-produce a series of ADAS and IMS and other in-vehicle visual perception intelligent driving solutions. The Dynamic Function Transition (DFX) capability of the Xilinx Zynq platform provides Autotech with higher performance, lower latency, and the flexibility of real-time loading and dynamic transition of multiple ADAS functions on a unified computing resource.

Industry Challenges

The Around View Monitor (AVM) is one of the ADAS functions with the highest penetration rate in recent years, also known as the 360-degree panoramic view system. The system senses environmental data outside the car through four camera sensors installed on the front, rear, left, and right of the vehicle, helping drivers understand the environment around the vehicle and providing timely reminders before danger occurs. Compared with a simple front-view or rear-view system, AVM needs to process 360° data from the car body in real time, and at the same time helps the driver to make quick decisions through technologies such as image stitching and video processing.

Traditional AVM systems are usually implemented based on ARM or GPU, for example, NXP’s I.MX6 series chips. The application of these chip platforms to AVM systems will face various limitations, such as low ARM efficiency, high GPU power consumption, slow startup time and high cost, which make them unable to meet the ultra-large-scale deployment of intelligent driving in car factories in terms of energy efficiency and cost. need.

At the same time, in the field of smart cockpit, with the growing demand of end consumers for diversified interactive experience of smart cockpit, in addition to traditional voice and touch interaction, intelligent interaction based on visual sensors and multi-sensor fusion intelligent interaction are also becoming more and more important. Favored by consumers. In the face of diverse and continuously changing experience requirements, intelligent requirements, sensor fusion requirements, and constant cost requirements, in a limited cockpit environment, existing fixed or standard processor platforms such as ASIC, ASSP, and DSP are all It is difficult to meet all of the above requirements in terms of size, performance, cost, etc. at the same time.

With rich experience in FPGA design, our own technology is the first to realize that FPGA has the characteristics of flexible deployment, fast startup speed, strong stability, rich logic resources, and self-adaptive deployment, which can well meet the needs of AVM and IMS systems. Various functions. They hope to combine their own world-leading AI technology and excellent FPGA development capabilities to create a new generation of AVM and IMS systems for the industry.

solution

Autotech selected Xilinx Automotive Grade (XA) Zynq™-7000 SoC series adaptive computing platform to build a new generation of AVM and IMS systems.

Dr. Guan Yanfeng, CEO of Zynq, said: “Zynq SoC, as an FPGA-based heterogeneous multiprocessor hardware processor platform, integrates programmable logic (PL) and on-chip processing system (PS), and not only has general processors, such as ASIC, The capabilities of ASSP and DSP, and more capabilities not found in other processors. These capabilities, combined with our leading AI deep learning technology and computer vision and sensor fusion technology, create a new generation of AVM and IMS systems with Valuable differentiation and leading edge.”

In response to customer needs, Autotech has developed an efficient AVM system based on Xilinx’s mature automotive-grade Zynq-7Z010 platform, which achieves fast startup speed, low splicing distortion, and high robustness, as well as dynamic viewing angle follow-up. And functions such as dynamic loading of front and rear auxiliary lines have injected a fresh impetus into the traditional AVM system in the Red Sea market. At the same time, with the help of its leading AI algorithm capabilities, Zynq-7Z010 and 7Z020 platforms have developed a cost-effective and rich and diverse IMS with a wide range of functions, including fatigue detection, attention detection, identity recognition, and status recognition. , behavior recognition, gesture recognition, emotion recognition, occupant status, rear row recognition, etc.

Compared with the existing solutions in the industry, the new generation AVM and IMS system of Autotech realizes:

● Higher performance. A single Zynq chip can realize the complete ADAS solution design from environmental characterization to sensor fusion and feature extraction, which makes the self-technical solution avoid the bandwidth bottleneck and security risks of inter-chip transmission of all other cross-chip platform solutions, thus improving the performance of the solution. higher, more stable and reliable.

● Lower latency. The ability to react in real time with low latency is critical to driving safety. The unique FPGA parallel processing advantage of Xilinx SoC enables it to flexibly connect to various sensors and process data from the entire front-end sensor in nanosecond or microsecond parallel, with lower system latency than GPU or CPU.

● Lower system total cost of ownership (TCO): Self-developed technology makes full use of rich hardware resources such as BRAM, lookup table, register, DSP in Zynq, and successfully deploys computationally complex deep learning networks on low-cost Zynq chips, and successfully develops cost-effective, versatile and highly competitive IMS and ADAS systems.

● More flexible. With the unique reconfigurable function of FPGA, and the deep learning modular design and reorganization technology of Autodesk, Autodesk’s new solution can support users to customize new functions according to their needs, or easily expand sensors. At the same time, it also lays a solid foundation for the rapid optimization and deployment of new deep neural networks and the subsequent expansion of various ADAS system functions.

● Smarter and more secure: The DFX function of Zynq SoC enables Zynq’s new solution to realize real-time loading and dynamic conversion of multiple ADAS functions on a unified computing resource. For example, other driver assistance functions such as Lane Departure Warning (LDW) and Blind Spot Detection (BSD) are implemented on the new AVM system to make driving smarter and safer.

Figure 2C Layout of intelligent products of self-driving technology vehicles

Design effectiveness

Based on FPGA hardware platform, Self-Technology continues to develop cost-effective ADAS, IMS and other intelligent driving solutions. Compared with the traditional I.MX6D solution, the brand-new AVM system of Autotech reduces the cost by more than 50% compared with the traditional chip (I.MX6D solution) only on the main chip, and realizes the current mainstream smart car in terms of function and performance. The level of Electronic product solutions can meet the needs of domestic and foreign smart car electronic product markets.

At present, all kinds of intelligent driving electronic products and solutions developed by Zibo Technology based on the FPGA platform have been successfully used in domestic commercial vehicles and passenger vehicles (China’s largest passenger car brand, and well-known joint venture car brand models with annual sales of 2 million, etc.). The installation market has achieved stable mass production, and the cumulative shipment has reached tens of thousands of units.

Figure 3C Schematic diagram of DMS products for passenger cars

Figure 4C Schematic diagram of automotive AVM products