AI ecosystem for autonomous driving

Anyone getting into a fully automated vehicle wants to reach their destination safely. This is ensured by numerous complex artificial intelligence (AI) algorithms that analyze enormous amounts of sensor data in the shortest possible time, for example to identify other road users. Before fully automated driving can be approved in Germany, the processors still have to meet stringent requirements. Not only must the computing power of hardware and algorithms increase, there is also a need for improvement in terms of energy efficiency, reliability, robustness and safety.

In order to be able to use the complex algorithms efficiently in vehicles, an entire AI ecosystem consisting of a powerful AI hardware platform including optimized AI signal processing is required. This is precisely what scientists at the L3S research center at Leibniz Universität Hannover (LUH) are researching together with other partners in the ZuSE-KI-Mobil project funded by the German Federal Ministry of Education and Research. In addition to LUH, the project consortium includes TU Dresden, Karlsruhe Institute of Technology (KIT), TU Munich, BMW, Dreamchip, ZF Friedrichshafen, Infineon and T3 Technologies.

L3S member Prof. Dr. Holger Blume and his team are involved in the research and development of the heterogeneous hardware platform. In addition to a processor system with several computing units that perform tasks in parallel and therefore faster, the platform consists of two dedicated AI accelerators and other dedicated processor units that are responsible for image pre-processing, among other things. A special feature of the hardware platform is the SPA-ML AI accelerator developed by scientists at TU Dresden, which efficiently calculates Convolutional Neural Networks (CNNs). CNNs are currently the most widely used type of AI algorithm in sensor signal processing. To support modern CNN algorithms, the hardware is also being expanded in the project.

A neural network compiler is required to execute CNNs as efficiently as possible on the new accelerator hardware. To this end, the L3S scientists have integrated the SPA-ML accelerator into the Apache TVM compiler framework. This makes it possible to map AI models that are available in an ONNX or Pytorch implementation, for example, to the accelerator. The compiler is responsible for reducing or adapting the high computing and memory requirements of the CNNs to such an extent that they can be executed on the accelerator. To do this, the compiler transfers the CNN into an intermediate format and performs optimization steps such as memory tiling or layer fusion. The compiler then creates an executable program.

In the project, the Hanover-based scientists are also researching new AI algorithms to better process sensor signals and optimize them for the ZuSE AI mobile platform. The algorithms are designed to detect objects in the vehicle environment quickly and accurately. To do this, the data from different sensors must be fused. Camera data can be used to classify objects as cars or motorcycles, for example, but determining their 3D position is imprecise. The data from other sensors is therefore added: LiDAR sensors to determine the position of an object; RADAR sensors to measure the speed of objects. A larger number of sensors leads to a more accurate and less error-prone result, but is accompanied by an increase in the computational requirements and complexity of the model.

The aim of the research and development work in the ZuSE-KI-Mobil project is to create a modern and efficient basis for a German AI ecosystem consisting of AI hardware and algorithms in order to position Germany as a sovereign and competitive player in the field of artificial intelligence for autonomous driving.