Navigating with a 360°-Camera

Today, industrial processes are highly linked to automation requirements. Essential components of modern storage and transport processes are therefore (partially) autonomous systems such as mobile robots, driverless transport systems or forklift control systems. However, conventional navigation methods represent an obstacle with regard to the requirement for flexibility and cost-effectiveness. In the BLINKER project, scientists are working in cooperation with a medium-sized company on a sensor for autonomous navigation for vehicles.

Intelligent image processing

The sensor will enable the mobile system to precisely determine and regulate its own pose (position and orientation) in in-house environments. The system will be able to orient itself towards artificial, retroreflective brands as well as natural landmarks, for example by means of deep learning-based approaches. Technologically, the approach is based on the fusion of several cameras, an LED illumination unit and intelligent image processing. The cameras are arranged on a ring so that the marks can be detected over the entire horizon (360°). In addition, an upward facing camera is provided. Thus the 360° camera to be developed can perceive a “hemisphere”. By means of image processing of the (artificial or natural) marks, a triangulation algorithm can calculate the exact pose.

During the development of the environmental camera, special attention is paid to the processing of visual information. In addition to natural landmarks in indoor and outdoor areas, the camera should also reliably detect and trace safety-critical objects such as people or obstacles. This enables the enrichment of semantic information and thus completely new services in the industry.

In a first phase, the system gets to know its surroundings and locates images at pose coordinates. These given pairs of image signal to coordinate are used as training data for a deep neural network so that it automatically learns a suitable mapping and can estimate a 3D position from previously unseen images.