4D radar imaging for target detection and classification using deep-learning

Abstract

The goal of this project is to develop an object detection and classification system for road crossing areas as part of a monitoring system using 4D radar imaging with a deep-learning neural network approach. In this work, we utilised deep neural networks powered by Keras and Tensorflow to detect and classify multiple pedestrians, cars, buses, and trucks. This paper presents Retina-4F, which is a multi-chip radar imaging system with high range resolution for object detection and localization. Retina-4F, which was developed by Smart Radar System, allows the system to provide 4D real-time information about the target. Retina-4F utilises a multi-chip cascade onboard with three transmitters and four receivers in each chip. We demonstrated two road-crossing scenes to collect data for creating a point cloud dataset with a target class label to be used for training and testing a deep learning model. There are two main sensors implemented in this work: Retina-4F as a 4D radar imaging and a mono-camera. The data from both sensors is pre-processed using DBSCAN and YOLOv7. Retina-4F operates at 77 GHz, and the test was conducted in two different road areas. After conducting data measurement at two road crossing areas, the collected data is passed for preprocessing and data fusion processing. This results in a complete point cloud dataset with approximately 10,000 frames of point cloud images that can be used for neural network training and testing. The model evaluation showed satisfying performance of the deep neural network in classifying multiple targets with 97 percent of overall accuracy. The approach of sensor data fusion for multiple target classification shows good results where it manages to distinguish different types of targets: cars, pedestrians, buses, and trucks. The proposed radar point cloud classification using sensor fusion can be applied to a wide variety of complex monitoring applications.