[Publication] Contextual Detection of Pedestrians and Vehicles in Orthophotography by Fusion of Deep Learning Algorithms

Monday, February 21, 2022 - 18:57

This work simultaneously combines three machine learning algorithms with the aim of detecting users crossing an area filmed in top view, identifying their type, locating them in the environment, analyzing their movements and estimating their speeds.

In the context of smart cities, monitoring pedestrian and vehicle movements is essential to recognize abnormal events and prevent accidents. T

he proposed method in this work focuses on analyzing video streams captured from a vertically installed camera, and performing contextual road user detection.

The final detection is based on the fusion of the outputs of three different convolutional neural networks. Our researchers were simultaneously interested in detecting road users, their motions, and their location respecting the static environment.

They used YOLOv4 for object detec-tion, FC-HarDNet for background semantic segmentation, and FlowNet 2.0 for motion detection. FC-HarDNet and YOLOv4 were retrained with their orthophotographs datas and images. The last step involves a data fusion module.

The results show that the method allows one to detect road users (pedestrian, biker, car driver), identify the surfaces on which they move (road, sidewalk, lawn), quantify their apparent velocity and estimate their actual velocity.

Reference:

Title: Contextual detection of pedestrians and vehicles in orthophotography by fusion of deep learning algorithms

Authors: Masoomeh Shireen Ansarnia, Etienne Tisserand, Patrick Schweitzer, Mohamed Amine Zidane et Yves Berviller

Journal : Sensors

Date of publication (online) : February 2022

Link : https://doi.org/10.3390/s22041381

Image caption: Results in real situation