Object Detection and Tracking for Multi-Sensor Autonomous Driving Systems

The future of human transportation is coming, we are on the brink of the next technological revolution: autonomous vehicles will become a part of road traffic. Autonomous driving promises an efficient, safe, and comfortable world of transportation. The data required to implement this revolution is provided by cameras and other sensors, processed in real-time by small onboard computers in fractions of a second. These vehicles will likely exchange information among themselves and with the transport infrastructure to make the collective imagination about a city of the future a reality. However, before all this happens, some steps still need to be taken. First of all, is it truly possible to make a vehicle totally aware of its environmental situation? Being able to perceive objects and obstacles in the environment and react in real-time, even through numerous sensors, is a challenge we have yet to fully overcome. Additionally, sensors and all other components of the architecture of these systems age, deteriorate, and experience faults. Is it possible to predict their failures? Monitoring techniques are necessary to help us implement graceful degradation to avoid catastrophic scenarios. Finally, how can we assess the moment when an autonomous vehicle attains a degree of maturity that qualifies it for safe navigation on our roads? This PhD thesis endeavors to answer the questions related to these topics, applying our findings to a real and challenging use-case such as autonomous driving for urban tram systems. In the first part, we introduce the topic of performance in obstacle detection and tracking systems used in autonomous driving to gain real-time awareness of the surrounding environment. Solutions are proposed based on both CPU and GPU architectures, leveraging heterogeneous embedded boards, to overcome current computational limitations of data association and tracking algorithms and enhance system responsiveness. In the second part, our investigation comes to the aspects of safety. We delve into the issue of fault detection within the autonomous driving system, focusing on early detection mechanisms to mitigate potential system errors. In this regard, we propose an innovative solution based on observer strategies, to enhance the resilience of critical applications without slowing them down and with low computational cost. In the third part, we then analyze possible techniques and strategies for the validation of autonomous driving systems and their subsequent deployment. Among the many possible methodologies, we focus on the analysis and evaluation of synthetic scenarios, as this type of testing is among the first to be carried out when it is still possible to steer the development of the driving system to improve its functioning. A robust system validation methodology has been devised, incorporating real-world dynamics and how they are perceived by sensors, to assess the safety of autonomous vehicles in various operational scenarios. The benefits we can expect from autonomous driving are numerous, with a significant social impact in many respects. With this thesis, we aim to advance the research in this field, by proposing solutions and improvements to some of the most critical problems that must be addressed to bring us one step closer to the future of mobility.