A Memristive Associative Learning Circuit for Fault-Tolerant Multi-Sensor Fusion in Autonomous Vehicle

Abstract

Autonomous vehicles completely rely on accurate multi-sensor fusion to perceive their environment and make driving decisions. However, conventional AI-based perception systems face challenges in irregular conditions such as poor visibility, occlusions, or adverse weather conditions, which can lead to incomplete or degraded information from sensors reaching the central computing/navigation system. This severely impacts perception accuracy, potentially compromising vehicle, and pedestrian safety. This work presents a memristor-based associative learning circuit that enhances fault tolerance by dynamically adapting to multi-sensor inputs, including camera, LiDAR, radar, and ultrasonic sensors. The proposed circuit dynamically reinforces patterns, allowing the system to retain decision-making capabilities even when certain sensors fail or provide incomplete data. The fault tolerance of the circuit is validated through error analysis, proving that accurate outputs are generated even with missing sensor inputs. The system demonstrates an average error of 6.98% across 10 critical driving scenarios, with a power consumption of approximate to 152 mW per scenario, confirming its robustness, energy efficiency and adaptability in case of sensor failures and under-performance. The response time of the circuit has been optimized from milliseconds to seconds, aligning with realistic human-like reaction times required for autonomous navigation.Autonomous vehicles completely rely on accurate multi-sensor fusion to perceive their environment and make driving decisions. However, conventional AI-based perception systems face challenges in irregular conditions such as poor visibility, occlusions, or adverse weather conditions, which can lead to incomplete or degraded information from sensors reaching the central computing/navigation system. This severely impacts perception accuracy, potentially compromising vehicle, and pedestrian safety. This work presents a memristor-based associative learning circuit that enhances fault tolerance by dynamically adapting to multi-sensor inputs, including camera, LiDAR, radar, and ultrasonic sensors. The proposed circuit dynamically reinforces patterns, allowing the system to retain decision-making capabilities even when certain sensors fail or provide incomplete data. The fault tolerance of the circuit is validated through error analysis, proving that accurate outputs are generated even with missing sensor inputs. The system demonstrates an average error of 6.98% across 10 critical driving scenarios, with a power consumption of approximate to 152 mW per scenario, confirming its robustness, energy efficiency and adaptability in case of sensor failures and under-performance. The response time of the circuit has been optimized from milliseconds to seconds, aligning with realistic human-like reaction times required for autonomous navigation.