A Hybrid GPU + FPGA System Design for Autonomous Driving Cars

Autonomous driving cars need highly complex hardware and software systems, which require high performance computing platforms in order to enable a real time AI-based perception and decision making pipeline. The industry has been exploring various in-vehicle accelerators such as GPUs, ASICs and FPGAs. Yet the autonomous driving platform design is far from mature when taking into account of system reliability, redundancy and higher level of autonomy. In this work, we propose a hybrid computing system design, which integrates a GPU as the primary computing system and a FPGA as a secondary system. This hybrid system architecture has multiple advantages: 1) The FPGA can be constantly running as a complementary system with very short latency, helping to detect main system failure and anomalous behavior, contributing to system functionality verification and reliability. 2) If the primary system fails (mostly from sensor or interconnection error), the FPGA will quickly detect the failure and run a safe-mode task with a subset of sensors. 3) The FPGA can be used as an independent computing system to run extra algorithm components to improve the overall system autonomy. For example, FPGA can handle driver monitoring tasks while GPU focuses on driving functions. Together they can boost the driving function from L2 (constantly requires driver's attention) to L3 (allows driver to mind off for 10 seconds). This paper defines how such a system works, discusses various use cases and potential design challenges, and shares some initial results and insights about how to make such a system deliver the maximum value for autonomous driving.