A 128 × 128 SPAD Motion-Triggered Time-of-Flight Image Sensor With In-Pixel Histogram and Column-Parallel Vision Processor

A 128 $\times $ 128 single-photon avalanche diode (SPAD) motion detection-triggered time-of-flight (ToF) sensor is implemented in STMicroelectronics 40 nm CMOS SPAD foundry process. The sensor combines vision and ToF ranging functions to acquire depth frames only when inter-frame intensity changes are detected. The 40 $\mu \text{m}\,\,\times $ 20 $\mu \text{m}$ pixels integrate two 16-bit time-gated counters to acquire ToF histograms and repurpose them to compare two vision frames without the requirement for additional out-of-pixel frame memory resources. An embedded column-parallel ToF and vision processor performs on-chip vision frame comparison and binary frame output compression as well as controlling the time-resolved histogram sampling. The sensor achieves a maximum 32.5 kframes/s in vision modality and 500 frames/s in motion detection-triggered ToF over a measured 3.5 m distance with 1.5 cm accuracy. The vision function reduces the sensor power consumption by 70% over continuous ToF operation and allows the sensor to gate the ToF laser emitter to reduce the system power when no motion activity is observed.