A 65 nm 2.02 mW 50 Mbps Direct Analog to MJPEG Converter for Video Sensor Nodes using low-noise Switched Capacitor MAC-Quantizer with automatic calibration and Sparsity-aware ADC

Recent expansions in camera deployments for applications like surveillance, healthcare, autonomous vehicles, robots necessitate low-power realizations even with large data volume. Standard digital video cameras generate $\sim$ Gbps data that are usually compressed to 10’s of Mbps, often in the camera (e.g. Omnivision 5640) using compression schemes, like lossy MJPEG before high energy/bit wireless communication. For instance (Fig. 1), a $2 \mathrm{~K}$ RGB 8 b video at 30 fps will generate $\sim 1.5 \mathrm{Gbps}$ that is reduced to $\sim 75 \mathrm{Mbps}$ (MJPEG: per-frame compression) or to $\sim 6 \mathrm{Mbps}$ (H.264: both per-frame and inter-frame compression). While H.264 enables lower data rate, it suffers from complex decoding (incurs high latency), and poor error tolerance, thus MJPEG is preferred for latency-sensitive error-prone communication of compressed video with real-time inference needs. However, such in-camera ADC with Digital compression suffers from 1) high data volume (Gbps) generated, consuming high ADC energy; 2) high compute energy required by the in-sensor digital accelerator to compress the data, increasing the power consumption of the processing modules in these nodes to $\sim 10$ ’s mW. This work draws on the recent progress of mixed-signal in-sensor computing to address the above challenges in these high-power and high datavolume nodes and presents a “direct Analog to MJPEG Conversion (d-AJC)” technique that (1) compresses the analog pixel values from imagers in the mixed-signal domain using Switched Capacitor (SC)-based multiply and accumulate (MAC) units, and Quantizer, before (2) digitizing them with an event-driven sparsity aware ADC with run-length encoding, effectively compressing Gbps data in SC domain with 10’s of Mbps ADC, leading to an order of magnitude system power benefit. With gaining prominence of low-power analog camera sensors (e.g., 25mW Omnivision 6946), along with the development of low-cost processing schemes, d-AJC, it is envisioned to solve issue of high-power imager nodes.