Enhanced visual perception through photon counting and computational imaging: what the time and number of photon events can tell us about the world around us

Classical electro-optical (EO) sensing focuses on detecting light intensity and displaying it in grayscale images. Therefore, the development of EO systems was mainly concerned with the trade-off between resolution, signal-to-noise ratio and image or data rate. In recent years, the development of silicon based single photon-counting avalanche diode (SPAD) detectors has initiated an almost unnoticed revolutionary development and paradigm shift in sensor technology. In contrast to intensity imaging, SPAD sensors are capable to sense a single photon event as a binary and noise free intensity. Furthermore, the occurrence of these photon events can be determined very precisely in time, allowing access to the sub-nanosecond or even picosecond time scale and application in precise distance measurement. Combined with computational imaging, i.e. linking the acquisition process to a strong mathematical model, it is possible to obtain information that was previously inaccessible and extend the perceptual range of EO sensors. In this paper, we provide an overview of various active and passive SPAD imaging techniques that are coupled with laser illumination or use ambient light. We focus on applications with relevance to military use. In detail, we summarize results in the areas of passive imaging, range finding and vision through scattering media, and advanced acquisition methods such as non-line-of-sight imaging.