Dual-Prior Integrated Image Reconstruction for Quanta Image Sensors Using Multi-Agent Consensus Equilibrium

Quanta image sensors (QIS) are a new type of single-photon imaging device that can oversample the light field to generate binary bit-streams. The reconstruction for QIS refers to the recovery of original scenes from these binary measurements. Conventional reconstruction algorithms for QIS generally depend solely on one instantiated prior and are certainly insufficient for capturing the statistical properties over high-dimensional space. On the other hand, deep learning-based methods have shown promising performance, due to their excellent ability to learn feature representations from relevant databases. However, most deep models only focus on exploring local features while generally overlooking long-range similarity. In view of this, a dual-prior integrated reconstruction algorithm for QIS (DPI-QIS) is proposed, which combines a deep prior with a non-local self-similarity one using the multi-agent consensus equilibrium (MACE) framework. In comparison to the approaches that utilize a single prior, DPI-QIS fits the reconstruction model sufficiently by leveraging the respective merits of both priors. An effective yet flexible MACE framework is employed to integrate the physical forward model allying with the two prior-based models to achieve an overall better result. Extensive experiments demonstrate that the proposed algorithm achieves state-of-the-art performance in terms of objective and visual perception at multiple oversampling factors, while having stronger robustness to noise.