Photonic convolutional processor for network edge computing (Conference Presentation)

Performing feature extractions in convolution neural networks for deep-learning tasks is computational expensive in electronics. Fourier optics allows convolutional filtering via dot-product multiplication in the Fourier domain similar to the distributive law in mathematics. Here we experimentally demonstrate convolutional filtering exploiting massive parallelism (10^6 channels, 8-bit at 1kHz) of digital mirror display technology, thus enabling 250 TMAC/s. An FPGA-PCIe board controls the ‘weights’ and handles the data I/O, whereas a high-speed camera detects the inverse-Fourier transformed (2nd lens) data. Gen-1 processes with a total delay (including I/O) of ~1ms, while Gen-2 at 1-10ns leveraging integrated photonics at 10GHz and changing the front-end I/O to a joint-transform-correlator (JTC). These processors are suited for image/pattern recognition, super resolution for geolocalization, or real-time processing in autonomous vehicles or military decision making.