Artificial intelligence optical hardware empowers high-resolution hyperspectral video understanding at 1.2 Tb/s
CoRR(2023)
摘要
Foundation models, exemplified by GPT technology, are discovering new
horizons in artificial intelligence by executing tasks beyond their designers'
expectations. While the present generation provides fundamental advances in
understanding language and images, the next frontier is video comprehension.
Progress in this area must overcome the 1 Tb/s data rate demanded to grasp
real-time multidimensional video information. This speed limit lies well beyond
the capabilities of the existing generation of hardware, imposing a roadblock
to further advances. This work introduces a hardware-accelerated integrated
optoelectronic platform for multidimensional video understanding in real-time.
The technology platform combines artificial intelligence hardware, processing
information optically, with state-of-the-art machine vision networks, resulting
in a data processing speed of 1.2 Tb/s with hundreds of frequency bands and
megapixel spatial resolution at video rates. Such performance, validated in the
AI tasks of video semantic segmentation and object understanding in indoor and
aerial applications, surpasses the speed of the closest technologies with
similar spectral resolution by three to four orders of magnitude. This platform
opens up new avenues for research in real-time AI video understanding of
multidimensional visual information, helping the empowerment of future
human-machine interactions and cognitive processing developments.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要