A visually guided beamformer to aid listening in complex acoustic environments

The purpose of this paper is to provide an overview of recent work on the visually guided hearing aid (VGHA; Kidd et al. JASA 133, EL202). The latest prototype VGHA consists of a lightweight, head-mounted, 18-microphone array combined with PC-based software to create a highly-tuned acoustic beamformer steered in real time by an integrated binocular eye tracker. Recent work on the two main functional sub-components of the VGHA (beamforming and eye-gaze control) focused on the benefits that may be obtained by listeners with hearing loss (and in some cases by listeners with normal hearing) attempting to understand speech masked by one or more competing sounds (such as Gaussian noise or additional speakers). These benefits include improved signal-to-noise ratio under different masked conditions and faster steering (compared to fixed directional amplification steered by head turns). The VGHA may be simulated over headphones by PC-based implementations using HRTFs and an optional external eye-tracker. This has ...