Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input

David Harwath

Adrià Recasens

[0]

Dídac Surís

Galen Chuang

Antonio Torralba

[0]

James Glass

[0]

International Journal of Computer Vision, pp. 620-641, 2020.

Cited by: 93|Views55

Keywords:

Vision and language Sound Speech Multimodal learning Language acquisitionMore(3+)

Abstract:

In this paper, we explore neural network models that learn to associate segments of spoken audio captions with the semantically relevant portions of natural images that they refer to. We demonstrate that these audio-visual associative localizations emerge from network-internal representations learned as a by-product of training to perform...More

Code:

Data:

Get fulltext within 24h

View via Publisher