Correlation-Based Background Music Recommendation By Incorporating Temporal Sequence Of Local Features

Background music plays an important role in making user-generated video more colorful and attractive. One of current research on automatic background music recommendation is the correlation-based approach in which the correlation model between visual and music features is discovered from training data and is utilized to recommend background music for query video. While existing correlation-based approaches consider global features only, in this work we proposed to integrate the temporal sequence of local features along with global features into the correlation modeling process. The local features are derived from segmented audiovisual clips. Then the temporal sequence of local features is transformed and incorporated into correlation modeling process. Cross-Modal Factor Analysis along with Multiple-type Latent Semantic Analysis and Canonical Correlation Analysis, are investigated for correlation modeling which recommends background music in ranking order. Experiments show that the proposed approach outperforms existing work.