Fusion of bottleneck, spectral and modulation spectral features for improved speaker verification of neutral and whispered speech.

•We have addressed the problem of speaker verification (SV) based on whispered speech.•The use of bottleneck features in the context of speaker verification using whispered speech.•Two fusion schemes were implemented to overcome existing challenges observed for the task at hand.•Challengue 1: Short duration utterances (4.5 s average), no whispered speech data during enrollment from target speakers.•Challenge 2: The negative effects observed when adding whispered speech recordings during enrollment.•Dedicated systems per vocal effort offer a promising solution. A neutral/whispered speech classification system was implemented.