Chapter 2 Comparative Face Soft Biometrics for Human Identification

The recent growth in CCTV systems and the challenges of automatically identifying humans under the adverse visual conditions of surveillance have increased the interest in soft biometrics, which are physical attributes that can be used to describe people semantically. Soft biometrics enable human identification based on verbal descriptions, and they can be captured in conditions where it is impossible to acquire traditional biometrics such as iris and fingerprint. The research on facial soft biometrics has tended to focus on identification using categorical attributes, whereas comparative attributes have shown a better accuracy. Nevertheless, the research in comparative facial soft biometrics has been limited to small constrained databases, while identification in surveillance systems involves unconstrained large databases. In this chapter, we explore human identification through comparative facial soft biometrics in large unconstrained databases using the Labelled Faces in the Wild (LFW) database. We propose a novel set of attributes and investigate their significance. Also, we analyse the reliability of comparative facial soft biometrics for realistic databases and explore identification and verification using comparative facial soft biometrics. The results of the performance analysis show that by comparing an unknown subject to a line up of ten subjects only; a correct match will be found in the top 2.08% retrieved subjects from a database of 4038 subjects. Face Biometrics and Semantic Face Recognition The crucial role of surveillance systems in crime prevention and public safety has motivated massive deployments of CCTV networks around the world [1–4]. N.Y. Almudhahka (✉) ⋅ M.S. Nixon ⋅ J.S. Hare University of Southampton, Southampton, UK e-mail: nya1g14@ecs.soton.ac.uk M.S. Nixon e-mail: msn@ecs.soton.ac.uk J.S. Hare e-mail: jsh2@ecs.soton.ac.uk © Springer International Publishing AG 2018 P. Karampelas and T. Bourlai (eds.), Surveillance in Action, Advanced Sciences and Technologies for Security Applications, https://doi.org/10.1007/978-3-319-68533-5_2 25 26 N.Y. Almudhahka et al. For instance, the number of CCTV cameras deployed in cities and town centres in the UK was estimated between 1.85 [5] and 5.9 million [6]. This expansion in the usage of surveillance systems has increased the reliance on CCTV imagery for suspect identification, which is the first challenge faced by law enforcement agencies in criminal investigations [3]. Thus, the need for identifying suspects from imagery databases (i.e. mugshots or CCTV footage) has motivated research in human identification using semantic descriptions based on eyewitnesses’ statements with a view to enabling searching a database of subjects through verbal descriptions [7–9]. These semantic descriptions are based on soft biometrics, which refer to physical and behavioural attributes that can be used to identify people. The main advantage of soft biometrics as compared with traditional hard biometrics (e.g. iris, DNA, and fingerprint) is that they can be acquired at a distance without individuals’ involvement. In addition, soft biometrics enjoy sufficient robustness to the challenging visual conditions of surveillance such as occlusion of features, viewpoint variance, low resolution, and changes in illumination [7, 9, 10]. Therefore, soft biometrics can play a significant role in criminal investigations, where it is required to retrieve the identity of a suspect from an imagery database using a verbal description (i.e. eyewitness statement). Furthermore, soft biometrics bridge the semantic gap resulted from the difference between machines and humans in estimating attributes, which enables retrieval from a database solely by verbal descriptions as illustrated in Fig. 2.1. Soft biometrics can be categorized according to their group and format. In terms of group, soft biometrics may fall under global, facial, body, or clothing attributes. While in terms of format, soft biometrics can be classified as: categorical, where individual attributes are assigned to specific classes (e.g. square versus round jaw); or comparative, where attributes of an individual are estimated relative to another individual (e.g. subject A has a more rounded jaw than subject B) [12]. Comparative soft biometrics are discussed further in more details in the next section. Figure 2.2 shows example categorical soft biometric attributes from the four different groups of soft biometrics: facial, body, clothing, and global. Figure 2.3 presents example soft biometric attributes in the comparative format. More highlights on the different soft biometric attribute formats are provided in Table 2.1. Subject ID ID Biometric signature 1 0.92 0.45 0.72 0.11 2 0.17 0.87 0.82 0.65 3 0.54 0.65 0.35 0.81 4 0.93 0.33 0.74 0.44 n 0.72 0.48 0.52 0.29 Soft biometric database Semantic descriptions “query” Retrieve biometric signatures