Biased Feature Learning for Occlusion Invariant Face Recognition

Changbin Shao
Changbin Shao
Lei Qi
Lei Qi
Zhen-Hua Feng
Zhen-Hua Feng
Chuanqi Dong
Chuanqi Dong

IJCAI, pp. 666-672, 2020.

Cited by: 0|Bibtex|Views65|Links
EI
Keywords:
fr systemOccluded LFW DatasetBatch Normalizationlow rankocclusion invariant face recognitionMore(20+)
Weibo:
The experimental results demonstrated the merits of our Biased Feature Learning method as well as its generalization capability with different network architectures and loss functions

Abstract:

To address the challenges posed by unknown occlusions, we propose a Biased Feature Learning (BFL) framework for occlusion-invariant face recognition. We first construct an extended dataset using a multi-scale data augmentation method. For model training, we modify the label loss to adjust the impact of normal and occluded samples. Further...More

Code:

Data:

0
Introduction
  • As an important authentication technique, Face Recognition (FR) has been widely used in many practical applications.
  • Occlusion-invariant FR aims to learn a model with good generalization capability such that it can be readily adapted to occluded faces, normal faces.
  • There are only few studies that focus on generalized feature learning for occlusion-invariant FR.
  • To close this gap, the authors aim to improve the generalization capability of a model for unknown occlusions
Highlights
  • As an important authentication technique, Face Recognition (FR) has been widely used in many practical applications
  • We propose a novel Biased Feature Learning (BFL) framework for occlusion-invariant Face Recognition, as shown in Fig. 1
  • There is no a public face dataset specially designed for occlusion-invariant Face Recognition
  • It is necessary to redefine the evaluation as three different types: normal face pairs verification (N-N), verification between normal and occluded faces (N-O) and occluded face pairs (O-O), as shown in Fig. 5
  • To address the challenges posed by unknown occlusions, we presented a reasonable model evaluation protocol and benchmarking dataset for occlusion-invariant face recognition
  • The experimental results demonstrated the merits of our Biased Feature Learning method as well as its generalization capability with different network architectures and loss functions
Methods
  • BFL+Dropout(0.5) BFL+Crop(0.6-1) BFL+Crop(0.8-1) N -N N -O O-O Dataset.
  • Baseline(IN ) Dropout(0.5) Crop(0.6-1) Crop(0.8-1) Baseline (IN +IO) BFL.
  • The authors compare totality and sub-components of BFL framework with some existing methods.This is equivalent to the ablation study for Section 5.3.
  • The results are reported with the average value between the 16th and 20th epochs in Table 1 and Table 2, in which Baseline denotes the conventional training way and BFL is the proposed method with λ=0.5
Results
  • To evaluate the performance of a model, a public dataset with reasonable evaluation protocol is necessary.
  • The well-known LFW dataset has been a widely used benchmark for normal face verification.
  • There is no a public face dataset specially designed for occlusion-invariant FR.
  • The authors modify LFW and extend its evaluation protocol for occlusion-invariant FR.
  • For face verification with occlusions, there are two types of faces on a dataset, normal and occluded faces.
  • It is necessary to redefine the evaluation as three different types: normal face pairs verification (N-N), verification between normal and occluded faces (N-O) and occluded face pairs (O-O), as shown in Fig. 5
Conclusion
  • To address the challenges posed by unknown occlusions, the authors presented a reasonable model evaluation protocol and benchmarking dataset for occlusion-invariant face recognition.
  • The authors proposed a novel biased feature learning framework for deep network training.
  • The proposed BFL framework uses a biased guidance strategy to promote the feature learning of a face recognition model.
  • The experimental results demonstrated the merits of the BFL method as well as its generalization capability with different network architectures and loss functions
Summary
  • Introduction:

    As an important authentication technique, Face Recognition (FR) has been widely used in many practical applications.
  • Occlusion-invariant FR aims to learn a model with good generalization capability such that it can be readily adapted to occluded faces, normal faces.
  • There are only few studies that focus on generalized feature learning for occlusion-invariant FR.
  • To close this gap, the authors aim to improve the generalization capability of a model for unknown occlusions
  • Methods:

    BFL+Dropout(0.5) BFL+Crop(0.6-1) BFL+Crop(0.8-1) N -N N -O O-O Dataset.
  • Baseline(IN ) Dropout(0.5) Crop(0.6-1) Crop(0.8-1) Baseline (IN +IO) BFL.
  • The authors compare totality and sub-components of BFL framework with some existing methods.This is equivalent to the ablation study for Section 5.3.
  • The results are reported with the average value between the 16th and 20th epochs in Table 1 and Table 2, in which Baseline denotes the conventional training way and BFL is the proposed method with λ=0.5
  • Results:

    To evaluate the performance of a model, a public dataset with reasonable evaluation protocol is necessary.
  • The well-known LFW dataset has been a widely used benchmark for normal face verification.
  • There is no a public face dataset specially designed for occlusion-invariant FR.
  • The authors modify LFW and extend its evaluation protocol for occlusion-invariant FR.
  • For face verification with occlusions, there are two types of faces on a dataset, normal and occluded faces.
  • It is necessary to redefine the evaluation as three different types: normal face pairs verification (N-N), verification between normal and occluded faces (N-O) and occluded face pairs (O-O), as shown in Fig. 5
  • Conclusion:

    To address the challenges posed by unknown occlusions, the authors presented a reasonable model evaluation protocol and benchmarking dataset for occlusion-invariant face recognition.
  • The authors proposed a novel biased feature learning framework for deep network training.
  • The proposed BFL framework uses a biased guidance strategy to promote the feature learning of a face recognition model.
  • The experimental results demonstrated the merits of the BFL method as well as its generalization capability with different network architectures and loss functions
Tables
  • Table1: Verification results (%) of L09 on O-LFW datasets
  • Table2: Verification results (%) of R18 on O-LFW datasets
  • Table3: Verification results (%) of 4 models on different losses
Download tables as Excel
Related work
  • In this section, we introduce the related work by dividing them into three categories.

    Linear regression. Assume that occlusion error is sparse relative to the standard (pixel) basis, SRC uses the L1 regularization to code a query sample as a linear combination of atoms and assigns the label to the class with the minimum reconstruction error. To enhance the discrimination of coding, structured sparse coding [Li et al, 2013] and nonnegative dictionary learning [Ou et al, 2018] were proposed. To address the small-sample-size problem, extended dictionaries [Deng et al, 2012; Shao et al, 2017] with intra-class face variations posed by occlusions were developed. Due to the low-rank characteristic of occlusion in comparison to face size, [Iliadis et al, 2017; Wu and Ding, 2018] appended low-rank constrains to occlusion error. To characterize the 2D structural information of occlusions, [Yang et al, 2017] used the nuclear norm to deal with occlusion and illumination variations. However, all these linear methods are limited to frontal faces under closed set scenarios.
Funding
  • This work is supported by the Science and Technology Innovation 2030 – “New Generation Artificial Intelligence” Major Project (No 2018AAA0100900), National Science Foundation of China (No 61806092, No 61902153) and Jiangsu Natural Science Foundation (No BK20180326)
Reference
  • [Candes et al., 2011] Emmanuel J Candes, Xiaodong Li, Yi Ma, and John Wright. Robust principal component analysis? Journal of the ACM, 58(3):11, 2011.
    Google ScholarLocate open access versionFindings
  • [Deng et al., 2012] Weihong Deng, Jiani Hu, and Jun Guo. Extended src: Undersampled face recognition via intraclass variant dictionary. TPAMI, 34(9):1864–1870, 2012.
    Google ScholarLocate open access versionFindings
  • [Deng et al., 2019] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In CVPR, pages 4685– 4694, 2019.
    Google ScholarLocate open access versionFindings
  • [Goodfellow et al., 2014] Ian Goodfellow, Jean Pougetabadie, Mehdi Mirza, Bing Xu, David Wardefarley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS, pages 2672–2680, 2014.
    Google ScholarLocate open access versionFindings
  • [He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016.
    Google ScholarLocate open access versionFindings
  • [Iliadis et al., 2017] Michael Iliadis, Haohong Wang, Rafael Molina, and Aggelos K Katsaggelos. Robust and low-rank representation for fast face identification with occlusions. TIP, 26(5):2203–2218, 2017.
    Google ScholarLocate open access versionFindings
  • [Li et al., 2013] Xiaoxin Li, Daoqing Dai, Xiaofei Zhang, and Chuanxian Ren. Structured sparse error coding for face recognition with occlusion. TIP, 22(5):1889–1900, 2013.
    Google ScholarLocate open access versionFindings
  • [Li et al., 2017] Yijun Li, Sifei Liu, Jimei Yang, and Minghsuan Yang. Generative face completion. In CVPR, pages 5892–5900, 2017.
    Google ScholarLocate open access versionFindings
  • [Liu et al., 2013] Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, and Yi Ma. Robust recovery of subspace structures by low-rank representation. TPAMI, 35(1):171–184, 2013.
    Google ScholarLocate open access versionFindings
  • [Loshchilov and Hutter, 2019] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019.
    Google ScholarLocate open access versionFindings
  • [Ou et al., 2018] Weihua Ou, Xiao Luan, Jianping Gou, Quan Zhou, Wenjun Xiao, Xiangguang Xiong, and Wu Zeng. Robust discriminative nonnegative dictionary learning for occluded face recognition. Pattern Recognition Letters, 107:41–49, 2018.
    Google ScholarLocate open access versionFindings
  • [Saeztrigueros et al., 2018] Daniel Saeztrigueros, Li Meng, and Margaret Hartnett. Enhancing convolutional neural networks for face recognition with occlusion maps and batch triplet loss. Image and Vision Computing, 79:99– 108, 2018.
    Google ScholarLocate open access versionFindings
  • [Schroff et al., 2015] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In CVPR, pages 815–823, 2015.
    Google ScholarLocate open access versionFindings
  • [Shao et al., 2017] Changbin Shao, Xiaoning Song, Zhenhua Feng, Xiaojun Wu, and Yuhui Zheng. Dynamic dictionary optimization for sparse-representation-based face classification using local difference images. Information Sciences, 393:1–14, 2017.
    Google ScholarLocate open access versionFindings
  • [Singh et al., 2019] Maneet Singh, Richa Singh, Mayank Vatsa, Nalini K Ratha, and Rama Chellappa. Recognizing disguised faces in the wild. TBIOM, 1(2):97–108, 2019.
    Google ScholarLocate open access versionFindings
  • [Song et al., 2019] Lingxue Song, Dihong Gong, Zhifeng Li, Changsong Liu, and Wei Liu. Occlusion robust face recognition based on mask learning with pairwise differential siamese network. In ICCV, pages 773–782, 2019.
    Google ScholarLocate open access versionFindings
  • [Taigman et al., 2014] Yaniv Taigman, Ming Yang, Marcaurelio Ranzato, and Lior Wolf. Deepface: Closing the gap to human-level performance in face verification. In CVPR, pages 1701–1708, 2014.
    Google ScholarLocate open access versionFindings
  • [Wang et al., 2018] Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In CVPR, pages 5265–5274, 2018.
    Google ScholarLocate open access versionFindings
  • [Wen et al., 2016] Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning approach for deep face recognition. In ECCV, pages 499– 515.
    Google ScholarLocate open access versionFindings
  • [Wright et al., 2009a] John Wright, Arvind Ganesh, Shankar R Rao, Yigang Peng, and Yi Ma. Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. In NIPS, pages 2080–2088, 2009.
    Google ScholarLocate open access versionFindings
  • [Wright et al., 2009b] John Wright, Allen Y Yang, Arvind Ganesh, Shankar Sastry, and Yi Ma. Robust face recognition via sparse representation. TPAMI, 31(2):210–227, 2009.
    Google ScholarLocate open access versionFindings
  • [Wu and Ding, 2018] Choying Wu and Jianjiun Ding. Occluded face recognition using low-rank regression with generalized gradient direction. Pattern Recognition, 80:256–268, 2018.
    Google ScholarLocate open access versionFindings
  • [Wu et al., 2018] Xiang Wu, Ran He, Zhenan Sun, and Tieniu Tan. A light cnn for deep face representation with noisy labels. TIFS, 13(11):2884–2896, 2018.
    Google ScholarLocate open access versionFindings
  • [Yang et al., 2017] Jian Yang, Lei Luo, Jianjun Qian, Ying Tai, Fanlong Zhang, and Yong Xu. Nuclear norm based matrix regression with applications to face recognition with occlusion and illumination changes. TPAMI, 39(1):156–171, 2017.
    Google ScholarLocate open access versionFindings
  • [Yuan and Park, 2019] Xiaowei Yuan and Inkyu Park. Face de-occlusion using 3d morphable model and generative adversarial network. In ICCV, pages 10062–10071, 2019.
    Google ScholarLocate open access versionFindings
  • [Zhang et al., 2015] Fanlong Zhang, Jian Yang, Ying Tai, and Jinhui Tang. Double nuclear norm-based matrix decomposition for occluded image recovery and background modeling. TIP, 24(6):1956–1966, 2015.
    Google ScholarLocate open access versionFindings
  • [Zhao et al., 2017] Fang Zhao, Jiashi Feng, Jian Zhao, Wenhan Yang, and Shuicheng Yan. Robust lstm-autoencoders for face de-occlusion in the wild. TIP, 27(2):778–790, 2017.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments