Adversarial Domain Adaptation with Domain Mixup

national conference on artificial intelligence, 2020.

Cited by: 7|Bibtex|Views114|Links
Keywords:
multiple kernel version of MMDinter domainlatent spacedomain adaptation with domain mixuplevel domainMore(25+)
Weibo:
In order to facilitate a more continuous domain-invariant latent space and fully utilize the inter-domain information, we propose the domain mixup on pixel and feature level

Abstract:

Recent works on domain adaptation reveal the effectiveness of adversarial learning on filling the discrepancy between source and target domains. However, two common limitations exist in current adversarial-learning-based methods. First, samples from two domains alone are not sufficient to ensure domain-invariance at most part of latent ...More

Code:

Data:

0
Introduction
  • With the appearance of Convolutional Neural Networks (CNNs), many classification-based challenges have been tackled with an extremely high accuracy
  • These powerful CNN architectures, like AlexNet (Krizhevsky, Sutskever, and Hinton 2012) and ResNet (He et al 2016), are capable of efficiently extracting low-level and high-level features with the guidance of labeled data.
  • In order to fool this domain discriminator, the extracted features
Highlights
  • In recent years, with the appearance of Convolutional Neural Networks (CNNs), many classification-based challenges have been tackled with an extremely high accuracy
  • We further explore the usage of domain mixup on pixel and feature level to enhance the robustness of adaptation models
  • Under the condition that large domain shift exists, like transferring from synthetic objects to real images in this task, we think that the triplet loss and soft label play a critical role in excavating intermediate status between two domains
  • We address the problem of unsupervised domain adaptation
  • In order to facilitate a more continuous domain-invariant latent space and fully utilize the inter-domain information, we propose the domain mixup on pixel and feature level
  • Experiments prove the effectiveness of our approach, and we achieve state-of-the-art in most settings
  • Extensive experiments on adaptation tasks with different extent of domain shift and data complexity demonstrate the predominant performance of our approach
Methods
  • Source only MMD (2015) RevGrad (2015) CoGAN (2016) DRCN (2016) ADDA (2017) PixelDA (2017) MSTN (2018) GTA (2018) ADR (2018a) DM-ADA

    MN → US (p) 76.0 ± 1.8 95.9 ± 0.7

    US → MN 59.5 ± 1.9.
  • Source only MMD (2015) RevGrad (2015) CoGAN (2016) DRCN (2016) ADDA (2017) PixelDA (2017) MSTN (2018) GTA (2018) ADR (2018a) DM-ADA.
  • The authors' approach achieves the stateof-the-art performance on all four settings.
  • It outperforms former GAN-based approaches (Bousmalis et al 2017; Ghifary et al 2016; Sankaranarayanan et al 2018), which illustrates the effectiveness of the proposed architecture on aligning source and target domains
Results
  • Table 1 presents the results of the approach in.
  • For two easier cases: W → D and D → W, the approach achieves accuracy higher than 99.5% and ranks the first two places.
  • Table 3 reports the results on the VisDA-2017 cross-domain classification dataset.
  • Inputted with extracted features of two domains, a SVM classifier is used to classify the source and target domain features, and (a) baseline (b) w/ PM (c) w/ PM & FM (d) full model
Conclusion
  • The domain discriminator is of high capacity to accurately judge the generated images containing oscillations to two domains
  • In the implementation, such discriminator is trained with pairs of linearly mixed image xm = λxs + (1 − λ)xt and corresponding soft label ldmom = λ, where xm simulates an oscillation mode to two domains and λ provides the guidance.
  • In order to facilitate a more continuous domain-invariant latent space and fully utilize the inter-domain information, the authors propose the domain mixup on pixel and feature level.
Summary
  • Introduction:

    With the appearance of Convolutional Neural Networks (CNNs), many classification-based challenges have been tackled with an extremely high accuracy
  • These powerful CNN architectures, like AlexNet (Krizhevsky, Sutskever, and Hinton 2012) and ResNet (He et al 2016), are capable of efficiently extracting low-level and high-level features with the guidance of labeled data.
  • In order to fool this domain discriminator, the extracted features
  • Methods:

    Source only MMD (2015) RevGrad (2015) CoGAN (2016) DRCN (2016) ADDA (2017) PixelDA (2017) MSTN (2018) GTA (2018) ADR (2018a) DM-ADA

    MN → US (p) 76.0 ± 1.8 95.9 ± 0.7

    US → MN 59.5 ± 1.9.
  • Source only MMD (2015) RevGrad (2015) CoGAN (2016) DRCN (2016) ADDA (2017) PixelDA (2017) MSTN (2018) GTA (2018) ADR (2018a) DM-ADA.
  • The authors' approach achieves the stateof-the-art performance on all four settings.
  • It outperforms former GAN-based approaches (Bousmalis et al 2017; Ghifary et al 2016; Sankaranarayanan et al 2018), which illustrates the effectiveness of the proposed architecture on aligning source and target domains
  • Results:

    Table 1 presents the results of the approach in.
  • For two easier cases: W → D and D → W, the approach achieves accuracy higher than 99.5% and ranks the first two places.
  • Table 3 reports the results on the VisDA-2017 cross-domain classification dataset.
  • Inputted with extracted features of two domains, a SVM classifier is used to classify the source and target domain features, and (a) baseline (b) w/ PM (c) w/ PM & FM (d) full model
  • Conclusion:

    The domain discriminator is of high capacity to accurately judge the generated images containing oscillations to two domains
  • In the implementation, such discriminator is trained with pairs of linearly mixed image xm = λxs + (1 − λ)xt and corresponding soft label ldmom = λ, where xm simulates an oscillation mode to two domains and λ provides the guidance.
  • In order to facilitate a more continuous domain-invariant latent space and fully utilize the inter-domain information, the authors propose the domain mixup on pixel and feature level.
Tables
  • Table1: Classification accuracy (mean ± std %) values of target domain over five independent runs on the digits datasets. The best performance is indicated in bold and the second best one is underlined
  • Table2: Classification accuracy (mean ± std %) values of target domain over five independent runs on the Office-31 dataset. The best performance is indicated in bold and the second best one is underlined
  • Table3: Classification accuracy on the validation set of VisDA-2017 challenge
  • Table4: Effectiveness of pixel-level mixup (PM), featurelevel mixup (FM) and triplet loss (Tri)
  • Table5: Effectiveness of Dcls and pseudo target labels
Download tables as Excel
Related work
  • Domain adaptation is a frequently used technique to promote the generalization ability of models trained on a single domain in many Computer Vision tasks. In this section, we describe existing domain adaptation methods and compare our approach with them.

    The transferability of Deep Neural Networks is proved in (Yosinski et al 2014), and deep learning methods for domain adaptation can be classified into several categories. Maximum Mean Discrepancy (MMD) (Gretton et al 2012; Tzeng et al 2014) is a way to measure the similarity of two distributions. Weighted Domain Adaptation Network (WDAN) (Yan et al 2017) defines the weighted MMD with class conditional distribution on both domains. The multiple kernel version of MMD (MK-MMD) is explored in (Long et al 2015) to define the distance between two distributions. In addition, specific deep neural networks are constructed to restrict the domain-invariance of top layers by aligning the second-order statistics (Sun and Saenko 2016).
Funding
  • This work was supported by National Science Foundation of China (61976137, U1611461)
  • This work was also supported by SJTU-BIGO Joint Research Fund, and CCFTencent Open Fund
Reference
  • Ajakan, H.; Germain, P.; Larochelle, H.; Laviolette, F.; and Marchand, M. 2013. Domain-adversarial neural networks. CoRR abs/1412.4446.
    Findings
  • Ben-David, S.; Blitzer, J.; Crammer, K.; Kulesza, A.; Pereira, F.; and Vaughan, J. W. 2010. A theory of learning from different domains. Machine Learning 79(1-2):151–175.
    Google ScholarLocate open access versionFindings
  • Berthelot, D.; Raffel, C.; Roy, A.; and Goodfellow, I. 2019. Understanding and improving interpolation in autoencoders via an adversarial regularizer. In ICLR.
    Google ScholarFindings
  • Bousmalis, K.; Silberman, N.; Dohan, D.; Erhan, D.; and Krishnan, D. 2017. Unsupervised pixel-level domain adaptation with generative adversarial networks. In CVPR.
    Google ScholarFindings
  • Ganin, Y., and Lempitsky, V. S. 201Unsupervised domain adaptation by backpropagation. In ICML.
    Google ScholarFindings
  • Ganin, Y.; Ustinova, E.; Ajakan, H.; Germain, P.; Larochelle, H.; Laviolette, F.; Marchand, M.; and Lempitsky, V. 201Domainadversarial training of neural networks. JMLR 17(1):2096–2030.
    Google ScholarLocate open access versionFindings
  • Ghifary, M.; Kleijn, W. B.; Zhang, M.; Balduzzi, D.; and Li, W. 2016. Deep reconstruction-classification networks for unsupervised domain adaptation. In ECCV.
    Google ScholarFindings
  • Goodfellow, I. J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; WardeFarley, D.; Ozair, S.; Courville, A. C.; and Bengio, Y. 2014. Generative adversarial nets. In NeurIPS.
    Google ScholarFindings
  • Gretton, A.; Borgwardt, K. M.; Rasch, M. J.; Scholkopf, B.; and Smola, A. J. 2012. A kernel two-sample test. JMLR 13:723–773.
    Google ScholarLocate open access versionFindings
  • He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In CVPR.
    Google ScholarFindings
  • Hoffman, J.; Tzeng, E.; Park, T.; Zhu, J.; Isola, P.; Saenko, K.; Efros, A. A.; and Darrell, T. 2018. Cycada: Cycle-consistent adversarial domain adaptation. In ICML.
    Google ScholarFindings
  • Hull, J. J. 1994. A database for handwritten text recognition research. IEEE Trans. Pattern Anal. Mach. Intell. 16(5):550–554.
    Google ScholarLocate open access versionFindings
  • Kang, G.; Zheng, L.; Yan, Y.; and Yang, Y. 2018. Deep adversarial attention alignment for unsupervised domain adaptation: The benefit of target expectation maximization. In ECCV.
    Google ScholarFindings
  • Kingma, D. P., and Welling, M. 2013. Auto-encoding variational bayes. CoRR abs/1312.6114.
    Findings
  • Krizhevsky, A.; Sutskever, I.; and Hinton, G. E. 2012. Imagenet classification with deep convolutional neural networks. In NeurIPS.
    Google ScholarFindings
  • Larsen, A. B. L.; Sønderby, S. K.; Larochelle, H.; and Winther, O. 20Autoencoding beyond pixels using a learned similarity metric. In ICML.
    Google ScholarFindings
  • Lin, T.; Maire, M.; Belongie, S. J.; Hays, J.; Perona, P.; Ramanan, D.; Dollar, P.; and Zitnick, C. L. 2014. Microsoft COCO: common objects in context. In ECCV.
    Google ScholarLocate open access versionFindings
  • Liu, M., and Tuzel, O. 2016. Coupled generative adversarial networks. In NeurIPS.
    Google ScholarFindings
  • Long, M.; Cao, Y.; Wang, J.; and Jordan, M. I. 2015. Learning transferable features with deep adaptation networks. In ICML.
    Google ScholarFindings
  • Long, M.; Zhu, H.; Wang, J.; and Jordan, M. I. 2017. Deep transfer learning with joint adaptation networks. In ICML.
    Google ScholarFindings
  • Long, M.; Cao, Z.; Wang, J.; and Jordan, M. I. 2018. Conditional adversarial domain adaptation. In NeurIPS.
    Google ScholarFindings
  • Lcun, Y.; Bottou, L.; Bengio, Y.; and Haffner, P. 1998. Gradientbased learning applied to document recognition. Proceedings of the IEEE 86(11):2278–2324.
    Google ScholarLocate open access versionFindings
  • Ma, X.; Zhang, T.; and Xu, C. 2019. Gcan: Graph convolutional adversarial network for unsupervised domain adaptation. In CVPR.
    Google ScholarFindings
  • Maaten, L. V. D., and Hinton, G. 2008. Visualizing data using t-sne. JMLR 9(2605):2579–2605.
    Google ScholarLocate open access versionFindings
  • Mansour, Y.; Mohri, M.; and Rostamizadeh, A. 2009. Domain adaptation: Learning bounds and algorithms. In Annual Conference on Learning Theory.
    Google ScholarLocate open access versionFindings
  • Netzer, Y.; Wang, T.; Coates, A.; Bissacco, A.; Wu, B.; and Ng, A. Y. 2011. Reading digits in natural images with unsupervised feature learning. NeurIPS Workshop.
    Google ScholarFindings
  • Pan, S. J., and Yang, Q. 2010. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10):1345–1359.
    Google ScholarLocate open access versionFindings
  • Pan, S. J.; Tsang, I. W.; Kwok, J. T.; and Yang, Q. 2011. Domain adaptation via transfer component analysis. IEEE Trans. Neural Networks 22(2):199–210.
    Google ScholarLocate open access versionFindings
  • Paszke, A.; Gross, S.; Chintala, S.; Chanan, G.; Yang, E.; DeVito, Z.; Lin, Z.; Desmaison, A.; Antiga, L.; and Lerer, A. 2017. Automatic differentiation in pytorch. In NeurIPS Workshop.
    Google ScholarLocate open access versionFindings
  • Pei, Z.; Cao, Z.; Long, M.; and Wang, J. 2018. Multi-adversarial domain adaptation. In AAAI.
    Google ScholarFindings
  • Peng, X.; Usman, B.; Kaushik, N.; Hoffman, J.; Wang, D.; and Saenko, K. 2017. Visda: The visual domain adaptation challenge. CoRR abs/1710.06924.
    Findings
  • Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M. S.; Berg, A. C.; and Li, F. 2015. Imagenet large scale visual recognition challenge. IJCV 115(3):211–252.
    Google ScholarLocate open access versionFindings
  • Saenko, K.; Kulis, B.; Fritz, M.; and Darrell, T. 2010. Adapting visual category models to new domains. In ECCV.
    Google ScholarFindings
  • Saito, K.; Ushiku, Y.; Harada, T.; and Saenko, K. 2018a. Adversarial dropout regularization. In ICLR.
    Google ScholarFindings
  • Saito, K.; Watanabe, K.; Ushiku, Y.; and Harada, T. 2018b. Maximum classifier discrepancy for unsupervised domain adaptation. In CVPR.
    Google ScholarFindings
  • Sankaranarayanan, S.; Balaji, Y.; Castillo, C. D.; and Chellappa, R. 2018. Generate to adapt: Aligning domains using generative adversarial networks. In CVPR.
    Google ScholarFindings
  • Sun, B., and Saenko, K. 2016. Deep coral: Correlation alignment for deep domain adaptation. In ECCV Workshop.
    Google ScholarLocate open access versionFindings
  • Tran, L.; Sohn, K.; Yu, X.; Liu, X.; and Chandraker, M. 2019. Gotta adapt ’em all: Joint pixel and feature-level domain adaptation for recognition in the wild. In CVPR.
    Google ScholarFindings
  • Tzeng, E.; Hoffman, J.; Zhang, N.; Saenko, K.; and Darrell, T. 2014. Deep domain confusion: Maximizing for domain invariance. CoRR abs/1412.3474.
    Findings
  • Tzeng, E.; Hoffman, J.; Saenko, K.; and Darrell, T. 2017. Adversarial discriminative domain adaptation. In CVPR.
    Google ScholarFindings
  • Xie, S.; Zheng, Z.; Chen, L.; and Chen, C. 2018. Learning semantic representations for unsupervised domain adaptation. In ICML.
    Google ScholarFindings
  • Yan, H.; Ding, Y.; Li, P.; Wang, Q.; Xu, Y.; and Zuo, W. 2017. Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation. In CVPR.
    Google ScholarFindings
  • Yosinski, J.; Clune, J.; Bengio, Y.; and Lipson, H. 2014. How transferable are features in deep neural networks? In NeurIPS.
    Google ScholarLocate open access versionFindings
  • Zhang, H.; Cisse, M.; Dauphin, Y. N.; and Lopez-Paz, D. 2018a. mixup: Beyond empirical risk minimization. In ICLR.
    Google ScholarFindings
  • Zhang, W.; Ouyang, W.; Li, W.; and Xu, D. 2018b. Collaborative and adversarial network for unsupervised domain adaptation. In CVPR.
    Google ScholarFindings
Your rating :
0

 

Tags
Comments