DBA: Distributed Backdoor Attacks against Federated Learning

ICLR, 2020.

Cited by: 3|Bibtex|Views32|Links
EI
Keywords:
distributed backdoor attack federated learning
Weibo:
Through extensive experiments on diverse datasets including LOAN and three image datasets in different settings, we show that in standard federated learning our proposed distributed backdoor attack is more persistent and effective than centralized backdoor attack: distributed bac...

Abstract:

Backdoor attacks aim to manipulate a subset of training data by injecting adversarial triggers such that machine learning models trained on the tampered dataset will make arbitrarily (targeted) incorrect prediction on the testset with the same trigger embedded. While federated learning (FL) is capable of aggregating information provided b...More

Code:

Data:

Introduction
  • The fact of limiting access to individual party’s data due to privacy concerns or regulation constraints may facilitate backdoor attacks on the shared model trained with FL.
  • Backdoor attack is a type of data poisoning attacks that aim to manipulate a subset of training data such that machine learning models trained on the tampered dataset will be vulnerable to the test set with similar trigger embedded (Gu et al, 2019)
Highlights
  • Federated learning (FL) has been recently proposed to address the problems for training machine learning models without direct access to diverse training data, especially for privacy-sensitive tasks (Smith et al, 2017; McMahan et al, 2017; Zhao et al, 2018)
  • Through extensive experiments on several financial and image datasets and in-depth analysis, we summarize our main contributions and findings as follows. We propose a novel distributed backdoor attack strategy distributed backdoor attack on federated learning and show that distributed backdoor attack is more persistent and effective than centralized backdoor attack
  • We report a prominent phenomenon that each adversarial party is only implanted with a local trigger pattern via distributed backdoor attack, their assembled pattern attains significantly better attack performance on the global model compared with the centralized attack
  • We find that the attack success rate of centralized attack in local triggers and the global trigger drops faster than that of distributed backdoor attack, which shows that distributed backdoor attack yields a more persistent attack
  • Through extensive experiments on diverse datasets including LOAN and three image datasets in different settings, we show that in standard federated learning our proposed distributed backdoor attack is more persistent and effective than centralized backdoor attack: distributed backdoor attack achieves higher attack success rate, faster convergence and better resiliency in single-shot and multiple-shot attack scenarios
  • We demonstrate that distributed backdoor attack is more stealthy and can successfully evade two robust federated learning approaches
Methods
  • Method Random Forest Classifier Extra Tree

    Classifier XGBoost Our decision tree

    Rank 1st out prncp out prncp inv out prncp out prncp

    Rank 2nd out prncp inv out prncp recoveries out prncp inv

    Rank 3rd last pymnt amnt total rec prncp funded amnt term

    Rank 4th recoveries last pymnt amnt total rec prncp recoveries

    Rank 5th total rec prncp total pymnt inv last pymnt amnt collection recovery fee.
  • Method Random Forest Classifier Extra Tree.
  • Classifier XGBoost Our decision tree.
  • Rank 1st out prncp out prncp inv out prncp out prncp.
  • Rank 2nd out prncp inv out prncp recoveries out prncp inv.
  • Rank 3rd last pymnt amnt total rec prncp funded amnt term.
  • Rank 4th recoveries last pymnt amnt total rec prncp recoveries.
  • Rank 5th total rec prncp total pymnt inv last pymnt amnt collection recovery fee
Conclusion
  • Through extensive experiments on diverse datasets including LOAN and three image datasets in different settings, we show that in standard FL our proposed DBA is more persistent and effective than centralized backdoor attack: DBA achieves higher attack success rate, faster convergence and better resiliency in single-shot and multiple-shot attack scenarios.
  • We demonstrate that DBA is more stealthy and can successfully evade two robust FL approaches.
  • We perform an in-depth analysis on the important factors that are unique to DBA to explore its properties and limitations.
  • Our results suggest DBA is a new and more powerful attack on FL than current backdoor attacks.
Summary
  • Introduction:

    The fact of limiting access to individual party’s data due to privacy concerns or regulation constraints may facilitate backdoor attacks on the shared model trained with FL.
  • Backdoor attack is a type of data poisoning attacks that aim to manipulate a subset of training data such that machine learning models trained on the tampered dataset will be vulnerable to the test set with similar trigger embedded (Gu et al, 2019)
  • Methods:

    Method Random Forest Classifier Extra Tree

    Classifier XGBoost Our decision tree

    Rank 1st out prncp out prncp inv out prncp out prncp

    Rank 2nd out prncp inv out prncp recoveries out prncp inv

    Rank 3rd last pymnt amnt total rec prncp funded amnt term

    Rank 4th recoveries last pymnt amnt total rec prncp recoveries

    Rank 5th total rec prncp total pymnt inv last pymnt amnt collection recovery fee.
  • Method Random Forest Classifier Extra Tree.
  • Classifier XGBoost Our decision tree.
  • Rank 1st out prncp out prncp inv out prncp out prncp.
  • Rank 2nd out prncp inv out prncp recoveries out prncp inv.
  • Rank 3rd last pymnt amnt total rec prncp funded amnt term.
  • Rank 4th recoveries last pymnt amnt total rec prncp recoveries.
  • Rank 5th total rec prncp total pymnt inv last pymnt amnt collection recovery fee
  • Conclusion:

    Through extensive experiments on diverse datasets including LOAN and three image datasets in different settings, we show that in standard FL our proposed DBA is more persistent and effective than centralized backdoor attack: DBA achieves higher attack success rate, faster convergence and better resiliency in single-shot and multiple-shot attack scenarios.
  • We demonstrate that DBA is more stealthy and can successfully evade two robust FL approaches.
  • We perform an in-depth analysis on the important factors that are unique to DBA to explore its properties and limitations.
  • Our results suggest DBA is a new and more powerful attack on FL than current backdoor attacks.
Tables
  • Table1: Dataset description and parameters
  • Table2: RFA Distance and FoolsGold Weight
  • Table3: Financial Dataset Label Distribution
  • Table4: The Five Most Important Features are Similar in Different Classification Methods
Download tables as Excel
Related work
  • Federated Learning. McMahan et al (2017) first introduced federated learning (FL) to solve the distributed machine learning problem. Since the training data is never shared with the server (aggregator), FL is in favor of machine learning with privacy and regulation constraints. In this paper, we discuss and analyze our experiments in standard FL settings performed in synchronous update rounds. Advanced FL for improving communication efficacy by compressing updates using random rotations and quantization has been recently studied in Konecnyet al. (2016). Backdoor Attack on Federated Learning. Bagdasaryan et al (2018) proposed a model-poisoning approach on FL which replaced the global model with a malicious local model by scaling up the attacker’s updates. Bhagoji et al (2019) considered the case of one malicious attacker aiming to achieve both global model convergence and targeted poisoning attack, by boosting the malicious updates. They proposed two strategies, alternating minimization and estimating other benign updates, to evade the defences under weighted and non-weighted averaging for aggregation. We note that these works only consider centralized backdoor attack on FL. Robust Federated Learning. Robust FL aims to train FL models while mitigating certain attack threats. Fung et al (2018) proposed a novel defense based on the party updating diversity without limitation on the number of adversarial parties. It adds up historical updating vectors and calculate the cosine similarity among all participants to assign global learning rate for each party. Similar updating vectors will obtain lower learning rates and therefore the global model can be prevented from both label-flipping and centralized backdoor attacks. Pillutla et al (2019) proposed a robust aggregation approach by replacing the weighted arithmetic mean with an approximate geometric median, so as to minimize the impacts of “outlier” updates.
Funding
  • This work was partly supported by IBM-ILLINOIS Center for Cognitive Computing Systems Research (C3SR) – a research collaboration as part of the IBM AI Horizons Network
Reference
  • Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. How to backdoor federated learning. arXiv preprint arXiv:1807.00459, 2018.
    Findings
  • Moran Baruch, Gilad Baruch, and Yoav Goldberg. A little is enough: Circumventing defenses for distributed learning. arXiv preprint arXiv:1902.06156, 2019.
    Findings
  • Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo. Analyzing federated learning through an adversarial lens. In International Conference on Machine Learning, pp. 634–643, 2019.
    Google ScholarLocate open access versionFindings
  • Peva Blanchard, Rachid Guerraoui, Julien Stainer, et al. Machine learning with adversaries: Byzantine tolerant gradient descent. In Advances in Neural Information Processing Systems, pp. 119–129, 2017.
    Google ScholarLocate open access versionFindings
  • Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526, 2017.
    Findings
  • Nicholas Frosst and Geoffrey Hinton. Distilling a neural network into a soft decision tree. arXiv preprint arXiv:1711.09784, 2017.
    Findings
  • Clement Fung, Chris JM Yoon, and Ivan Beschastnikh. Mitigating sybils in federated learning poisoning. arXiv preprint arXiv:1808.04866, 2018.
    Findings
  • Tianyu Gu, Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Evaluating backdooring attacks on deep neural networks. IEEE Access, 7:47230–47244, 2019.
    Google ScholarLocate open access versionFindings
  • Rachid Guerraoui, Sebastien Rouault, et al. The hidden vulnerability of distributed learning in byzantium. In International Conference on Machine Learning, pp. 3518–3527, 2018.
    Google ScholarLocate open access versionFindings
  • Andrew Hard, Kanishka Rao, Rajiv Mathews, Francoise Beaufays, Sean Augenstein, Hubert Eichner, Chloe Kiddon, and Daniel Ramage. Federated learning for mobile keyboard prediction. arXiv preprint arXiv:1811.03604, 2018.
    Findings
  • Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
    Google ScholarLocate open access versionFindings
  • Wendy Kan. Lending club loan data, Mar 2019. URL https://www.kaggle.com/wendykan/lending-club-loan-data.
    Findings
  • Jakub Konecny, H Brendan McMahan, Felix X Yu, Peter Richtarik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492, 2016.
    Findings
  • Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-Efficient Learning of Deep Networks from Decentralized Data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pp. 1273–1282. PMLR, 20–22 Apr 2017.
    Google ScholarLocate open access versionFindings
  • Thomas Minka. Estimating a dirichlet distribution, 2000.
    Google ScholarFindings
  • Krishna Pillutla, Sham M. Kakade, and Zaid Harchaoui. Robust Aggregation for Federated Learning. arXiv preprint, 2019.
    Google ScholarFindings
  • Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626, 2017.
    Google ScholarLocate open access versionFindings
  • Claude E Shannon. Communication theory of secrecy systems. Bell system technical journal, 28(4): 656–715, 1949.
    Google ScholarLocate open access versionFindings
  • Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar. Federated multi-task learning. In Advances in Neural Information Processing Systems, pp. 4424–4434, 2017.
    Google ScholarLocate open access versionFindings
  • Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):12, 2019.
    Google ScholarLocate open access versionFindings
  • Timothy Yang, Galen Andrew, Hubert Eichner, Haicheng Sun, Wei Li, Nicholas Kong, Daniel Ramage, and Francoise Beaufays. Applied federated learning: Improving google keyboard query suggestions. arXiv preprint arXiv:1812.02903, 2018.
    Findings
  • Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. arXiv preprint arXiv:1806.00582, 2018.
    Findings
  • In Attack A-M, we found that if DBA poisons from scratch, the main accuracy was low and hard to converge. Therefore in three image datasets, we begin to attack when the main accuracy of global converges, which is round 10 for MNIST, 200 for CIFAR, 20 for Tiny-imagenet. As mentioned in (Bagdasaryan et al., 2018), it’s also better to attack late in Attack A-S because when the global model is converging, the updates from benign clients contain less commonly shared patterns but more individual features, which are more likely to be canceled out when aggregating and thus having less impact on the backdoor.
    Google ScholarLocate open access versionFindings
  • To evaluate DBA on irregular shape triggers, we decomposed the logo ‘ICLR’ into ‘I’, ‘C’, ‘L’, ‘R’ as local triggers on three image datasets and we decomposed the physical pattern glasses (Chen et al., 2017) into four parts as the examples shown in Fig. 14.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments