Towards Achieving Near-optimal Utility for Privacy-Preserving Federated Learning via Data Generation and Parameter Distortion
arxiv(2023)
摘要
Federated learning (FL) enables participating parties to collaboratively
build a global model with boosted utility without disclosing private data
information. Appropriate protection mechanisms have to be adopted to fulfill
the requirements in preserving \textit{privacy} and maintaining high model
\textit{utility}. The nature of the widely-adopted protection mechanisms
including \textit{Randomization Mechanism} and \textit{Compression Mechanism}
is to protect privacy via distorting model parameter. We measure the utility
via the gap between the original model parameter and the distorted model
parameter. We want to identify under what general conditions privacy-preserving
federated learning can achieve near-optimal utility via data generation and
parameter distortion. To provide an avenue for achieving near-optimal utility,
we present an upper bound for utility loss, which is measured using two main
terms called variance-reduction and model parameter discrepancy separately. Our
analysis inspires the design of appropriate protection parameters for the
protection mechanisms to achieve near-optimal utility and meet the privacy
requirements simultaneously. The main techniques for the protection mechanism
include parameter distortion and data generation, which are generic and can be
applied extensively. Furthermore, we provide an upper bound for the trade-off
between privacy and utility, \blue{which together with the lower bound provided
by no free lunch theorem in federated learning (\cite{zhang2022no}) form the
conditions for achieving optimal trade-off.
更多查看译文
关键词
federated
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要