Public-data Assisted Private Stochastic Optimization: Power and Limitations
arxiv(2024)
摘要
We study the limits and capability of public-data assisted differentially
private (PA-DP) algorithms. Specifically, we focus on the problem of stochastic
convex optimization (SCO) with either labeled or unlabeled public data. For
complete/labeled public data, we show that any (ϵ,δ)-PA-DP has
excess risk
Ω̃(min{1/√(n_pub),1/√(n)+√(d)/nϵ}), where d is the dimension, n_pub is the number of
public samples, n_priv is the number of private samples, and
n=n_pub+n_priv. These lower bounds are established via
our new lower bounds for PA-DP mean estimation, which are of a similar form. Up
to constant factors, these lower bounds show that the simple strategy of either
treating all data as private or discarding the private data, is optimal. We
also study PA-DP supervised learning with unlabeled public samples. In
contrast to our previous result, we here show novel methods for leveraging
public data in private supervised learning. For generalized linear models (GLM)
with unlabeled public data, we show an efficient algorithm which, given
Õ(n_privϵ) unlabeled public samples, achieves the
dimension independent rate Õ(1/√(n_priv) +
1/√(n_privϵ)). We develop new lower bounds
for this setting which shows that this rate cannot be improved with more public
samples, and any fewer public samples leads to a worse rate. Finally, we
provide extensions of this result to general hypothesis classes with finite
fat-shattering dimension with applications to neural networks and non-Euclidean
geometries.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要