Privacy-Preserving Federated Primal-Dual Learning for Non-Convex Problems With Non-Smooth Regularization

Recently, the federated learning (FL) has been a machine learning paradigm for the preservation of data privacy, though high communication cost and privacy protection are still the main concerns of FL. However, in many practical applications, the trained model needs certain nature or characteristics, such as sparseness in classification, otherwise learning performance loss is inevitable. In order to upgrade the learning performance, a suitable non-smooth regularizer (e.g., $\ell_{1}$-norm for the model sparseness) can be added to the loss function (often non-convex) in the considered optimization problem. This paper proposes a novel primal-dual learning algorithm to handle such non-smooth regularization aided non-convex FL problems, that yields much superior learning performance over some state-of-the-art FL algorithms under privacy guarantee by means of differential privacy. Finally, some experimental results are provided to demonstrate the efficacy of the proposed algorithm.