Fine-grained Analysis of Non-Parametric Estimation for Pairwise Learning
arXiv ยท Machine Learning
City University of Hong Kong
Abstract
In this paper, we are concerned with the generalization performance ofnon-parametric estimation for pairwise learning. Most of the existing workrequires the hypothesis space to be convex or a VC-class, and the loss to beconvex. However, these restrictive assumptions limit the applicability of theresults in studying many popular methods, especially kernel methods and neuralnetworks. We significantly relax these restrictive assumptions and establish asharp oracle inequality of the empirical minimizer with a general hypothesisspace for the Lipschitz continuous pairwise losses. Our results can be used tohandle a wide range of pairwise learning problems including ranking, AUCmaximization, pairwise regression, and metric and similarity learning. As anapplication, we apply our general results to study pairwise least squaresregression and derive an excess generalization bound that matches the minimaxlower bound for pointwise least squares regression up to a logrithmic term. Thekey novelty here is to construct a structured deep ReLU neural network as anapproximation of the true predictor and design the targeted hypothesis spaceconsisting of the structured networks with controllable complexity. Thissuccessful application demonstrates that the obtained general results indeedhelp us to explore the generalization performance on a variety of problems thatcannot be handled by existing approaches.
MoreTranslated text
PDF
View via Publisher
AI Read Science
Must-Reading Tree
Example

Generate MRT to find the research sequence of this paper
Data Disclaimer
The page data are from open Internet sources, cooperative publishers and automatic analysis results through AI technology. We do not make any commitments and guarantees for the validity, accuracy, correctness, reliability, completeness and timeliness of the page data. If you have any questions, please contact us by email: report@aminer.cn
Chat Paper
Summary is being generated by the instructions you defined