Modular structured learning: an integrated framework and its applications

A new perspective to review the existing learning models by considering their structureness is proposed in this thesis. The performance of the learning models could be improved if the modular structure in the data is utilized in a proper way; we name it as the modular structured learning. This thesis mainly concerns incorporating data structures in the SVM and its extensions, although this philosophy is applicable in designing various supervised and unsupervised learning models. Guided by this general idea, several modular structured counterparts of the SVM and its extensions have been constructed. Firstly, the structured large margin machine (SLMM) is introduced as a modular structured large margin learning model. The modular data structure is extracted by agglomerative hierarchical clustering and the optimization problem is solved by converting to a sequential second order cone programming (SOCP) problem. SLMM possesses appealing features such as high classification accuracy, scalability, extensibility, and noise tolerance. Secondly, the probabilistic large margin machine (PLMM) is another modular structured large margin model by integrating the prior probabilities into the decision hyperplane calculation. We also propose an extended PLMM that is robust to estimation errors. The optimization problem involved in the PLMM can be treated as one SOCP problem, which can be solved efficiently. Thirdly, structured one-class classifier (TOCC) is designed to consider the data structure into solving one-class classification problems. TOCC is able to describe modular structured target data, so that the chance of outlier occurrence inside the boundary can be reduced. Fourthly, we construct a generic type of modular structured kernel functions, i.e., weighted Mahalanobis distance (WMD) kernels, by incorporating data specific knowledge into existing kernels. In WMD kernels, the similarity between two pattern images is determined by the Mahalanobis distance between their corresponding input patterns as well as the sizes of the clusters they reside in. Aside from testing on the benchmark datasets, the proposed modular structured learning models have been applied on two medical applications: automatic detection of breast cancers in mammograms and the functional MRI activation analysis. Experimental results show the success of using the modular structured learning approaches in solving real world problems.