Show simple item record

dc.contributor.advisorRay, Indrakshi
dc.contributor.advisorChitsaz, Hamidreza
dc.contributor.authorAlnnfiai, Majdi Khalid
dc.contributor.committeememberPallickara, Sangmi Lee
dc.contributor.committeememberAboellail, Tawfik
dc.date.accessioned2019-09-10T14:35:24Z
dc.date.available2019-09-10T14:35:24Z
dc.date.issued2019
dc.description2019 Summer.
dc.descriptionIncludes bibliographical references.
dc.description.abstractClassification is a major task in machine learning and data mining applications. Many of these applications involve building a classification model using a large volume of imbalanced data. In such an imbalanced learning scenario, the area under the ROC curve (AUC) has proven to be a reliable performance measure to evaluate a classifier. Therefore, it is desirable to develop scalable learning algorithms that maximize the AUC metric directly. The kernelized AUC maximization machines have established a superior generalization ability compared to linear AUC machines. However, the computational cost of the kernelized machines hinders their scalability. To address this problem, we propose a large-scale nonlinear AUC maximization algorithm that learns a batch linear classifier on approximate feature space computed via the k-means Nyström method. The proposed algorithm is shown empirically to achieve comparable AUC classification performance or even better than the kernel AUC machines, while its training time is faster by several orders of magnitude. However, the computational complexity of the linear batch model compromises its scalability when training sizable datasets. Hence, we develop a second-order online AUC maximization algorithms based on a confidence-weighted model. The proposed algorithms exploit the second-order information to improve the convergence rate and implement a fixed-size buffer to address the multivariate nature of the AUC objective function. We also extend our online linear algorithms to consider an approximate feature map constructed using random Fourier features in an online setting. The results show that our proposed algorithms outperform or are at least comparable to the competing online AUC maximization methods. Despite their scalability, we notice that online first and second-order AUC maximization methods are prone to suboptimal convergence. This can be attributed to the limitation of the hypothesis space. A potential improvement can be attained by learning stochastic online variants. However, the vanilla stochastic methods also suffer from slow convergence because of the high variance introduced by the stochastic process. We address the problem of slow convergence by developing a fast convergence stochastic AUC maximization algorithm. The proposed stochastic algorithm is accelerated using a unique combination of scheduled regularization update and scheduled averaging. The experimental results show that the proposed algorithm performs better than the state-of-the-art online and stochastic AUC maximization methods in terms of AUC classification accuracy. Moreover, we develop a proximal variant of our accelerated stochastic AUC maximization algorithm. The proposed method applies the proximal operator to the hinge loss function. Therefore, it evaluates the gradient of the loss function at the approximated weight vector. Experiments on several benchmark datasets show that our proximal algorithm converges to the optimal solution faster than the previous AUC maximization algorithms.
dc.format.mediumborn digital
dc.format.mediumdoctoral dissertations
dc.identifierAlnnfiai_colostate_0053A_15476.pdf
dc.identifier.urihttps://hdl.handle.net/10217/197256
dc.languageEnglish
dc.publisherColorado State University. Libraries
dc.relation.ispartof2000-2019 - CSU Theses and Dissertations
dc.rightsCopyright of the original work is retained by the author.
dc.titleScalable large margin pairwise learning algorithms
dc.typeText
dcterms.rights.dplaThe copyright and related rights status of this Item has not been evaluated (https://rightsstatements.org/vocab/CNE/1.0/). Please refer to the organization that has made the Item available for more information.
thesis.degree.disciplineComputer Science
thesis.degree.grantorColorado State University
thesis.degree.levelDoctoral
thesis.degree.nameDoctor of Philosophy (Ph.D.)


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record