A Unified Framework and Efficient Novel Linear Programming Variants for Sparse Support Vector Machines

Support Vector Machines (SVMs) are among the most influential machine learning methods, owing to their solid theoretical foundations and excellent generalization performance. However, the growing demand for edge intelligence, where AI capabilities are deployed directly on end devices, imposes stringent requirements on inference speed and computational efficiency. sparse learning in SVMs has therefore emerged as an important paradigm for enabling fast prediction, low resource consumption, and model parsimony. Despite its significance, current research on sparse SVMs remains fragmented in terms of theoretical analysis, model design, and application scope. This dissertation presents a systematic investigation of sparse support vector machines from both theoretical and methodological perspectives. A unified analytical framework is first established by integrating their historical evolution, structural taxonomy, sparsity-inducing mechanisms, theoretical foundations, evaluation methodologies, and application paradigms. This framework provides a coherent perspective for understanding sparse SVMs through structural, mechanistic, and methodological dimensions, and identifies key challenges in sparse learning. Building upon this foundation, a unified linear-programming-based sparse learning approach is developed for classification, robust classification, and regression tasks. By introducing ℓ1 regularization on dual variables, a family of sparse SVM models is proposed, reformulating conventional quadratic programming problems (QPPs) into linear programming problems (LPPs) while preserving convexity and promoting sample-level sparsity. This methodology is further extended to robust variants and regression settings, enabling consistent sparse learning across multiple tasks. Comprehensive theoretical analyses are conducted to investigate the sparsity mechanisms, optimization properties, computational complexity, and geometric characteristics of the proposed models. Extensive experimental results on benchmark datasets demonstrate that the proposed methods significantly reduce the number of support vectors while maintaining competitive or superior predictive performance compared with existing approaches. Overall, this dissertation establishes a unified analytical framework for sparse support vector machines and develops a linear-programming-based sparse learning methodology for classification, robust classification, and regression, providing new theoretical insights and efficient optimization methods for sparse machine learning.