An efficient weighted Lagrangian twin support vector machine for imbalanced data classification
Pattern RecognitionPublished 25 March 2014
Yuan‐Hai Shao, Wei-Jie Chen, Jingjing Zhang, Zhen Wang, Nai-Yang Deng
Citations157
SJR quartileQ1
SJR score2.06
SNIP2.67
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A graph based under-sampling strategy is introduced to keep the proximity information, which is robustness to outliers, and the weight biases are embedded in the Lagrangian TWSVM formulations, which overcomes the bias phenomenon.
Abstract
In this paper, we propose an efficient weighted Lagrangian twin support vector machine (WLTSVM) for the imbalanced data classification based on using different training points for constructing the two proximal hyperplanes. The main contributions of our WLTSVM are: (1) a graph based under-sampling strategy is introduced to keep the proximity information, which is robustness to outliers, (2) the weight biases are embedded in the Lagrangian TWSVM formulations, which overcomes the bias phenomenon in the original TWSVM for the imbalanced data classification, (3) the convergence of the training procedure of Lagrangian functions is proven and (4) it is tested and compared with some other TWSVMs on synthetic and real datasets to show its feasibility and efficiency for the imbalanced data classification.
Keywords
Computer Science
ACM Transactions on Intelligent Systems and TechnologyLIBSVM
41,340 Citations2011Chih-Chung Chang, Chih‐Jen Lin
Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.
Machine LearningSupport-vector networks
40,463 Citations1995Corinna Cortes, Vladimir Vapnik
High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
Machine LearningSupport-Vector Networks
33,035 Citations1995Corinna Cortes, Vladimir Vapnik
High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
Journal of Artificial Intelligence ResearchSMOTE: Synthetic Minority Over-sampling Technique
31,194 Citations2002Nitesh V. Chawla, Kevin W. Bowyer +2 more
A combination of the method of oversampling the minority (abnormal) class and under-sampling the majority class can achieve better classifier performance (in ROC space) and a combination of these methods and the area under the Receiver Operating Characteristic curve (AUC) and the ROC convex hull strategy is evaluated.
Data Mining and Knowledge DiscoveryA Tutorial on Support Vector Machines for Pattern Recognition
16,433 Citations1998Christopher J. C. Burges
There are several arguments which support the observed high accuracy of SVMs, which are reviewed and numerous examples and proofs of most of the key theorems are given.
now publishers, Inc. eBooksDistributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers
13,365 Citations2010Stephen Boyd
It is argued that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas.
Neural Processing LettersLeast Squares Support Vector Machine Classifiers
9,375 Citations1999Johan A. K. Suykens, Joos Vandewalle
A least squares version for support vector machine (SVM) classifiers that follows from solving a set of linear equations, instead of quadratic programming for classical SVM's.
LIBLINEAR: A Library for Large Linear Classification
6,625 Citations2008Rong-En Fan, Kai‐Wei Chang +3 more
The MIT Press eBooksFast Training of Support Vector Machines Using Sequential Minimal Optimization
5,462 Citations1998John Platt
Advanced textbooks in control and signal processingPattern Classification
5,015 Citations2001Leo H. Chiang, Evan L. Russell +1 more
WORLD SCIENTIFIC eBooksLeast Squares Support Vector Machines
3,640 Citations2002Johan A. K. Suykens, Tony Van Gestel +3 more
Intelligent Data AnalysisThe class imbalance problem: A systematic study1
3,298 Citations2002Nathalie Japkowicz, Shaju Stephen
The assumption that the class imbalance problem does not only affect decision tree systems but also affects other classification systems such as Neural Networks and Support Vector Machines is investigated.
Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples
3,266 Citations2006Mikhail Belkin, Partha Niyogi +1 more
A semi-supervised framework that incorporates labeled and unlabeled data in a general-purpose learner is proposed and properties of reproducing kernel Hilbert spaces are used to prove new Representer theorems that provide theoretical basis for the algorithms.
IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics)Exploratory Undersampling for Class-Imbalance Learning
2,476 Citations2008Xuying Liu, Jianxin Wu +1 more
Experimental results show that the proposed EasyEnsemble and BalanceCascade algorithms have higher Area Under the ROC Curve, F-measure, and G-mean values than many existing class-imbalance learning methods.
IEEE Transactions on Pattern Analysis and Machine IntelligenceTwin Support Vector Machines for Pattern Classification
1,780 Citations2007Jayadeva, R. Khemchandani +1 more
A binary SVM classifier that determines two nonparallel planes by solving two related SVM-type problems, each of which is smaller than in a conventional SVM, which shows good generalization on several benchmark data sets.
IEEE Transactions on Knowledge and Data EngineeringTraining cost-sensitive neural networks with methods addressing the class imbalance problem
1,316 Citations2006Zhi‐Hua Zhou, Xuying Liu
The empirical study suggests that some methods that have been believed to be effective in addressing the class imbalance problem may, in fact, only be effective on learning with imbalanced two-class data sets.
NeurocomputingWeighted least squares support vector machines: robustness and sparse approximation
1,249 Citations2002Johan A. K. Suykens, Jos De Brabanter +2 more
The methods of this paper are illustrated for RBF kernels and demonstrate how to obtain robust estimates with selection of an appropriate number of hidden units, in the case of outliers or non-Gaussian error distributions with heavy tails.
Lecture notes in computer scienceApplying Support Vector Machines to Imbalanced Datasets
1,132 Citations2004Rehan Akbani, Stephen Kwek +1 more
An algorithm is proposed based on a variant of the SMOTE algorithm by Chawla et al, combined with Veropoulos et al's different error costs algorithm for overcoming problems of imbalanced datasets in which negative instances heavily outnumber the positive instances.
IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics)SVMs Modeling for Highly Imbalanced Classification
951 Citations2008Yuchun Tang, Yanqing Zhang +2 more
Of the four SVM variations considered in this paper, the novel granular SVMs-repetitive undersampling algorithm (GSVM-RU) is the best in terms of both effectiveness and efficiency.
IEEE Transactions on Pattern Analysis and Machine IntelligenceAsymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval
876 Citations2006Dacheng Tao, Xiaoou Tang +2 more
An asymmetric bagging and random subspace SVM (ABRS-SVM) is built to solve three problems and further improve the relevance feedback performance.
Proximal support vector machine classifiers
844 Citations2001Glenn Fung, O. L. Mangasarian
Computational results on publicly available datasets indicate that the proposed proximal SVM classifier has comparable test set correctness to that of standard S VM classifiers, but with considerably faster computational time that can be an order of magnitude faster.
IEEE Transactions on Pattern Analysis and Machine IntelligenceMultisurface proximal support vector machine classification via generalized eigenvalues
792 Citations2006O. L. Mangasarian, Edward W. Wild
Tests on simple examples as well as on a number of public data sets show the advantages of the proposed approach in both computation time and test set correctness.
Evaluating Learning Algorithms: A Classification Perspective
684 Citations2011Nathalie Japkowicz, Mohak Shah
This book examines various aspects of the evaluation process with an emphasis on classification algorithms, describing several techniques for classifier performance assessment, error estimation and resampling, obtaining statistical significance as well as selecting appropriate domains for evaluation.
IEEE Transactions on Neural NetworksImprovements on Twin Support Vector Machines
559 Citations2011Yuan‐Hai Shao, Chunhua Zhang +2 more
An improved version of the TBSVM is proposed, named twin bounded support vector machines (TBSVM), based on TWSVM, that the structural risk minimization principle is implemented by introducing the regularization term.
On Manifold Regularization.
323 Citations2005Misha Belkin, Partha Niyogi +1 more
A semisupervised framework that incorporates labeled and unlabeled data in a generalpurpose learner and gives rise to a regularized form of spectral clustering with an out-of-sample extension is proposed.
Machine LearningMulticategory Proximal Support Vector Machine Classifiers
318 Citations2005Glenn Fung, O. L. Mangasarian
This work proposes balancing the k classes and a novel Newton refinement modification to PSVM in order to deal with the resulting two-class problems, which are often very unbalanced, leading in some cases to poor performance.
Fuzzy Sets and SystemsA study of the behaviour of linguistic fuzzy rule based classification systems in the framework of imbalanced data-sets
310 Citations2008Alberto Fernández, Salvador García +2 more
The necessity of applying a preprocessing step to deal with the problem of imbalanced data-sets is analyzed and the granularity of the fuzzy partitions, the use of distinct conjunction operators, the application of some approaches to compute the rule weights and theUse of different fuzzy reasoning methods are analyzed.
Pattern RecognitionRobust twin support vector machine for pattern classification
293 Citations2012Zhiquan Qi, Yingjie Tian +1 more
A new robust twin support vector machine (called R-TWSVM) via second order cone programming formulations for classification, which can deal with data with measurement noise efficiently and successfully overcomes the existing shortcomings of TWSVM is proposed.
Knowledge and Information SystemsBoosting support vector machines for imbalanced data sets
279 Citations2009Benjamin Wang, Nathalie Japkowicz
This work uses support vector machines with soft margins as the base classifier and counter the excessive bias introduced by this approach with a boosting algorithm, and finds that this ensemble of SVMs makes an impressive improvement in prediction performance.
CERN Document Server (European Organization for Nuclear Research)Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions
277 Citations2012Nai-Yang Deng, Yingjie Tian +1 more
IEEE Transactions on CyberneticsNonparallel Support Vector Machines for Pattern Classification
255 Citations2014Yingjie Tian, Zhiquan Qi +3 more
Least squares support vector machine classifiers: a large scale algorithm
221 Citations1999Johan A. K. Suykens, Luděk Lukáš +2 more
An iterative training algorithm for LS-SVM's which is based on a conjugate gradient method which enables solving large scale classification problems which is illustrated on a multi two-spiral benchmark problem.
IEEE Transactions on Knowledge and Data EngineeringText Document Preprocessing with the Bayes Formula for Classification Using the Support Vector Machine
217 Citations2008Dino Isa, Lam Hong Lee +2 more
This work implements an enhanced hybrid classification method through the utilization of the naive Bayes approach and the support vector machine (SVM) that shows a significant reduction in training time as compared to the Lsquare method and significant improvement in the classification accuracy when compared to pure naive BayES systems and also the TF-IDF/SVM hybrids.
Neural NetworksLaplacian twin support vector machine for semi-supervised classification
213 Citations2012Zhiquan Qi, Yingjie Tian +1 more
A novel Laplacian Twin Support Vector Machine (called Lap-TSVM) is proposed for the semi-supervised classification problem, which can exploit the geometry information of the marginal distribution embedded in unlabeled data to construct a more reasonable classifier and be a useful extension of TSVM.
Technological and Economic Development of EconomyRECENT ADVANCES ON SUPPORT VECTOR MACHINES RESEARCH
196 Citations2012Yingjie Tian, Yong Shi +1 more
The purpose of this paper is to understand SVM from the optimization point of view, review several representative optimization models in SVMs, their applications in economics, in order to promote the research interests in both optimization-based SVMs theory and economics applications.
Protein Engineering Design and SelectionWeighted-support vector machines for predicting membrane protein types based on pseudo-amino acid composition
184 Citations2004M. Wang, Jing Yang +3 more
The spectral analysis technique is introduced to represent the statistical sample of a protein and the weighted support vector machine (SVM) algorithm is applied, showing remarkable power in dealing with the bias caused by the situation when one subset in the training dataset contains many more samples than the other.
Information SciencesNonparallel hyperplane support vector machine for binary classification problems
162 Citations2013Yuan‐Hai Shao, Wei-Jie Chen +1 more
The proposed NHSVM is formulated by clustering the training points according to the similarity between classes, and is consistent between its predicting and training processes - an essential difference that distinguishes it from other nonparallel SVMs.
Weighted support vector machine for classification with uneven training class sizes
120 Citations2005Yimin Huang, Shuxin Du
Weighted support vector machines for classification are proposed in this paper where penalty of misclassification for each training sample is different and the effectiveness of the proposed methods on breast cancer diagnosis is shown.
Knowledge-Based SystemsA regularization for the projection twin support vector machine
104 Citations2012Yuan‐Hai Shao, Zhen Wang +2 more
This work proposes a simpler and reasonable variant from theoretical point of view, called projection twin support vector machine with regularization term, RPTSVM, which reformulates the primal problems of PTSVM by adding a maximum margin regularizationterm, and, therefore, the singularity is avoided and the regularized risk principle is implemented.
Expert Systems with ApplicationsA new weighted approach to imbalanced data classification problem via support vector machine with quadratic cost function
79 Citations2011Jae Pil Hwang, Seongkeun Park +1 more
A new weighted approach on Lagrangian support vector machine for imbalanced data classification problem is proposed and the weight parameters are embedded in the Lagrangia SVM formulation.
Rare & Special e-Zone (The Hong Kong University of Science and Technology)Efficient variable selection in support vector machines via the alternating direction method of multipliers
52 Citations2011Gui–Bo Ye, Yifei Chen +1 more
This work develops an efficient algorithm based on the alternating direction method of multipliers (ADMM) to solve the optimization problem in DrSVM, a doubly regularized support vector machine.
Protein and Peptide LettersImproved Prediction of Palmitoylation Sites Using PWMs and SVM
33 Citations2011Yuxin Li, Yuan‐Hai Shao +1 more
PPWMs, a computational predictor using Position Weight Matrices (PWMs) encoding scheme and support vector machine (SVM) for identifying protein palmitoylation sites shows a nice predictive performance and comparison results show the superiority of PPWMs over two existing widely known palMIToylation site predictors CSS-Palm 2.0 and CKSAAP-palm in many cases.
Knowledge-Based SystemsA proximal classifier with consistency
30 Citations2013Yuan‐Hai Shao, Nai-Yang Deng +1 more
This paper proposes a new proximal classifier, called PCC for short, with consistency, which is always based on comparing two distances between a point and two hyperplanes, which naturally leads to a simpler decision function with less computation cost.
NeurocomputingProximal support vector machine using local information
29 Citations2009Xubing Yang, Songcan Chen +2 more
A new supervised learning method called proximal support vector machine using local information (LIPSVM) is proposed, which not only keeps aforementioned characteristics of GEPSVM, but also has its additional advantages.