A new boosting design of Support Vector Machine classifiers

Efrain Tito Mayhua Lopez, Vanessa Gómez-Verdejo, Aníbal R. Figueiras-Vidal

Research output: Contribution to journalArticlepeer-review

13 Scopus citations

Abstract

Boosting algorithms pay attention to the particular structure of the training data when learning, by means of iteratively emphasizing the importance of the training samples according to their difficulty for being correctly classified. If common kernel Support Vector Machines (SVMs) are used as basic learners to construct a Real AdaBoost ensemble, the resulting ensemble can be easily compacted into a monolithic architecture by simply combining the weights that correspond to the same kernels when they appear in different learners, avoiding to increase the operation computational effort for the above potential advantage. This way, the performance advantage that boosting provides can be obtained for monolithic SVMs, i.e., without paying in classification computational effort because many learners are needed. However, SVMs are both stable and strong, and their use for boosting requires to unstabilize and to weaken them. Yet previous attempts in this direction show a moderate success. In this paper, we propose a combination of a new and appropriately designed subsampling process and an SVM algorithm which permits sparsity control to solve the difficulties in boosting SVMs for obtaining improved performance designs. Experimental results support the effectiveness of the approach, not only in performance, but also in compactness of the resulting classifiers, as well as that combining both design ideas is needed to arrive to these advantageous designs.

Original languageEnglish
Pages (from-to)63-71
Number of pages9
JournalInformation Fusion
Volume25
DOIs
StatePublished - Sep 2015
Externally publishedYes

Bibliographical note

Funding Information:
This work was supported in part by the Spanish MICINN under Grants TEC 2011-22480 and TIN 2011-24533 .

Publisher Copyright:
© 2014 Elsevier B.V.All rights reserved.

Keywords

  • Ensemble classifiers
  • Linear programming
  • Real AdaBoost
  • Subsampling
  • Support vector machines

Fingerprint

Dive into the research topics of 'A new boosting design of Support Vector Machine classifiers'. Together they form a unique fingerprint.

Cite this