Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-745bb68f8f-b95js Total loading time: 0 Render date: 2025-01-11T22:29:31.712Z Has data issue: false hasContentIssue false

7 - Binary Classification with Support Vector Machines

Published online by Cambridge University Press:  05 December 2012

By
Patrick Nichols ,
Bobbie-Jo Webb-Robertson  and
Christopher Oehmen
Edited by
Ian Gorton  and
Deborah K. Gracio
Patrick Nichols
Affiliation:
Pacific Northwest National Laboratory
Bobbie-Jo Webb-Robertson
Affiliation:
Pacific Northwest National Laboratory
Christopher Oehmen
Affiliation:
Pacific Northwest National Laboratory
Ian Gorton
Affiliation:
Pacific Northwest National Laboratory, Washington
Deborah K. Gracio
Affiliation:
Pacific Northwest National Laboratory, Washington
Get access

Summary

Introduction

Support vector machines (SVM) are currently one of the most popular and accurate methods for binary data classification and prediction. They have been applied to a variety of data and situations such as cyber-security, bioinformatics, web searches, medical risk assessment, financial analysis, and other areas [1]. This type of machine learning is shown to be accurate and is able to generalize predictions based upon previously learned patterns. However, current implementations are limited in that they can only be trained accurately on examples numbering to the tens of thousands and usually run only on serial computers. There are exceptions. A prime example is the annual machine learning and classification competitions such as the International Conference on Artificial Neural Networks (ICANN), which present problems with more than 100,000 elements to be classified. However, in order to treat such large test cases the formalism of the support vector machines must be modified.

SVMs were first developed by Vapnik and collaborators [2] as an extension to neural networks. Assume that we can convert the data values associated with an entity into numerical values that form a vector in the mathematical sense. These vectors form a space. Also, assume that this space of vectors can be separated by a hyperplane into the vectors that belong to one class and those that form the opposing class.

Type
Chapter
Information
Data-Intensive Computing
Architectures, Algorithms, and Applications
 , pp. 157 - 179
Publisher: Cambridge University Press
Print publication year: 2012

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1. Christianini, N. and Shawe-Taylor, J.An Introduction to Support Vector Machines and other Kernel Based Learning Methods. Cambridge, UK: Cambridge University Press, 2000.CrossRefGoogle Scholar
2. Boser, B. E., Guyon, I. M., and V. N., Vapnik. “A Training Algorithm for Optimal Margin Classifiers.” In Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, edited by D., Haussler, 144–52. New York: ACM Press, 1992.Google Scholar
3. Jin, J. “Impossibility of Successful Classification When Useful Features are Rare and Weak.” Proceedings of the National Academy of Sciences of the United States of America 106, no. 22 (2009): 8859–64.CrossRefGoogle ScholarPubMed
4. Platt, J. “Using Analytic QP and Sparseness to Speed Training of Support Vector Machines.” Proc. Advances in Neural Information Processing Systems 11, 557–63. Cambridge, MA: MIT Press, 1999.Google Scholar
5. Brown, M. P. S., Grundy, W. N., Lin, D., Cristianini, N., Sugnet, Furey Jr., , C., , T. S., Ares, M., and Haussler, D.Knowledge-Based Analysis of Microarray Gene Expression Data Using Support Vector Machines. Proceedings of the National Academy of Science 97, no. 1 (2000): 262–67.CrossRefGoogle ScholarPubMed
6. Frieß, T. T., Cristianini, N., and Campbell, C.The Kernel Adatron: a Fast and Simple Learning Procedure for Support Vector Machines. Proceedings of the Fifteenth International Conference on Machine Learning. Madison, Wisconsin: Morgan Kaufmann, 1998.Google Scholar
7. Opper, M. “Learning Times of Neural Networks: Exact Solution for a Perceptron Algorithm.” Physical Review A 38 (1988): 3824–26.CrossRefGoogle ScholarPubMed
8. Chang, E., Zhu, K., Wang, H., Bai, H., Li, J., Qiu, Z., and Cui, H. 2008. “Parallelizing support vector machines on distributed computers.” In Advances in Neural Information Processing Systems 20, J., Platt, D., Koller, Y., Singer, and S., Roweis Eds., MIT Press, Cambridge, MA, 257–64.Google Scholar
9. PSVM, accessed August 2010, http://code.google.com/p/psvm.
10. Fine, S., Scheinberg, K., Cristianini, N., Shawe-Taylor, J., and Williamson, B. “Efficient SVMTraining Using Low-Rank Kernel Representations1.” Journal of Machine Learning Research 2 (2001): 243–64.Google Scholar
11. Osuna, R., Freund, R., and Girosi, F. “Training Support VectorMachines: an Application to Face Detection.” In Proceedings of the Conference on Computer Vision and Pattern Recognition, 130, 1997.CrossRefGoogle Scholar
12. Dai, Y. H., and Fletcher, R.New Algorithms for Singly Linearly Constrained Quadratic Programs Subject to Lower and Upper Bounds. Research Report NA/216, Department of Mathematics, University of Dundee, 2003.Google Scholar
13. Zanghirati, G., and Zanni, L. “A Parallel Solver for Large Quadratic Programs in Training Support Vector Machines.” Parallel Computing 29 (2003): 535–51.CrossRefGoogle Scholar
14. Serafini, T.Zanghirati, G., and Zanni, L. “Gradient Projection Methods for Large Quadratic Programs and Applications in Training Support Vector Machines.” Optim. Meth. Soft. 20 (2005): 353–78.CrossRefGoogle Scholar
15. Joachims, T. “Making Large-Scale SVMLearning Practical.” In Advances in Kernel Methods – Support Vector Learning, edited by B., Schölkopf, C., Burges, and A., Smola. Boston: MIT-Press, 1999.Google Scholar
16. Platt, John C. “Fast Training of SupportVector Machines Using Sequential Minimal Optimization.” In Advances in Kernel Methods – Support Vector Learning, edited by B., Schölkopf, C., Burges, and A., Smola. Boston: MIT-Press, 1998.Google Scholar
17. Keerthi, S. S., Shevade, S. K., Bhattacharyya, C., and Murthy, K. R. K. “Improvements to Platt's SMOAlgorithm for SVMClassifier Design.” Neural Computation 13, no. 3 (2001): 637.CrossRefGoogle Scholar
18. Fan, R. E., Chen, P. H., and Lin, C. J. “Working set selection using second order information for training SVM.” Journal of Machine Learning Research 6, (2005): 1889–1918.Google Scholar
19. Chih-Chung, Chang and Chih-Jen, Lin. “LIBSVM: A Library for Support Vector Machines,” accessed August 2010, http://www.csie.ntu.edu.tw/~cjlin/libsvm.
20. Cao, L. J., Ong, C. J., Zhang, J. Q., Periyathamby, U., Fu, X. J., and Lee, H. P. “Parallel Sequential Minimal Optimization for the Training of Support Vector Machines,” IEEE Transactions on Neural Networks 17, no. 4 (2006): 1039–49.CrossRefGoogle ScholarPubMed
21. MILDE, accessed August 2010, http://www.nec-labs.com/research/machine/mlwebsite/software.php?project=milde.
22. Woodsend, K., and Gondzio, J. “Hybrid MPI/OpenMPI Parallel Support Vector Maschine Training.” Journal of Machine Learning Research 10 (2009): 1937.Google Scholar
23. UCI Machine Learning Repository, University of California, School of Information and Computer Science, accessed May 2009, http://www.ics.uci.edu/~mlearn/MLRepository.html.
24. SVMLink from John Platt's web page, accessed August 2010, http://research.microsoft.com/enus/projects/svm/default.aspx.
25. Webb-Robertson, B-J.Oehmen, C., and Shah, A. “A Feature Vector Integration Approach for a Generalized Support Vector Machine Pairwise Homology Algorithm.” Computational Biology and Chemistry 32, no. 6 (2008): 458–61.CrossRefGoogle ScholarPubMed

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×