Multiclass classification using Support Vector Machine (SVM) is an ongoing research issue. SVM is mainly a binary classifier, but for classification efficiency, it is also used for multiclass classification. In multiclass classification, there are two or more classes and classification is not so easy. That’s why many methods are introduced to extend the classification efficiency of SVM. Directed Acyclic Graph Support Vector Machine (DAGSVM), Binary Tree of Support Vector Machine (BTS) and Error Correcting Output Codes (ECOC) methods are more favourable because of their computation efficiency. In the case of DAGSVM there are many improved methods like Decision Directed Acyclic Graph (DDAG), Divide-by-2 (DB2), and Weighted Directed Acyclic Graph of Support Vector Machine (WDAG SVM) have been developed. The BTS-based methods are SVM with Binary Tree Architecture, and Adaptive Binary Tree (ABT). There are many methods related to ECOC like One-Per-Class (OPC), Discriminant Error Correcting Output Codes (DECOC), and Adaptive ECOC. This paper presented a comparative and analytical survey of those methods and introduces a new model which is an improvement over the existing DAGSVM methods. This model uses Gaussian Mixture Model, K-Means Clustering and Naïve Bayes Classifier for data classification. This model can give better results than existing DAGSVM methods.

Multiclass SVM, Directed Acyclic Graph SVM, Binary Tree SVM, Error Correcting Output Codes.

[1]. Zhang, Xian-Da, and Xian-Da Zhang. "Support vector machines." A Matrix Algebra Approach to Artificial Intelligence: pp.617-679, 2020.
[2]. James, Gareth, et al. "Support vector machines." An introduction to statistical learning: with applications in R: pp.367-402, 2021.
[3]. Cervantes, Jair, et al. "A comprehensive survey on support vector machine classification: Applications, challenges and trends." Neurocomputing 408: pp.189-215, 2020.
[4]. Schölkopf, Bernhard, Alexander J. Smola, and Francis Bach. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, 2002.
[5]. Nalepa, Jakub, and Michal Kawulok. "Selecting training sets for support vector machines: a review." Artificial Intelligence Review 52.2: pp.857-900, 2019.
[6]. Das, Subhankar, and Sanjib Saha. "Data mining and soft computing using support vector machine: A survey." International Journal of Computer Applications 77.14, 2013.
[7]. Datta, R. P., and Sanjib Saha. "Applying rule-based classification techniques to medical databases: an empirical study." International Journal of Business Intelligence and Systems Engineering 1.1: pp.32-48, 2016.
[8]. Saha, Sanjib, and Debashis Nandi. "Data Classification based on Decision Tree, Rule Generation, Bayes and Statistical Methods: An Empirical Comparison." Int. J. Comput. Appl 129.7: pp.36-41, 2015.
[9]. Saha, Sanjib. "Non-rigid Registration of De-noised Ultrasound Breast Tumors in Image Guided Breast-Conserving Surgery." Intelligent Systems and Human Machine Collaboration. Springer, Singapore, pp.191-206, 2023.
[10]. Saha, Sanjib, et al. "ADU-Net: An Attention Dense U-Net based deep supervised DNN for automated lesion segmentation of COVID-19 from chest CT images." Biomedical Signal Processing and Control 85: 104974, 2023.
[11]. Platt, John, Nello Cristianini, and John Shawe-Taylor. "Large margin DAGs for multiclass classification." Advances in neural information processing systems 12, 1999.
[12]. Vural, Volkan, and Jennifer G. Dy. "A hierarchical method for multi-class support vector machines." Proceedings of the twenty-first international conference on Machine learning. 2004.
[13]. Fei, Liu, et al. "A peer-to-peer hypertext categorization using directed acyclic graph support vector machines." Parallel and Distributed Computing: Applications and Technologies: 5th International Conference, PDCAT 2004, Singapore, December 8-10, 2004. Proceedings. Springer Berlin Heidelberg, 2005.
[14]. Sabzekar, Mostafa, et al. "Improved DAG SVM: A New Method for Multi-Class SVM Classification." IC-AI. 2009.
[15]. Yi, Hui, Xiaofeng Song, and Bin Jiang. "Structure selection for DAG-SVM based on misclassification cost minimization." International Journal of Innovative Computing, Information & Control 7.9: pp.5133-5143, 2011.
[16]. Brunner, Carl, et al. "Pairwise support vector machines and their application to large scale problems." The Journal of Machine Learning Research 13.1 (2012): 2279-2292, ©2012 Carl Brunner, Andreas Fischer, Klaus Luig and Thorsten Thies.
[17]. Takahashi, Fumitake, and Shigeo Abe. "Optimizing directed acyclic graph support vector machines." Artificial Neural Networks in Pattern Recognition (ANNPR): pp.166-173, 2003.
[18]. Cheong, Sungmoon, Sang Hoon Oh, and Soo-Young Lee. "Support vector machines with binary tree architecture for multi-class classification." Neural Information Processing-Letters and Reviews 2.3: pp.47-51, 2004.
[19]. Zhang, Gexiang, and Weidong Jin. "Automatic construction algorithm for multi-class support vector machines with binary tree architecture." International Journal of Computer Science and Network Security 6.2A: pp.119-126, 2006.
[20]. Fei, Ben, and Jinbai Liu. "Binary tree of SVM: a new fast multiclass training and classification algorithm." IEEE transactions on neural networks 17.3: pp.696-704, 2006.
[21]. Chen, Jin, Cheng Wang, and Runsheng Wang. "Combining support vector machines with a pairwise decision tree." IEEE Geoscience and Remote Sensing Letters 5.3: pp.409-413, 2008.
[22]. Madzarov, Gjorgji, and Dejan Gjorgjevikj. "Multi-class classification using support vector machines in decision tree architecture." IEEE EUROCON 2009. IEEE, 2009.
[23]. Chen, Jin, Cheng Wang, and Runsheng Wang. "Adaptive binary tree for fast SVM multiclass classification." Neurocomputing 72.13-15: pp.3370-3375, 2009.
[24]. Sidaoui, Boutkhil, and Kaddour Sadouni. "Efficient binary tree multiclass svm using genetic algorithms for vowels recognition." Proceedings of the 10th WSEAS international conference on Computational Intelligence, Man-Machine Systems and Cybernetics, and proceedings of the 10th WSEAS international conference on Information Security and Privacy. 2011.
[25]. Madzarov, Gjorgji, and Dejan Gjorgjevikj. "Evaluation of distance measures for multi-class classification in binary svm decision tree." Artificial Intelligence and Soft Computing: 10th International Conference, ICAISC 2010, Zakopane, Poland, June 13-17, 2010, Part I 10. Springer Berlin Heidelberg, 2010.
[26]. Dietterich, Thomas G., and Ghulum Bakiri. "Solving multiclass learning problems via error-correcting output codes." Journal of artificial intelligence research 2: pp.263-286, 1994.
[27]. Aha, Divid W., and Richard L. Bankert. "Cloud classification using error-correcting output codes." Ai Applications 11.1: pp.13-28, 1997.
[28]. Kittler, Josef, et al. "Face verification using error correcting output codes." Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001. Vol.1. IEEE, 2001.
[29]. Passerini, Andrea, Massimiliano Pontil, and Paolo Frasconi. "New results on error correcting output codes of kernel machines." IEEE transactions on neural networks 15.1: pp.45-54, 2004.
[30]. Pujol, Oriol, Petia Radeva, and Jordi Vitria. "Discriminant ECOC: A heuristic method for application dependent design of error correcting output codes." IEEE Transactions on Pattern Analysis and Machine Intelligence 28.6 (2006): 1007-1012.
[31]. Zhang, Hongming, et al. "Robust multi-view face detection using error correcting output codes." Computer Vision–ECCV 2006: 9th European Conference on Computer Vision, Graz, Austria, May 7-13, 2006, Proceedings, Part IV 9. Springer Berlin Heidelberg, 2006.
[32]. Zhong, Guoqiang, and Mohamed Cheriet. "Adaptive error-correcting output codes." Twenty-Third International Joint Conference on Artificial Intelligence. 2013.
[33]. Reynolds, Douglas A. "Gaussian mixture models." Encyclopedia of biometrics 741.pp.659-663, 2009.
[34]. Dempster, Arthur P., Nan M. Laird, and Donald B. Rubin. "Maximum likelihood from incomplete data via the EM algorithm." Journal of the royal statistical society: series B (methodological) 39.1 (1977): pp.1-22, 1977.
[35]. MacQueen, J. "Classification and analysis of multivariate observations." 5th Berkeley Symp. Math. Statist. Probability. Los Angeles LA USA: University of California, 1967.
[36]. Murphy, Kevin P. "Naive bayes classifiers." University of British Columbia 18.60: pp.1-8, 2006.
[37]. WEKA3 tool for machine learning and knowledge analysis. Online available at http://www.cs.waikato.ac.nz/~ml/weka/
[38]. Blake, C. and Merz, C. J. "UCI repository of machine learning datasets." University of California, Irvine, Dept. of Information and Computer Sciences.(http://www.cs.waikato.ac.nz/~ml/weka/)