Support Vector Machines (SVMs) are used in large-scale linear and non-linear non-probabilistic binary or multi-class classification. Classification using SVM techniques gives better accuracy than other machine learning classification methods. Various Support Vector Classification (SVC) algorithms are available in the literature, and many researchers are facing the problem of choosing the best methods for real-world applications. This paper integrates LibSVM and LibLINEAR tools with the Weka tool. The Radial Basis Function (RBF), Polynomial, Sigmoid and Linear kernel-based C-SVC and nu-SVC models, as well as predictive linear SVM models, are applied to six UCI machine learning datasets. The presentations of various SVC methods are empirically matched using Classification Accuracy (CA), Root Mean Square Error (RMSE), and Area Under Curve (AUC) metrics. The proposed method for this article is RBF kernel and linear kernel in C-SVC and nu-SVC models. The performance of the proposed models is trained and tested with UCI machine learning datasets for non-linear and linear classification. The results are compared with state-of-the-art SVC models. RBF kernel in C-SVC and nu-SVC models has achieved an accuracy of 97.3% and 98%, respectively, for non-linear classification on the Iris dataset. The linear kernel in C-SVC and nu-SVC models has achieved 96.6% and 98% accuracy for linear classification on the Iris dataset. L2-Regularized L2-Hinge Loss dual and primal SVC model has a classification accuracy of 96% for large-scale linear classification on the Iris dataset. Therefore, some conclusions based on overall performances on six datasets are as follows. (i) RBF kernel-based C-SVC model performs better than other non-linear SVC methods. (ii) Linear kernel-based C-SVC and nu-SVC methods perform better in the case of linear classification. (iii) In large-scale linear classification, L1-Regularized L2-Loss SVC, Multi-class SVC by Crammer Singer and L2-Regularized L2-Loss SVC methods perform better than other linear SVC methods. (iv) Most of these methods give good results in the case of datasets having all the most numeric attributes or dimensions and a large number of instances or vectors.

Support Vector Machines, Kernel Functions, Regularized Losses, Classification

[1]. Vapnik, Vladimir. The nature of statistical learning theory. Springer science & business media, 1999.
[2]. Hearst, Marti A., et al. "Support vector machines." IEEE Intelligent Systems and their applications 13.4 (1998): 18-28.
[3]. Zhang, Xian-Da, and Xian-Da Zhang. "Support vector machines." A Matrix Algebra Approach to Artificial Intelligence (2020): 617-679.
[4]. James, Gareth, et al. "Support vector machines." An introduction to statistical learning: with applications in R (2021): 367-402.
[5]. Das, Subhankar, and Sanjib Saha. "Data mining and soft computing using support vector machine: A survey." International Journal of Computer Applications 77.14 (2013).
[6]. Cervantes, Jair, et al. "A comprehensive survey on support vector machine classification: Applications, challenges and trends." Neurocomputing 408 (2020): 189-215.
[7]. Chapelle, Olivier, et al. "Choosing multiple parameters for support vector machines." Machine learning 46 (2002): 131-159.
[8]. Schölkopf, Bernhard, Alexander J. Smola, and Francis Bach. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, 2002.
[9]. Rakotomamonjy, Alain. "Variable selection using SVM-based criteria." Journal of machine learning research 3.Mar (2003): 1357-1370.
[10]. Cortes, Corinna, and Vladimir Vapnik. "Support-vector networks." Machine learning 20 (1995): 273-297.
[11]. Schölkopf, Bernhard, et al. "New support vector algorithms." Neural computation 12.5 (2000): 1207-1245.
[12]. Schölkopf, Bernhard, et al. "Estimating the support of a high-dimensional distribution." Neural computation 13.7 (2001): 1443-1471.
[13]. Vapnik, Vladimir N. "An overview of statistical learning theory." IEEE transactions on neural networks 10.5 (1999): 988-999.
[14]. Knerr, Stefan, Léon Personnaz, and Gérard Dreyfus. "Single-layer learning revisited: a stepwise procedure for building and training a neural network." Neurocomputing: algorithms, architectures and applications. Springer Berlin Heidelberg, 1990.
[15]. Chang, Chih-Chung, and Chih-Jen Lin. "LIBSVM: a library for support vector machines." ACM transactions on intelligent systems and technology (TIST) 2.3 (2011): 1-27..
[16]. Wu, Ting-Fan, Chih-Jen Lin, and Ruby Weng. "Probability estimates for multi-class classification by pairwise coupling." Advances in Neural Information Processing Systems 16 (2003).
[17]. Lin, Chih-Jen, and Ruby C. Weng. "Simple probabilistic predictions for support vector regression." National Taiwan University, Taipei (2004).
[18]. Que, Z., and Lin, C J. "One-class SVM probabilistic outputs." Technical report, National Taiwan University, 2022. URL http://www.csie.ntu.edu.tw/~cjlin/papers/oneclass_prob/oneclass_prob.pdf.
[19]. Fan, Rong-En, et al. "LIBLINEAR: A library for large linear classification." the Journal of machine Learning research 9 (2008): 1871-1874.
[20]. Boser, Bernhard E., Isabelle M. Guyon, and Vladimir N. Vapnik. "A training algorithm for optimal margin classifiers." Proceedings of the fifth annual workshop on Computational learning theory. 1992.
[21]. Hsieh, Cho-Jui, et al. "A dual coordinate descent method for large-scale linear SVM." Proceedings of the 25th international conference on Machine learning. 2008.
[22]. Lin, Chih-Jen, Ruby C. Weng, and S. Sathiya Keerthi. "Trust region newton methods for large-scale logistic regression." Proceedings of the 24th international conference on Machine learning. 2007.
[23]. Keerthi, S. Sathiya, et al. "A sequential dual method for large scale multi-class linear SVMs." Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. 2008.
[24]. Chauhan, Vinod Kumar, Kalpana Dahiya, and Anuj Sharma. "Problem formulations and solvers in linear SVM: a review." Artificial Intelligence Review 52.2 (2019): 803-855.
[25]. Crammer, Koby, and Yoram Singer. "On the learnability and design of output codes for multiclass problems." Machine learning 47 (2002): 201-233.
[26]. Tan, Xian, Fasheng Yu, and Xifeng Zhao. "Support vector machine algorithm for artificial intelligence optimization." Cluster Computing 22 (2019): 15015-15021.
[27]. Chen, Yanhua, et al. "A hybrid application algorithm based on the support vector machine and artificial intelligence: An example of electric load forecasting." Applied Mathematical Modelling 39.9 (2015): 2617-2632.
[28]. Chiang, Hsiu-Sen, et al. "A novel artificial bee colony optimization algorithm with SVM for bio-inspired software-defined networking." International Journal of Parallel Programming 48 (2020): 310-328.
[29]. Frohlich, Holger, Olivier Chapelle, and Bernhard Scholkopf. "Feature selection for support vector machines by means of genetic algorithm." Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence. IEEE, 2003.
[30]. Tang, Yichuan. "Deep learning using linear support vector machines." arXiv preprint arXiv:1306.0239 (2013).
[31]. Campbell, Colin. "Kernel methods: a survey of current techniques." Neurocomputing 48.1-4 (2002): 63-84.
[32]. Koshiba, Yoshiaki, and Shigeo Abe. "Comparison of L1 and L2 support vector machines." Proceedings of the International Joint Conference on Neural Networks, 2003.. Vol. 3. IEEE, 2003.
[33]. Chapelle, Olivier. "Training a support vector machine in the primal." Neural computation 19.5 (2007): 1155-1178.
[34]. Nalepa, Jakub, and Michal Kawulok. "Selecting training sets for support vector machines: a review." Artificial Intelligence Review 52.2 (2019): 857-900.
[35]. Yuan, Guo-Xun, Chia-Hua Ho, and Chih-Jen Lin. "Recent advances of large-scale linear classification." Proceedings of the IEEE 100.9 (2012): 2584-2603.
[36]. Van Den Doel, Kees, Uri Ascher, and Eldad Haber. "The lost honour of l2-based regularization." Large Scale Inverse Problems, Radon Ser. Comput. Appl. Math 13 (2012): 181-203.
[37]. Halder, Chayan, Jaya Paul, and Kaushik Roy. "Comparison of the classifiers in Bangla handwritten numeral recognition." 2012 International Conference on Radar, Communication and Computing (ICRCC). IEEE, 2012.
[38]. Vidhya, M. "Efficient classification of portscan attacks using Support Vector Machine." 2013 International Conference on Green High Performance Computing (ICGHPC). IEEE, 2013..
[39]. Datta, R. P., and Sanjib Saha. "Applying rule-based classification techniques to medical databases: an empirical study." International Journal of Business Intelligence and Systems Engineering 1.1 (2016): 32-48.
[40]. Blake, C. and Merz, C. J. "UCI repository of machine learning datasets." University of California, Irvine, Dept. of Information and Computer Sciences.(http://www.cs.waikato.ac.nz/~ml/weka/)
[41]. WEKA3 tool for machine learning and knowledge analysis. Online available at http://www.cs.waikato.ac.nz/~ml/weka/
[42]. Saha, Sanjib, and Debashis Nandi. "Data Classification based on Decision Tree, Rule Generation, Bayes and Statistical Methods: An Empirical Comparison." Int. J. Comput. Appl 129.7 (2015): 36-41.
[43]. Shawe-Taylor, John, and Shiliang Sun. "A review of optimization methodologies in support vector machines." Neurocomputing 74.17 (2011): 3609-3618.
[44]. Saha, Sanjib. "Non-rigid Registration of De-noised Ultrasound Breast Tumors in Image Guided Breast-Conserving Surgery." Intelligent Systems and Human Machine Collaboration. Springer, Singapore, 2023. 191-206.
[45]. Saha, Sanjib, et al. "ADU-Net: An Attention Dense U-Net based deep supervised DNN for automated lesion segmentation of COVID-19 from chest CT images." Biomedical Signal Processing and Control 85 (2023): 104974.
[46]. Chen, Pai-Hsuen, Chih-Jen Lin, and Bernhard Schölkopf. "A tutorial on ?-support vector machines." Applied Stochastic Models in Business and Industry 21.2 (2005): 111-136.