A Comparative Analysis of ChatGPT and Traditional Machine Learning Algorithms on Real-World Data

Bnar Kamaran Arif; Aso M. Aladdin

doi:10.24017/

Authors

Bnar Kamaran Arif Computer Science Department, College of Science, Charmo University, Chamchamal, Iraq| Information Technology Department, College of Informatics, Sulaimani Polytechnic University, Sulaymaniyah, Iraq. https://orcid.org/0009-0003-3316-2655
Aso M. Aladdin Computer Science Department, College of Science, Charmo University, Chamchamal, Iraq | Information Technology Department, Tishk International University, Sulaymaniyah, Iraq https://orcid.org/0000-0002-8734-0811

Abstract

The rapid growth of computer-based technologies has transformed many sectors, with artificial intelligence playing a key role in automating tasks previously performed by humans. In this context, natural language processing models such as chatbots, including Chat Generative Pre-Trained Transformer (ChatGPT), are increasingly being used as analytical tools alongside traditional machine learning algorithms. However, despite these advancements, concerns remain regarding the accuracy, processing time, and overall reliability of ChatGPT compared to traditional coding-based machine learning algorithms. This study provides a comparative evaluation of ChatGPT’s ability to generate intelligent responses. It focuses on three key aspects: accuracy across various datasets at different time intervals using the same account, performance relative to traditional machine learning algorithms in terms of accuracy, and the variability of ChatGPT’s results across diverse data sources. To address these concerns, 15 algorithms were tested against ChatGPT. Tests were done at four different time intervals using healthcare and education datasets. ChatGPT showed competitive accuracy but had more variability and slower processing. As a result, this study highlights notable performance limitations for ChatGPT. For instance, in the heart disease dataset, the Random Forest model achieved an accuracy of 0.672 in 0.012 seconds, whereas the average performance of ChatGPT was 0.608 with a processing time of 0.274 seconds. In comparison, the traditional Gradient Boosting Machine model attained an accuracy of 0.623 in 0.124 seconds, while ChatGPT recorded an accuracy of 0.589 in 1.019 seconds. Finally, this study draws specific conclusions based on the results and offers recommendations for future research.

Keywords:

ChatGPT, Algorithm, Machine Learning, Accuracy, Time Processing

References

S. Zhu et al., “Intelligent computing: The latest advances, challenges, and future,” Intelligent Computing, vol. 2, pp. 1–88, Jan. 2023, doi: 10.34133/icomputing.0006.

Y. K. Dwivedi, A. Sharma, N. P. Rana, M. Giannakis, P. Goel, and V. Dutot, “Evolution of artificial intelligence research in Technological Forecasting and Social Change: Research topics, trends, and future directions,” Technological Forecasting and Social Change, vol. 192, p. 122579, Jul. 2023, doi: 10.1016/j.techfore.2023.122579.

T. R. G. Babu, U. Saravanakumar, and B. Pattanaik, Computational Imaging and Analytics in Biomedical Engineering. New York: Apple Academic Press, 2024. doi: 10.1201/9781032669687.

G. Jeffson Sagala and Y. T. Samuel, “Sentiment analysis on ChatGPT App reviews on google play store using random forest algorithm, support vector machine and naïve bayes,” International Journal of Engineering Business and Social Science, vol. 2, no. 04, pp. 1194–1204, Mar. 2024, doi: 10.58451/ijebss.v2i04.148.

M. Spring, J. Faulconbridge, and A. Sarwar, “How information technology automates and augments processes: Insights from Artificial‐Intelligence‐based systems in professional service operations,” Journal of Operations Management, vol. 68, no. 6–7, pp. 592–618, Sep. 2022, doi: 10.1002/joom.1215.

L. De Angelis et al., “ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health,” Frontiers in Public Health, vol. 11, Apr. 2023, doi: 10.3389/fpubh.2023.1166120.

OpenAI et al., “GPT-4 Technical Report,” arXiv preprint arXiv:2303.08774, 2023. doi: 10.48550/arXiv.2303.08774.

B. Zohur and F. M. Rahmani, “ChatGPT vs Chatbots unleashing the power of conversational AI,” Journal of Material Sciences & Manufacturing Research, vol. 4, no. 5, pp. 1–5, 2023, doi: 10.47363/JMSMR/2023(4)158.

N. Khan, Z. Khan, A. Koubaa, M. K. Khan, and R. bin Salleh, “Global insights and the impact of generative AI-ChatGPT on multidisciplinary: a systematic review and bibliometric analysis,” Connection Science, vol. 36, no. 1, Article 2353630, Dec. 2024, doi: 10.1080/09540091.2024.2353630.

A. M. Rizki, B. Bustami, and S. F. Anshari, “Comparison of support vector machine and NaÃ¯ve Bayes algorithms in sentiment analysis of Tiktokshop application user reviews,” Journal of Renewable Energy, Electrical, and Computer Engineering, vol. 5, no. 1, pp. 18–29, Mar. 2025, doi: 10.29103/jreece.v5i1.21342.

N. Ansari, V. Babaei, and M. M. Najafpour, “Enhancing catalysis studies with chat generative pre-trained transformer (ChatGPT): Conversation with ChatGPT,” Dalton Transactions, vol. 53, no. 8, pp. 3534–3547, 2024, doi: 10.1039/D3DT04178F.

A. S. Pamungkas and N. Cahyono, “Analisis Sentimen Review ChatGPT di Play Store menggunakan Support Vector Machine dan K-Nearest Neighbor,” Edumatic: Jurnal Pendidikan Informatika, vol. 8, no. 1, pp. 1–10, Jun. 2024, doi: 10.29408/edumatic.v8i1.24114.

Y. Akbar, A. N. H. Regita, Sugiyono, and T. Wahyudi, “Analisa Sentimen Pada Media Sosial‘X’ Pencarian keyword ChatGPT menggunakan algoritma K-Nearest Neighbors (KNN),” Jurnal Indonesia : Manajemen Informatika dan Komunikasi, vol. 5, no. 3, pp. 3291–3305, Sep. 2024, doi: 10.35870/jimik.v5i3.1016.

N. Cong-Lem, A. Soyoof, and D. Tsering, “A Systematic review of the limitations and associated opportunities of ChatGPT,” International Journal of Human–Computer Interaction, vol. 41, no. 7, pp. 3851–3866, May 2024, doi: 10.1080/10447318.2024.2344142.

A. A. Alqudah et al., “Evaluating accuracy and reproducibility of ChatGPT responses to patient-based questions in Ophthalmology: An observational study,” Medicine, vol. 103, no. 32, p. e39120, Aug. 2024, doi: 10.1097/MD.0000000000039120.

A. Keles, O. G. Illeez, B. Erbagci, and E. Giray, “Artificial intelligence-generated responses to frequently asked questions on coccydynia: Evaluating the accuracy and consistency of GPT-4o’s performance,” Archives of Rheumatology, vol. 40, no. 1, pp. 63–71, Mar. 2025, doi: 10.46497/ArchRheumatol.2025.10966.

J. M. Lapates, M. D. G. Dacer, and D. N. Gaylo, “Analysis of student output on the use of ChatGPT: A predictive model approach,” International Journal of Engineering Trends and Technology, vol. 72, no. 10, pp. 216–224, Oct. 2024, doi: 10.14445/22315381/IJETT-V72I10P121.

D. O. Hasan, A. M. Aladdin, H. S. Talabani, T. A. Rashid, and S. Mirjalili, “The fifteen puzzle—a new approach through hybridizing three heuristics methods,” Computers, vol. 12, no. 1, p. 11, Jan. 2023, doi: 10.3390/computers12010011.

M. Maktabdar Oghaz, L. Babu Saheer, K. Dhame, and G. Singaram, “Detection and classification of ChatGPT-generated content using deep transformer models,” Frontiers in Artificial Intelligence, vol. 8, Article 1458707, Apr. 2025, doi: 10.3389/frai.2025.1458707.

B. Zhao, W. Jin, J. Del Ser, and G. Yang, “ChatAgri: Exploring potentials of ChatGPT on cross-linguistic agricultural text classification,” Neurocomputing, vol. 557, p. 126708, Nov. 2023, doi: 10.1016/j.neucom.2023.126708.

I. Tri Julianto, D. Kurniadi, Y. Septiana, and A. Sutedi, “Alternative text pre-processing using Chat GPT Open AI,” Jurnal Nasional Pendidikan Teknik Informatika, vol. 12, no. 1, pp. 67–77, Mar. 2023, doi: 10.23887/janapati.v12i1.59746.

Y. Guo and C. Wang, “Improvement path of legal system related to chatgpt application combined with decision tree algorithm,” Applied Mathematics and Nonlinear Sciences, vol. 9, no. 1, pp. 1–18, Jan. 2024, doi: 10.2478/amns-2024-1396.

P. P. Ray, “ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope,” Internet of Things and Cyber-Physical Systems, vol. 2, pp. 121–154, 2023, doi: 10.1016/j.iotcps.2023.04.003.

A. Haleem, M. Javaid, and R. P. Singh, “Exploring the competence of ChatGPT for customer and patient service management,” Intelligent Pharmacy, vol. 2, no. 3, pp. 392–414, Jun. 2024, doi: 10.1016/j.ipha.2024.03.002.

W. B. L. G. A. A. and S. L. A. Koubaa, “ Exploring ChatGPT capabilities and limitations: A survey,” IEEE Access, vol. 11, pp. 118698–118721, 2023, doi: 10.1109/ACCESS.2023.3326474.

T. H. Kung et al., “Performance of ChatGPT on USMLE: potential for ai-assisted medical education using large language models,” PLOS Digital Health, vol. 2, no. 2, p. e0000198, Feb. 2023, doi: 10.1371/journal.pdig.0000198.

A. Azaria, R. Azoulay, and S. Reches, “ChatGPT is a remarkable tool—for experts,” Data Intelligence, vol. 6, no. 1, pp. 240–296, Feb. 2024, doi: 10.1162/dint_a_00235.

Y. K. Dwivedi et al., “Opinion Paper: ‘So what if ChatGPT wrote it?’ Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy,” International Journal of Information Management, vol. 71, p. 102642, Aug. 2023, doi: 10.1016/j.ijinfomgt.2023.102642.

A. M. Aladdin and T. A. Rashid, “LEO: Lagrange elementary optimization,” Neural Computing Application, May 2025, doi: 10.1007/s00521-025-11225-2.

G. P. B. P. V. Prathamesh Muzumdar1, “An empirical comparison of machine learning models for student’s mental health illness assessment,” Asian Journal of Computer and Information Systems, vol. 10, no. 1, Feb. 2022.

C. K. Subasri and V. Jeyakumar, “Comparative analysis of machine learning algorithms for diabetes prediction using Real-time data-set,” in 2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT), IEEE, Jul. 2023, pp. 1–5. doi: 10.1109/ICCCNT56998.2023.10306851.

M. Sallam, “ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns,” Healthcare, vol. 11, no. 6, p. 887, Mar. 2023, doi: 10.3390/healthcare11060887.

D. R. Nugroho, N. Idris, K. Kurniawan, D. Fitrianah, E. Irwansyah, and G. P. Kusuma, “Logistic regression and random forest comparison in predicting students’ qualification based on students’ half-semester performance,” in 2023 11th International Conference on Information and Communication Technology (ICoICT), IEEE, Aug. 2023, pp. 214–219. doi: 10.1109/ICoICT58202.2023.10262783.

N. Hasan, J. A. Polin, Md. R. Ahmmed, Md. M. Sakib, Md. F. Jahin, and Md. M. Rahman, “A novel approach to analyzing the impact of AI, ChatGPT, and chatbot on education using machine learning algorithms,” Bulletin of Electrical Engineering and Informatics, vol. 13, no. 4, pp. 2951–2958, Aug. 2024, doi: 10.11591/eei.v13i4.7158.

H. Chen et al., “Multi role ChatGPT framework for transforming medical data analysis,” Scientific Reports, vol. 14, no. 1, p. 13930, Jun. 2024, doi: 10.1038/s41598-024-64585-5.

B. Basheer and M. Obaidi, “Enhancing healthcare data classification: Leveraging machine learning on ChatGPT-generated datasets,” International Journal of Advances in Applied Computational Intelligence, vol. 5, no. 2, pp. 34–45, 2024, doi: 10.54216/IJAACI.050203.

A. Rejeb, K. Rejeb, A. Appolloni, H. Treiblmaier, and M. Iranmanesh, “Exploring the impact of ChatGPT on education: A web mining and machine learning approach,” The International Journal of Management Education, vol. 22, no. 1, p. 100932, Mar. 2024, doi: 10.1016/j.ijme.2024.100932.

G. Narasimhan and A. Victor, “Empirical analysis of predicting heart disease using diverse datasets and classification procedures of machine learning,” Ain Shams Engineering Journal, vol. 16, no. 8, p. 103470, Aug. 2025, doi: 10.1016/j.asej.2025.103470.

S. Malik et al., “Advancing educational data mining for enhanced student performance prediction: a fusion of feature selection algorithms and classification techniques with dynamic feature ensemble evolution,” Scientific Reports, vol. 15, no. 1, p. 8738, Mar. 2025, doi: 10.1038/s41598-025-92324-x.

U. J. Nzenwata, E. Edwin, E. A. Chukwu, D. Osilaja, J. O. Hinmikaiye, and C. Enyinnah, “Extra trees model for heart disease prediction,” Journal of Data Analysis and Information Processing, vol. 13, no. 02, pp. 125–139, 2025, doi: 10.4236/jdaip.2025.132008.

K. Zhou et al., “A machine learning model for predicting acute respiratory distress syndrome risk in patients with sepsis using circulating immune cell parameters: a retrospective study,” BMC Infectious Diseases, vol. 25, no. 1, p. 568, Apr. 2025, doi: 10.1186/s12879-025-10974-8.

J. A. Benítez-Andrades, C. Prada-García, N. Ordás-Reyes, M. E. Blanco, A. Merayo, and A. Serrano-García, “Enhanced prediction of spine surgery outcomes using advanced machine learning techniques and oversampling methods,” Health Information Science and Systems, vol. 13, no. 1, p. 24, Mar. 2025, doi: 10.1007/s13755-025-00343-9.

A. A. H. Amin, A. M. Aladdin, D. O. Hasan, S. R. Mohammed-Taha, and T. A. Rashid, “Enhancing algorithm selection through comprehensive performance evaluation: statistical analysis of stochastic algorithms,” Computation, vol. 11, no. 11, p. 231, Nov. 2023, doi: 10.3390/computation11110231.

P. B. Serafim, P. Crescenzi, G. Gezici, E. Cappuccio, S. Rinzivillo, and F. Giannotti, “Exploring large language models capabilities to explain decision trees,” vol. 386, pp. 64–72, 2024. doi: 10.3233/FAIA240183.

Hemin Sardar Abdulla, Azad A. Ameen, Sarwar Ibrahim Saeed, and Tarik A. Rashid, “MRSO: Balancing exploration and exploitation through,” Algorithms, vol. 17, no. 9, 2024, doi: 10.3390/a17090423.

I. Lintang, A. D. Lestari, and B. Prasetiyo, “Application of k-nearest neighbor algorithm in classification of engine performance in car companies using Rapidminer,” Journal of Student Research Exploration, vol. 2, no. 2, pp. 120–130, Jul. 2024, doi: 10.52465/josre.v2i2.345.

A. Sabir, H. A. Ali, and M. A. Aljabery, “ChatGPT Tweets sentiment analysis using machine learning and data classification,” Informatica, vol. 48, no. 7, pp. 103–112, May 2024, doi: 10.31449/inf.v48i7.5535.

S. Kodera, O. Yokoi, M. Kaneko, Y. Sato, S. Ito, and K. Hata, “Age estimation from blood test results using a random forest model,” Feb. 06, 2024. doi: 10.1101/2024.02.06.24302114.

J. P. Munggaran, A. A. Alhafidz, M. Taqy, D. A. R. Agustini, and M. Munawir, “Sentiment analysis of twitter users’ Opinion data regarding the use of chatgpt in education,” Journal of Computer Engineering, Electronics and Information Technology, vol. 2, no. 2, pp. 75–88, Jun. 2023, doi: 10.17509/coelite.v2i2.59645.

C. Duncan and I. Mcculloh, “Unmasking bias in ChatGPT responses,” in Proceedings of the International Conference on Advances in Social Networks Analysis and Mining, New York, NY, USA: ACM, Nov. 2023, pp. 687–691. doi: 10.1145/3625007.3627484.

S. Rabbani, D. Safitri, F. Try Puspa Siregar, R. Rahmaddeni, and L. Efrizoni, “Evaluation of support vector machine, naive bayes, decision tree, and gradient boosting algorithms for sentiment analysis on ChatGPT twitter dataset,” Indonesian Journal of Artificial Intelligence and Data Mining, vol. 7, no. 1, pp. 11–21, Nov. 2023, doi: 10.24014/ijaidm.v7i1.24662.

O. Shobayo, S. Adeyemi-Longe, O. Popoola, and B. Ogunleye, “innovative sentiment analysis and prediction of stock price using finbert, gpt-4 and logistic regression: A data-driven approach,” Big Data and Cognitive Computing, vol. 8, no. 11, p. 143, Oct. 2024, doi: 10.3390/bdcc8110143.

C. Zhong and J. B. Kim, “Teaching case teaching business students logistic regression in r with the aid of ChatGPT,” Journal of Information Systems Education, vol. 35, no. 2, pp. 138–143, 2024, doi: 10.62273/DYLI2468.

S.-L. Cheng et al., “Comparisons of quality, correctness, and similarity between chatgpt-generated and human-written abstracts for basic research: Cross-sectional study,” Journal of Medical Internet Research, vol. 25, p. e51229, Dec. 2023, doi: 10.2196/51229.

C. A. Sequeira and E. M. Borges, “enhancing statistical education in chemistry and STEAM Using JAMOVI. Part 2. Comparing dependent groups and principal component analysis (PCA),” Journal of Chemical Education, vol. 101, no. 11, pp. 5040–5049, Nov. 2024, doi: 10.1021/acs.jchemed.4c00342.

S. Sotirov and I. Dimitrov, “Application of machine learning algorithms for prediction of tumor T-cell immunogens,” Applied Sciences, vol. 14, no. 10, p. 4034, May 2024, doi: 10.3390/app14104034.

A. Li, Y. Wang, and H. Chen, “AI driven cardiovascular risk prediction using NLP and large language models for personalized medicine in athletes,” SLAS Technology, vol. 32, p. 100286, Jun. 2025, doi: 10.1016/j.slast.2025.100286.

H. Yildiz Durak, F. Eğin, and A. Onan, “A comparison of human‐written versus AI ‐generated text in discussions at educational settings: investigating features for ChatGPT , Gemini and BingAI,” European Journal of Education, vol. 60, no. 1, Mar. 2025, doi: 10.1111/ejed.70014.

F. Hauth et al., “electronic patient-reported outcome measures in radiation oncology: Initial experience after workflow implementation,” JMIR mHealth and uHealth, vol. 7, no. 7, p. e12345, Jul. 2019, doi: 10.2196/12345.

K. Abdalgader, A. A. Matroud, and K. Hossin, “Experimental study on short-text clustering using transformer-based semantic similarity measure,” PeerJ Computer Science, vol. 10, p. e2078, May 2024, doi: 10.7717/peerj-cs.2078.

X. Zhao, “Capitalized comparison of three machine learning models: Linear model, decision tree, neural network,” Highlights in Science, Engineering and Technology, vol. 85, pp. 790–796, Mar. 2024, doi: 10.54097/pxcpca72.

H. Luo and H. Kong, “ChatGPT plagiarism in the academic field: Exploration and analysis of plagiarism effects,” International Conference on Machine Learning and Intelligent Computing, vol. 245, pp. 1–11, 2024, Accessed: Aug. 14, 2025. [Online]. Available: https://proceedings.mlr.press/v245/haiqiong24a.html.

H. Fakour and M. Imani, “Socratic wisdom in the age of AI: a comparative study of ChatGPT and human tutors in enhancing critical thinking skills,” Frontiers in Education (Lausanne), vol. 10, pp. 1–12, 2025, doi: 10.3389/feduc.2025.1528603.

R. Krimsony, “UCI Heart Disease Data,” Kaggle. Accessed: Jul. 22, 2025. [Online]. Available: https://www.kaggle.com/datasets/redwankarimsony/heart-disease-data.

L. Tharmalingam, “Disease Symptoms and Patient Profile Dataset,” Kaggle. Accessed: Jul. 22, 2025. [Online]. Available: https://www.kaggle.com/datasets/uom190346a/disease-symptoms-and-patient-profile-dataset.

R. Elkharoua, “Students’ Performance Dataset,” Kaggle. Accessed: Jul. 22, 2025. [Online]. Available: https://www.kaggle.com/datasets/rabieelkharoua/students-performance-dataset.

Larsen0966, “Student Performance Data Set,” Kaggle. Accessed: Jul. 22, 2025. [Online]. Available: https://www.kaggle.com/datasets/larsen0966/student-performance-data-set.

O. B. Elhalid, Z. Alm Alhelal, and S. HASSAN, “Exploring the fundamentals of python programming: A comprehensive guide for beginners,” SSRN Electronic Journal, 2023, doi: 10.2139/ssrn.4612765.

C. J. Weiss, “Visualizing protein big data using Python and Jupyter notebooks,” Biochemistry and Molecular Biology Education, vol. 50, no. 5, pp. 431–436, Sep. 2022, doi: 10.1002/bmb.21621.

O. Lederman et al., “Promises and perils of generative artificial intelligence: a narrative review informing its ethical and practical applications in clinical exercise physiology,” BMC Sports Science, Medicine and Rehabilitation, vol. 17, no. 1, p. 131, May 2025, doi: 10.1186/s13102-025-01182-7.

Y. Fu, “A comparative study of house price prediction using linear regression and random forest models,” Highlights in Science, Engineering and Technology, vol. 107, pp. 96–103, Aug. 2024, doi: 10.54097/vcy5n584.

H. Li, “House price prediction and analysis based on random forest and XGBoost models,” Highlights in Business, Economics and Management, vol. 21, pp. 934–938, Dec. 2023, doi: 10.54097/hbem.v21i.14837.

A. B. Adetunji, O. N. Akande, F. A. Ajala, O. Oyewo, Y. F. Akande, and G. Oluwadara, “House price prediction using random forest machine learning technique,” Procedia Computer Science, vol. 199, pp. 806–813, 2022, doi: 10.1016/j.procs.2022.01.100.

M. Suaza-Medina, R. Peñabaena-Niebles, and M. Jubiz-Diaz, “A model for predicting academic performance on standardised tests for lagging regions based on machine learning and Shapley additive explanations,” Scientific Reports, vol. 14, no. 1, p. 25306, Oct. 2024, doi: 10.1038/s41598-024-76596-3.