A Comparative Analysis of Tuned and Untuned Machine Learning Models for Cybersecurity Intrusion Detection

Abstract

This study benchmarks tuned and untuned machine learning models for intrusion detection across a 63-class attack taxonomy using the Cybersecurity Attack and Defense Dataset ($n=14{,}133$). A leakage-safe pipeline is designed, fitting all encoders and scalers exclusively on training data. Nine models are evaluated: five classical methods (Random Forest, Histogram-based Gradient Boosting, Extra Trees, SGD-based SVM, and Logistic Regression), two boosting frameworks (XGBoost and LightGBM), and two hybrids (soft-voting ensemble and two-level stacking with a LightGBM meta-learner). All models are systematically tuned via stratified randomized search and assessed on twelve metrics. XGBoost performs best, achieving a weighted $F_1$ of 0.8062, accuracy of 0.8094, MCC of 0.8030, and Cohen's $\kappa$ of 0.8028, outperforming Random Forest ($F_1=0.7062$), LightGBM ($F_1=0.7925$), linear classifiers, and hybrid models. LightGBM shows the largest tuning gain ($+0.755$ $F_1$). The proposed protocol offers a reproducible, bias-controlled benchmark for high-cardinality intrusion detection.