An Explainable AI Based Alert Triage Framework for Security Operations Centres: Architecture, Evaluation, And Analyst Centred Design
DOI:
https://doi.org/10.51173/ijds.v3i2.84Keywords:
Explainable Artificial Intelligence (XAI), Security Operations Center (SOC), Alert Triage, SHAP, Intrusion DetectionAbstract
Security operations centers (SOCs) today are overwhelmed by a continually growing volume of security alerts from a variety of detection technologies. Analysts typically receive thousands of alerts per shift. Many of these alerts are false positives or low-priority events that cause serious analyst fatigue, delayed incident response, and increased organizational risk. The false-positive rate is typically greater than 50% in business environments. This paper proposes and evaluates ExTriage, a novel Explainable Artificial Intelligence (XAI)-based alert triage framework integrating: (i) a high-performance ensemble classifier combining XGBoost and a multi-layer perceptron; (ii) a SHAP-based global and local explanation engine; (iii) a counterfactual explanation module; and (iv) a RankNet-inspired alert prioritization module. Evaluated on CICIDS-2017, CICIDS-2018, and UNSW-NB15, ExTriage achieves a macro F1-score of 97.89% and AUC-ROC of 0.9963 while reducing false-positive rates by 70.8%. In a controlled study with 24 professional SOC analysts, average triage time was reduced from 47 to 8.2 seconds per alert, with statistically significant improvements in trust calibration, decision confidence, and alert disposition accuracy.
Downloads
References
A. Chakraborty, A. Biswas, and A. K. Khan, "Artificial Intelligence for Cybersecurity: Threats, Attacks and Mitigation," Intelligent Systems Reference Library, vol. 231, pp. 3–25, 2023, doi: 10.1007/978-3-031-12419-8_1.
S. Freitas, J. Kalajdjieski, A. Gharib, and R. McCann, "AI-Driven Guided Response for Security Operation Centers with Microsoft Copilot for Security," WWW Companion 2025 - Companion Proceedings of the ACM Web Conference 2025, vol. 1, pp. 191–200, May 2025, doi: 10.1145/3701716.3715209.
D. Bhamare, T. Salman, M. Samaka, A. Erbad and R. Jain, "Feasibility of Supervised Machine Learning for Cloud Security," 2016 International Conference on Information Science and Security (ICISS), Pattaya, Thailand, 2016, pp. 1-5, doi: 10.1109/ICISSEC.2016.7885853.
B. Avanzi, X. Tan, G. Taylor, and B. Wong, "On the Evolution of Data Breach Reporting Patterns and Frequency in the United States: A Cross-State Analysis," North American Actuarial Journal, vol. 29, no. 4, pp. 833–864, 2025, doi: 10.1080/10920277.2025.2457491.
F. Doshi-Velez and B. Kim, "Towards A Rigorous Science of Interpretable Machine Learning," Feb. 2017, Accessed: Jul. 11, 2026. [Online]. Available: https://arxiv.org/pdf/1702.08608.
T. Miller, "Explanation in artificial intelligence: Insights from the social sciences," Artif. Intell., vol. 267, pp. 1–38, Feb. 2019, doi: 10.1016/J.ARTINT.2018.07.007.
S. M. Lundberg, P. G. Allen, and S.-I. Lee, "A Unified Approach to Interpreting Model Predictions," Adv. Neural Inf. Process. Syst., vol. 30, 2017, Accessed: Jul. 11, 2026. [Online]. Available: https://github.com/slundberg/shap.
M. T. Ribeiro, S. Singh, and C. Guestrin, " 'Why should i trust you?' Explaining the predictions of any classifier," Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, vol. 13-17-August-2016, pp. 1135–1144, Aug. 2016, doi: 10.1145/2939672.2939778.
R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D. Pedreschi, "A survey of methods for explaining black box models," ACM Comput. Surv., vol. 51, no. 5, Sep. 2019, doi: 10.1145/3236009.
A. Nadeem et al., "SoK: Explainable Machine Learning for Computer Security Applications," 2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P), Delft, Netherlands, 2023, pp. 221-240, doi: 10.1109/EuroSP57164.2023.00022.
M. Keshk, N. Koroniotis, N. Pham, N. Moustafa, B. Turnbull, and A. Y. Zomaya, "An explainable deep learning-enabled intrusion detection framework in IoT networks," Inf. Sci. (N. Y)., vol. 639, p. 119000, Aug. 2023, doi: 10.1016/J.INS.2023.119000.
R. Sommer and V. Paxson, "Outside the Closed World: On Using Machine Learning for Network Intrusion Detection," 2010 IEEE Symposium on Security and Privacy, Oakland, CA, USA, 2010, pp. 305-316, doi: 10.1109/SP.2010.25.
K. Julisch, "Clustering intrusion detection alarms to support root cause analysis," ACM Transactions on Information and System Security, vol. 6, no. 4, pp. 443–471, Nov. 2003, doi: 10.1145/950191.950192.
J. Tilbury and S. V. Flowerday, "The vigilance paradox: automation reliance inside the modern SOC," Information and Computer Security, pp. 1–24, 2025, doi: 10.1108/ICS-08-2025-0318/1334832.
L. Zhang, Z. Shao, B. Chen and J. Benitez, "Unraveling Generative AI Adoption in Enterprise Digital Platforms: The Effect of Institutional Pressures and the Moderating Role of Internal and External Environments," in IEEE Transactions on Engineering Management, vol. 72, pp. 335-348, 2025, doi: 10.1109/TEM.2024.3513773.
I. Sharafaldin, A. H. Lashkari, and A. A. Ghorbani, "Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization," 2018, doi: 10.5220/0006639801080116.
K. Alrawashdeh and C. Purdy, "Toward an Online Anomaly Intrusion Detection System Based on Deep Learning," 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), Anaheim, CA, USA, 2016, pp. 195-200, doi: 10.1109/ICMLA.2016.0040.
M. Shoaib et al., "A deep learning-assisted visual attention mechanism for anomaly detection in videos," Multimedia Tools and Applications 2023 83:29, vol. 83, no. 29, pp. 73363–73390, Dec. 2023, doi: 10.1007/S11042-023-17770-Z.
N. Khan, K. Ahmad, A. Al Tamimi, M. M. Alani, A. Bermak, and I. Khalil, "Explainable AI-Based Intrusion Detection Systems for Industry 5.0 and Adversarial XAI: A Systematic Review," Information (Switzerland), vol. 16, no. 12, p. 1036, Dec. 2025, doi: 10.3390/INFO16121036.
P. Bajpayi, S. Mathur and M. S. Gaur, "Dual Stream Cross-Attentional Driven Adaptive Anomaly Detection for Zero-Day Attacks in IoMT Networks," 2026 13th International Conference on Computing for Sustainable Global Development (INDIACom), New Delhi, India, 2026, pp. 1-6, doi: 10.23919/INDIACom70271.2026.11526185.
S. M. Lundberg et al., "From local explanations to global understanding with explainable AI for trees," Nature Machine Intelligence 2020 2:1, vol. 2, no. 1, pp. 56–67, Jan. 2020, doi: 10.1038/s42256-019-0138-9.
D. Slack, S. Hilgard, E. Jia, S. Singh, and H. Lakkaraju, "Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods," AIES 2020 - Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pp. 180–186, Feb. 2020, doi: 10.1145/3375627.3375830.
A. Warnecke, D. Arp, C. Wressnegger and K. Rieck, "Evaluating Explanation Methods for Deep Learning in Security," 2020 IEEE European Symposium on Security and Privacy (EuroS&P), Genoa, Italy, 2020, pp. 158-174, doi: 10.1109/EuroSP48549.2020.00018.
S. Verma, V. Boonsanong, M. Hoang, K. Hines, J. Dickerson, and C. Shah, "Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review," ACM Comput. Surv., vol. 56, no. 12, Oct. 2024, doi: 10.1145/3677119.
L. Allodi and F. Massacci, "Comparing vulnerability severity and exploits using case-control studies," ACM Transactions on Information and System Security, vol. 17, no. 1, Aug. 2014, doi: 10.1145/2630069.
J. B. Awotunde and S. Misra, "Feature Extraction and Artificial Intelligence-Based Intrusion Detection Model for a Secure Internet of Things Networks," Lecture Notes on Data Engineering and Communications Technologies, vol. 109, pp. 21–44, 2022, doi: 10.1007/978-3-030-93453-8_2.
Y. Jiang, N. Oo, Q. Meng, H. W. Lim, and B. Sikdar, "VulRG: Multi-Level Explainable Vulnerability Patch Ranking for Complex Systems Using Graphs," vol. 1, Feb. 2025, doi: https://doi.org/10.48550/arXiv.2502.11143.
C. Burges et al., "Learning to rank using gradient descent," ICML 2005 - Proceedings of the 22nd International Conference on Machine Learning, pp. 89–96, 2005, doi: 10.1145/1102351.1102363.
J. Dodge, Q. V. Liao, Y. Zhang, R. K. E. Bellamy, and C. Dugan, "Explaining models," IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces, pp. 275–285, Feb. 2019, doi: 10.1145/3301275.3302310.
G. Bansal, B. Nushi, E. Kamar, D. S. Weld, W. S. Lasecki, and E. Horvitz, "Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Trade-off," Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 2429–2437, Jul. 2019, doi: 10.1609/AAAI.V33I01.33012429.
R. Parasuraman and V. Riley, "Humans and Automation: Use, Misuse, Disuse, Abuse," Hum. Factors, vol. 39, no. 2, pp. 230–253, 1997, doi: 10.1518/001872097778543886.
C.-K. Yeh, C.-Y. Hsieh, A. Suggala, D. I. Inouye, and P. K. Ravikumar, "On the (In)fidelity and Sensitivity of Explanations," Adv. Neural Inf. Process. Syst., vol. 32, 2019, Accessed: Jul. 11, 2026. [Online]. Available: https://github.com/chihkuanyeh/saliency_evaluation.
T. Laugel, M.-J. Lesot, C. Marsala, X. Renard, and M. Detyniecki, "Inverse Classification for Comparison-based Interpretability in Machine Learning," Dec. 2017, Accessed: Jul. 11, 2026. [Online]. Available: https://arxiv.org/pdf/1712.08443.
Z. Lin, Y. Shi, and Z. Xue, "IDSGAN: Generative Adversarial Networks for Attack Generation Against Intrusion Detection," Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13282 LNAI, pp. 79–91, 2022, doi: 10.1007/978-3-031-05981-0_7.
Caroline E. Zsambok, Naturalistic Decision Making. Psychology Press, 2014. doi: 10.4324/9781315806129.
S. Mane and D. Rao, "Explaining Network Intrusion Detection System Using Explainable AI Framework," Mar. 2021, Accessed: Jul. 11, 2026. [Online]. Available: https://arxiv.org/pdf/2103.07110.
P. Vibhandik, S. Sawant, A. Joshi, and R. Bidwe, "A systematic literature review on forest fire susceptibility mapping using geo-spatial technology and future research directions," Discover Artificial Intelligence 2026 6:1, vol. 6, no. 1, pp. 396-, Mar. 2026, doi: 10.1007/S44163-026-00921-0.
B. Shehu and I. Jordanov, "Machine Learning for Fault Detection in Wireless Sensor Networks: A Survey," in IEEE Internet of Things Journal, vol. 13, no. 10, pp. 20437-20451, 15 May15, 2026, doi: 10.1109/JIOT.2026.3669436.
N. Khan, K. Ahmad, A. Al Tamimi, M. M. Alani, A. Bermak, and I. Khalil, "Explainable AI-based Intrusion Detection System for Industry 5.0: An Overview of the Literature, associated Challenges, the existing Solutions, and Potential Research Directions," Jul. 2024, Accessed: Jul. 11, 2026. [Online]. Available: https://arxiv.org/pdf/2408.03335.
A. Zaslavsky, C. Perera, and D. Georgakopoulos, "Sensing as a Service and Big Data," Jan. 2013, Accessed: Jul. 11, 2026. [Online]. Available: https://arxiv.org/pdf/1301.0159.
Downloads
Published
How to Cite
Issue
Section
