A Correlation-based Feature Extraction method and intrusion detection framework for ML-based Intrusion Detection System (IDS)

Abstract

The number of cyber-attacks has increased in recent years in both the number and varieties which demands a dynamic way of detection. Network Intrusion Detection System (IDS) leverages the key feature of Machine Learning algorithms to analyze network traffic and to build a sophisticated and dynamic system. However, the performance of Machine Learning algorithms depends on the representation of dataset.Recent research on Network Intrusion Detection has focused on feature selection and feature extraction techniques to obtain the best output and to adapt to continuously varying attacks. In this paper, we present a correlation-based technique for feature extraction from the traffic information. Our feature extraction framework builds a normal traffic profile and consider the deviation of network traffic information from normal traffic profile as the new feature set. The new derived set of features optimizes the anomaly detection technique using classification algorithm. Our evaluation conducted on KDD-CUP99,UNSW-NB15,NSL-KDD,AWID and CIC-IDS2017 dataset and outperformed detection rate for intrusions compared to other recent state-ofthe-art anomaly detection methods.