A Correlation-based Feature Extraction method and intrusion detection framework for ML-based Intrusion Detection System (IDS)

Mazumder, Md. Mumtahin Habib Ullah

dc.contributor.author	Mazumder, Md. Mumtahin Habib Ullah
dc.date.accessioned	2022-04-15T17:16:13Z
dc.date.available	2022-04-15T17:16:13Z
dc.date.issued	2021-03-30
dc.identifier.citation	[1] “Cyber security report.” https://docs.broadcom.com/doc/ istr-22-2017-en. Accessed: 2017-07-10. [2] “Cyber security summary-2020.” https://symantec-enterprise-blogs. security.com/blogs/feature-stories/symantec-security-summary-june-202Accessed: 2020-06-04. [3] M. Husak, J. Kom ´ arkov ´ a, E. Bou-Harb, and P. eleda, “Survey of attack projection, predic- ´ tion, and forecasting in cyber security,” IEEE Communications Surveys Tutorials, vol. 21, pp. 640–660, 2019. [4] I. Gondal, P. Vamplew, and J. Kamruzzaman, “Survey of intrusion detection systems: techniques, datasets and challenges,” Cybersecurity, vol. 2, 12 2019. [5] T. Porter and M. Gough, “Chapter 7 - active security monitoring,” in How to Cheat at VoIP Security (T. Porter and M. Gough, eds.), How to Cheat, pp. 185–206, Burlington: Syngress, 2007. [6] H.-J. Liao, C.-H. Richard Lin, Y.-C. Lin, and K.-Y. Tung, “Intrusion detection system: A comprehensive review,” Journal of Network and Computer Applications, vol. 36, no. 1, pp. 16–24, 2013. [7] C. Pascoal, M. R. Oliveira, R. Valadas, P. Filzmoser, P. Salvador, and A. Pacheco, “Robust feature selection and robust pca for internet traffic anomaly detection,” 2012 Proceedings IEEE INFOCOM, pp. 1755–1763, 2012. [8] R. Sommer and V. Paxson, “Outside the closed world: On using machine learning for network intrusion detection,” 2010 IEEE Symposium on Security and Privacy, pp. 305– 316, 2010. [9] R. Zuech and T. Khoshgoftaar, “A survey on feature selection for intrusion detection,” pp. 150–155, 01 2015. 46 REFERENCES 47 [10] H. Liu and L. Yu, “Toward integrating feature selection algorithms for classification and clustering,” IEEE Transactions on Knowledge and Data Engineering, vol. 17, pp. 491– 502, 2005. [11] Y. Li, J.-L. Wang, Z.-H. Tian, T.-B. Lu, and C. Young, “Building lightweight intrusion detection system using wrapper-based feature selection mechanisms,” Computers Security, vol. 28, no. 6, pp. 466–475, 2009. [12] I. Guyon, S. Gunn, M. Nikravesh, and L. Zadeh, “Feature extraction: foundations and applications,” 01 2006. [13] H. T. Nguyen, K. Franke, and S. Petrovic, “Feature extraction methods for intrusion detection systems,” 2012. [14] R. Agrawal, T. Imieli’nski, and A. Swami, “Mining association rules between sets of items in large databases,” vol. 22, pp. 207–216, 01 1993. [15] H. Mannila and H. Toivonen, “Discovering generalized episodes using minimal occurrences,” in Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD’96) (E. Simoudis, J. Han, and U. Fayyad, eds.), (United States), pp. 146–151, AAAI Press, Aug. 1996. [16] K. Wang, J. Parekh, and S. Stolfo, “Anagram: A content anomaly detector resistant to mimicry attack,” vol. 4219, pp. 226–248, 09 2006. [17] N. Ye, S. M. Emran, Q. Chen, and S. Vilbert, “Multivariate statistical analysis of audit trails for host-based intrusion detection,” IEEE Trans. Computers, vol. 51, pp. 810–820, 2002. [18] D. Yeung, S. Jin, and X. Wang, “Covariance-matrix modeling and detecting various flooding attacks,” IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, vol. 37, pp. 157–169, 2007. [19] Z. Tan, A. Jamdagni, X. He, P. Nanda, and R. Liu, “A system for denial-of-service attack detection based on multivariate correlation analysis,” IEEE Transactions on Parallel and Distributed Systems, vol. 25, pp. 447–456, 02 2014. [20] Q. Li, Z. Tan, A. Jamdagni, P. Nanda, X. He, and W. Han, “An intrusion detection system based on polynomial feature correlation analysis,” 2017 IEEE Trustcom/BigDataSE/ICESS, pp. 978–983, 2017. [21] I. Gondal, P. Vamplew, and J. Kamruzzaman, “Survey of intrusion detection systems: techniques, datasets and challenges,” Cybersecurity, vol. 2, 12 2019. REFERENCES 48 [22] F. Gottwalt, E. Chang, and T. Dillon, “Corrcorr: A feature selection method for multivariate correlation network anomaly detection techniques,” Comput. Secur., vol. 83, pp. 234– 245, 2019. [23] X. J. Zhou and T. S. Dillon, “A statistical-heuristic feature selection criterion for decision tree induction,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 13, no. 8, pp. 834–841, 1991. [24] J. Viinikka, H. Debar, L. Me, A. Lehikoinen, and M. Tarvainen, “Processing intrusion ´ detection alert aggregates with time series modeling,” Information Fusion, vol. 10, no. 4, pp. 312–324, 2009. Special Issue on Information Fusion in Computer Security. [25] Qingtao Wu and Zhiqing Shao, “Network anomaly detection using time series analysis,” in Joint International Conference on Autonomic and Autonomous Systems and International Conference on Networking and Services - (icas-isns’05), pp. 42–42, 2005. [26] N. Walkinshaw, R. Taylor, and J. Derrick, “Inferring extended finite state machine models from software executions,” Empirical Software Engineering, vol. 21, pp. 811–853, 2013. [27] I. Studnia, E. Alata, V. Nicomette, M. Kaaniche, and Y. Laarouchi, “A language-based ˆ intrusion detection approach for automotive embedded networks,” Int. J. Embed. Syst., vol. 10, pp. 1–12, 2018. [28] G. Kim, S. Lee, and S. Kim, “A novel hybrid intrusion detection method integrating anomaly detection with misuse detection,” Expert Systems with Applications, vol. 41, no. 4, Part 2, pp. 1690–1700, 2014. [29] P. S. Kenkre, A. Pai, and L. Colaco, “Real time intrusion detection and prevention system,” in Proceedings of the 3rd International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA) 2014 (S. C. Satapathy, B. N. Biswal, S. K. Udgata, and J. Mandal, eds.), (Cham), pp. 405–411, Springer International Publishing, 2015. [30] S. Dua and X. Du, Data Mining and Machine Learning in Cybersecurity. USA: Auerbach Publications, 1st ed., 2011. [31] K. Bajaj and A. Arora, “Improving the intrusion detection using discriminative machine learning approach and improve the time complexity by data mining feature selection methods,” International Journal of Computer Applications, vol. 76, pp. 5–11, 08 2013. [32] S. Thaseen and C. Kumar, “An analysis of supervised tree based classifiers for intrusion detection system,” 2013 International Conference on Pattern Recognition, Informatics and Mobile Engineering, pp. 294–299, 2013. REFERENCES 49 [33] A. Khraisat, I. Gondal, and P. Vamplew, “An anomaly intrusion detection system using c5 decision tree classifier,” in Trends and Applications in Knowledge Discovery and Data Mining (M. Ganji, L. Rashidi, B. C. M. Fung, and C. Wang, eds.), (Cham), pp. 149–155, Springer International Publishing, 2018. [34] L. Koc, T. A. Mazzuchi, and S. Sarkani, “A network intrusion detection system based on a hidden na¨ıve bayes multiclass classifier,” Expert Systems with Applications, vol. 39, no. 18, pp. 13492–13500, 2012. [35] M. S. Hoque, M. Mukit, and M. A. N. Bikas, “An implementation of intrusion detection system using genetic algorithm,” International Journal of Network Security Its Applications, vol. 4, pp. 109–120, 03 2012. [36] S. Elhag, A. Fernandez, A. Bawakid, S. Alshomrani, and F. Herrera, “On the combination ´ of genetic fuzzy systems and pairwise learning for improving detection rates on intrusion detection systems,” Expert Systems with Applications, vol. 42, no. 1, pp. 193–202, 2015. [37] Y. Li, J. Xia, S. Zhang, J. Yan, X. Ai, and K. Dai, “An efficient intrusion detection system based on support vector machines and gradually feature removal method,” Expert Systems with Applications, vol. 39, no. 1, pp. 424–430, 2012. [38] C. Annachhatre, T. Austin, and M. Stamp, “Hidden markov models for malware classification,” Journal of Computer Virology and Hacking Techniques, vol. 11, 05 2014. [39] W.-C. Lin, S.-W. Ke, and C.-F. Tsai, “Cann: An intrusion detection system based on combining cluster centers and nearest neighbors,” Knowledge-Based Systems, vol. 78, pp. 13– 21, 2015. [40] R. A. R. Ashfaq, X.-Z. Wang, J. Z. Huang, H. Abbas, and Y.-L. He, “Fuzziness based semi-supervised learning approach for intrusion detection system,” Information Sciences, vol. 378, pp. 484–497, 2017. [41] N. B. R. S. S. P. P. Rath, L. F. S. Davoodkhani, and A. T. M. Ahmed, “A prototype multiview approach for reduction of false alarm rate in network intrusion detection system,” Journal of Computer Networks and Communications, vol. 5, pp. 49–59, 2017. [42] J. Lyngdoh, M. I. Hussain, S. Majaw, and H. Kalita, An Intrusion Detection Method Using Artificial Immune System Approach: Second International Conference, ICAICR 2018, Shimla, India, July 14–15, 2018, Revised Selected Papers, Part II, pp. 379–387. 01 2019. [43] M. Goldstein, “Fastlof: An expectation-maximization based local outlier detection algorithm,” Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pp. 2282–2285, 2012. REFERENCES 50 [44] H. Sadreazami, A. Mohammadi, A. Asif, and K. Plataniotis, “Distributed-graph-based statistical approach for intrusion detection in cyber-physical systems,” IEEE Transactions on Signal and Information Processing over Networks, vol. 4, pp. 137–147, 2018. [45] A. A. Aburomman and M. B. Ibne Reaz, “A novel svm-knn-pso ensemble method for intrusion detection system,” Applied Soft Computing, vol. 38, pp. 360–372, 2016. [46] M. Jabbar, R. Aluvalu, and S. S. Reddy S, “Rfaode: A novel ensemble intrusion detection system,” Procedia Computer Science, vol. 115, pp. 226–234, 2017. 7th International Conference on Advances in Computing Communications, ICACC-2017, 22-24 August 2017, Cochin, India. [47] D. Gaikwad and R. C. Thool, “Intrusion detection system using bagging with partial decision treebase classifier,” Procedia Computer Science, vol. 49, pp. 92–98, 2015. Proceedings of 4th International Conference on Advances in Computing, Communication and Control (ICAC3’15). [48] N. Moustafa, B. Turnbull, and K. R. Choo, “An ensemble intrusion detection technique based on proposed statistical flow features for protecting network traffic of internet of things,” IEEE Internet of Things Journal, vol. 6, no. 3, pp. 4815–4830, 2019. [49] N. Paulauskas and J. Auskalnis, “Analysis of data pre-processing influence on intrusion detection using nsl-kdd dataset,” in 2017 Open Conference of Electrical, Electronic and Information Sciences (eStream), pp. 1–5, 2017. [50] Y. Zhou, G. Cheng, S. Jiang, and M. Dai, “Building an efficient intrusion detection system based on feature selection and ensemble classifier,” Computer Networks, vol. 174, p. 107247, 2020. [51] M. Abdullah, A. Balamash, A. Al-Shannaq, and S. Almabdy, “Enhanced intrusion detection system using feature selection method and ensemble learning algorithms,” International Journal of Computer Science and Information Security,, vol. 16, pp. 48–55, 02 2018. [52] H. Hota and A. Shrivas, “Decision tree techniques applied on nsl-kdd data and its comparison with various feature selection techniques,” Smart Innovation, Systems and Technologies, vol. 27, pp. 205–212, 01 2014. [53] C. Khammassi and S. Krichen, “A ga-lr wrapper approach for feature selection in network intrusion detection,” Computers Security, vol. 70, pp. 255–277, 2017. [54] H. H. Pajouh, R. Javidan, R. Khayami, A. Dehghantanha, and K. R. Choo, “A two-layer dimension reduction a	en_US
dc.identifier.uri	http://hdl.handle.net/123456789/1312
dc.description	Supervised by Prof. Muhammad Mahbub Alam PhD, Professor, Department of Computer Science and Engineering(CSE), Islamic University of Technology(IUT) Board Bazar, Gazipur-1704, Bangladesh	en_US
dc.description.abstract	The number of cyber-attacks has increased in recent years in both the number and varieties which demands a dynamic way of detection. Network Intrusion Detection System (IDS) leverages the key feature of Machine Learning algorithms to analyze network traffic and to build a sophisticated and dynamic system. However, the performance of Machine Learning algorithms depends on the representation of dataset.Recent research on Network Intrusion Detection has focused on feature selection and feature extraction techniques to obtain the best output and to adapt to continuously varying attacks. In this paper, we present a correlation-based technique for feature extraction from the traffic information. Our feature extraction framework builds a normal traffic profile and consider the deviation of network traffic information from normal traffic profile as the new feature set. The new derived set of features optimizes the anomaly detection technique using classification algorithm. Our evaluation conducted on KDD-CUP99,UNSW-NB15,NSL-KDD,AWID and CIC-IDS2017 dataset and outperformed detection rate for intrusions compared to other recent state-ofthe-art anomaly detection methods.	en_US
dc.language.iso	en	en_US
dc.publisher	Department of Computer Science and Engineering(CSE)	en_US
dc.title	A Correlation-based Feature Extraction method and intrusion detection framework for ML-based Intrusion Detection System (IDS)	en_US
dc.type	Thesis	en_US