Chinese Journal of Computers   Full Text
Title	Supervised Intrusion Detection Based on Active Learning and TCM-KNN Algorithm
Authors	LI Yang1),2) FANG Bin-Xing1) GUO Li1) TIAN Zhi-Hong1)
Address	1)(Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100080) 2)(Graduate University of Chinese Academy of Sciences, Beijing 100039)
Year	2007
Issue	No.8(1464—1473)
Abstract &   Background	Abstract Supervised network intrusion detection has been an active and difficult research topic in the field of intrusion detection for many years. However, there still exist some unresolved and scarcely addressed problems such as the difficulties in obtaining adequate qualified attack data for the supervised classifiers to model the attack patterns, the data acquisition task is always time-consuming and greatly relies on the domain experts, etc. Based on these, the authors first propose a novel supervised intrusion detection method based on TCM-KNN (Transductive Confidence Machines for K-Nearest Neighbors) data mining algorithm. Moreover, the authors introduce active learning method to select the most qualified data for training and thus assist TCM-KNN effectively in fulfilling the intrusion detection task. Experimental results demonstrate the proposed method has better results both in detection rate and false positives than the state-of-the-art intrusion detection methods. The method can also ensure good detection performance after optimizations by using instance selection and feature selection mechanisms. Therefore, it is more suitable for the real network applications than the traditional ones. keywords network security； intrusion detection； TCM-KNN(Transductive Confidence Machines for K-Nearest Neighbors) algorithm； active learning； data mining background The problem addressed in this paper is one of the most significant problems in network security and especially in intrusion detection field. Current intrusion detection methods are mostly based on data mining or machine learning schemes and they greatly depend on the quality of training dataset for building intrusion detection model. However, in the complex network environment training data is very scarce, difficult to acquire and the collection work is time-consuming, thus eventually result in that the detection performances of those methods are not ideal. Moreover, current researches usually ignored and hardly addressed the above problems in the recent years. To solve these problems, this paper first presents a novel supervised intrusion detection methods based on TCM-KNN data mining scheme. Secondly, it introduces Active Learning method to fulfill instance selection task for TCM-KNN, which can effectively reduce the computational cost of TCM-KNN while keeping good intrusion detection performance, thus make TCM-KNN be a good candidate for intrusion detection in real network environment. This work is supported in part by the National Natural Science Foundation of China under grant No.60573134 and the National Information Security Project of China under grant No.2005C39. These projects mainly are focused on how to secure the network security and information infrastructures by early detecting network intrusions and making corresponding responses as soon as possible. Because accurate and effective intrusion detection is the premise of intrusion response, the relevant work presented in this paper plays a vita role in the projects. To date, the authors have successfully developed supervised intrusion detection methods and unsupervised anomaly detection methods based on TCM-KNN scheme. They are embarking on applying them to large-scale network intrusion detection and response applications.