A New Approach for Evaluating Intrusion Detection System

Abstract

The anomaly based intrusion detection system (IDS) is widely used based on different machine learning algorithms. The IDS is usually evaluated by its ability to make accurate predictions of attacks. In case of the binary classifier IDS four possible outcomes are possible. Attacks correctly predicted as attacks (TP), or incorrectly predicted as normal (FP). Normal correctly predicted as normal (TN), or incorrectly predicted as attack (FN). However, in case of multi classifier, when a class of attack is incorrectly predicted as another class of attack, it could not be any of the existing four instances. In this paper, a new approach is proposed to evaluate the anomaly based IDS. A new proposed metric F-score per Cost (FPC) is a one value calculated for each attack predictor. A new instance misclassification of attack class “MC” is proposed to represent the cases of wrong predicted attacks as another attack class. Based on the five instances a numerical evaluation can apply different measures to quantify the performance of IDS. In order to test the effectiveness of the proposed approach, three competitors of the “KDD CUP’99” competition are selected to measure their results by the proposed metrics. The results show that it was effective to add the MC instance. It achieves deep understanding of the IDS performance, and makes it more accurate to compare different intrusion detection systems and reflects the trade-off between the harmonic mean of the sensitivity, precision of the IDS and the misclassification paid against its detection accuracy.