27-07-2012, 02:22 PM
Proactive Network Anomaly Detection
Proactive Network Anomaly Detection.docx (Size: 19.3 KB / Downloads: 23)
Abstract-:
Network anomaly detection is the detection of abnormal conditions
in the monitored network either due to a malicious attack such as
DoS, network intrusions or non malicious events like equipment failure,
improper network configuration, etc. The ability to detect such
abnormal conditions before the actual faults occur in the network allows
the network administrator to take corrective actions in order to
minimize the losses due to the faults. Several approaches for network
anomaly detection exist such as traffic modeling, network probes, rule
based approaches, etc. One such approach uses statistical techniques
to model the network behavior and analyze deviations from the normal
behavior in order to detect network anomalies.
Introduction-:
The most commonly used method for network anomaly detection today is to
Monitor a set of network parameters for threshold violations. Any deviation
beyond the threshold is treated as anomalous behavior and correspondingly
alarms are generated. However using hard thresholds for alarm generation
leads to undetected faults and/or high false alarm rates. Based on this model
the following view can be stated : Network anomalies are characterized by
correlated transient changes in measured network data that occur prior to or
during the anomalous event.
A statistical approach to the problem is to use rigorous statistical analysis
of network data collected to quantify network behavior. The normal behavior
of the network is modeled using signal processing techniques. Abrupt
change detection is then used to detect anomalous behavior patterns in the
network traffic data. Anomaly detection is done at device level to obtain
the device view of the network health. Information from all such devices is
then combined together to obtain the overall network health.
In section 2, a brief overview of other approaches to anomaly detection
is presented. The rest of the document discusses about the statistical approach
in detail. Section 3 introduces the data sources for the system while
section 4 looks into the segmentation mechanism used for obtaining stationary
windows. Abrupt change detection is explained in section 5. Section
6 talks about how the variable level alarms can be combined together to
generate node level alarms. Section 7 concludes the discussion with fault
classification.
2 Anomaly Detection Approaches
2.1 Rule based approaches
The approach is based on expert systems where rules for faulty behavior of
network are maintained in a huge database. These rules are used to detect
anomalous network behavior. The method is however too slow and requires
previous knowledge of network fault conditions in the network. The system
is unable to detect unknown faults or faults with varying behavior and as
such do not adapt well to evolving network environments.
2.2 Finite State Machines
In this approach a probabilistic finite state machine model is built for known
faults using history data. The alarm sequences collected from various parts
of the network are modeled as states of the system. The best mapping for the
given sequence of alarms is identified. The difficulty with this approach is
that the number of states increases with the number of faults and extensive
offline learning is required before its deployment.
2.3 Pattern Matching
The pattern matching approach builds a traffic profile for the given network
using online learning. The templates thus obtained for various network
parameters are used as baselines for comparisons with instantaneous values
to obtain an indication of the network state. This approach is dependent on
the quality of the network profile generated and might not scale well with
the network size.
2.4 Network Probes
Tools such as ping and traceroute are used to measure various network parameters
like end-to-end delay and packet loss. These probes provide instantaneous
measure of the network behavior and can be used to detect the
network faults. However the approach assumes the existence of symmetric
paths between the source and destination and that the probe packets are not
differentiated from regular packets which might not always be true. Thus
the network behavior modeled may not be accurate.
2.5 Statistical Analysis
Statistical techniques can be combined with on-line learning to continuously
monitor the network behavior. Individual nodes in the network are monitored
to obtain its view of the network health. The network health function
information obtained from all the network nodes is then combined to obtain
an indicator for the entire network state.
3 Data Source
Obtaining the right type of network performance data is essential for effective
anomaly detection. Management Information Base (MIB) variables provide
fine grained data for each network device and hence are ideal data source
for network anomaly detection.The MIB variables can be classified into various
groups like system interfaces, address translation, internet protocol(ip),
internet control message protocol(icmp), transmission control protocol(tcp),
user datagram protocol(udp), exterior gateway protocol(egp), simple network
management protocol(snmp), etc. Each group of variables describes
special functionality of the network. The group of MIB variables to be used
depends upon the device being monitored as well as the protocol level at
which the device works. No single variable provides complete information
about the behavior of the network device and hence the proper choice of a
subset of the variables is essential.
For example, if the network device to be monitored is a router which
works at the network layer then the group of MIB variable relevant to the
router are the ip group of variables. Some redundancy exists even within
the group and hence proper subset of these must be chosen so that the
node information is adequately obtained. For the router the following MIB
variables could be chosen
1. ipIR - Number of datagrams received by the ip layer of the router.
2. ipIDe - Number of datagrams forwarded to the higher layers.
3. ipOR - Number of datagrams received from the higher layers.
4 Segmentation
The MIB variables are sampled periodically to obtain a time series which
is processed independently using a sequential change point algorithm. The
statistical properties of the MIB variables change in response to the network
fault. These changes are subtle and must be differentiated from normal traffic
variations in the network. The challenge here is to detect these subtle
changes that precede any fault conditions in the network inspite of the nonstationary
nature of the MIB variables. The non-stationary time series is
segmented into piecewise stationary segments and signal processing techniques
are used to detect abrupt changes in the statistical properties of the
the variables. Within each segment the MIB data is modeled using a linear
first order Auto-Regressive(AR) process that takes into consideration the
auto-correlation in the time series.