Virtual Burglar Alarm - Intrusion Detection Systems (Part 1)
Thomas Biege
Table of Contents
- What Is An Intrusion Detection System?
- The Structure Of An Intrusion Detection System
- Analysis Techniques
1. What Is An Intrusion Detection System?
Simply put, intrusion detection systems (ID systems) are virtual burglar
alarms. ID systems automatically recognize attacks on network security and
computer systems. Specifically, ID systems monitor compliance with security
policies.
An IDS collects current state data from one or more computer systems and/or
computer networks, analyzes the data, and initiates pre-defined actions
based on the analysis results. Depending on the analysis procedure, the
analysis result can yield the following conclusions:
- No security policy violation
- Security policy violation
- Behavior that is suspicious to some extent and may lead to a security
policy violation (see state devices).
Also, the system can always state the cause for the alarm.
Generally, intrusion detection systems can guard passively as well as
actively against attacks. In addition, ID systems are intended to
facilitate the legal and technological prosecution of attackers. They also
provide information that can be used to estimate and repair damage and to
identify the security hole the attacker exploited.
2. The Structure Of An Intrusion Detection System
For the purposes of this discussion, we are using the Common Intrusion
Detection Framework's (CIDF) IDS specification. According to the
CIDF, an IDS consists of four components, known as boxes.
- Event (E-) Box
- Analysis (A-) Box
- Countermeasure (C-) Box
- Storage/Data (D-) Box
The event box is responsible for the seamless collection of audit data,
such as raw network packages, user-level and/or kernel-level log data. The
data is then converted into a unified format and
sent to the analysis box. Depending on the configuration, the data is also
sent to the data box for a more detailed analysis later on, if needed.
First, however, the system removes redundant information to save space and
to speed up information processing. The bulk of IDS work occurs in the
analysis box. This is where the data from the event box is examined for
attack patterns, anomalies and non-compliance with security provisions. If
there is a user-defined action in response to the overall analysis result,
the countermeasure box, also known as the intrusion response unit, executes
this action. Moreover, the analysis result is sent to the data box. The
following analysis procedures are commonly used:
- Misuse Detection:
-
- Expert system
- State transition
- Description languages
- Anomaly Detection:
-
- quantitative analysis
- statistical measurement
- rule-based systems
- neural networks
While the reactive measures executed by the countermeasure box can have
many different forms, they generally fit into one of two categories:
- Passive:
-
- Send E-Mail
- Send message to a pager or cell phone
- Play audio data
- Open a dialog box
- Increase the volume of audit data to record
- Turn off critical systems and services
- Customize the packet filter rules for firewalls
- Send e-mail to the administrator of the system that was attacked
- ...
- Active:
-
- Information about the system from which the attacks originate, such as, for
example, logged-in users (fingerd, rusersd or identd), services offered
(port scanning) and the operating system being used (OS
fingerprinting) - the latter two pieces of information can be used for
additional attacks
- shut down the attacking computer system with
denial-of-service (DoS) attacks
- ...
Generally, active measures are not deployed, since doing so might cause the
attacked party, the victim itself, to commit crimes. The issue is further
complicated by the possibility that the opponent may not even be using the
system from which the attacks originate. The attacker can use IP address
spoofing to hide his actual location. IP address spoofing refers to the
process of forging an IP packet's originating address. The system the
attacker is currently using may simply be another one of the attacker's
victims.
Unfortunately, the attacker can also use the passive actions
initiated by the C-box and turn them against the network that is being
protected. If the attacker disguises his attacks to make them appear as if
they originated from a business partner's IP address, and if the C-box then
changes the firewall configuration to block packets from the partner's
network, collaboration with the business partner is no longer possible. In
addition, if the IDS sends too many alarm messages by e-mail or pager, the
excessive traffic can overload the mail or modem servers (denial of service
attacks). As a result, the security officer (SO) may overlook truly
important messages. He may turn off the pager / cell phone altogether or
delete the e-mail messages unread.
To prevent these dangerous consequences,
you can, for example, configure the IDS so that certain IP addresses will
not be blocked. Instead, only the SO will be notified or checks will be
performed. Identical types of alarm messages should be brought to the SO's
attention only once, along with the quantity of messages received.
3. Analysis Techniques
In recent years, three types of analysis procedures were refined. Misuse
detection is the oldest method for spotting intrusions. This procedure uses
a pattern matching approach. The system compares event box data with attack
signatures from a database. If the comparison results in a positive match,
the system recognizes a security policy violation and reacts accordingly.
Misuse detection remains the most commonly used procedure in the commercial
and non-commercial sectors. The procedure is easy to implement and use and
it is not very prone to false alarms (false positives). Misuse detection,
however, has a major drawback. This method recognizes known attacks only.
Consequently, new attack patterns that have not yet been added to the
attack signature database do not trigger an alarm (false negative) and thus
go unnoticed.
To compensate for this flaw, a new approach to solving the
problem led to the development of a method known as anomaly detection.
Anomaly detection is based on the premise that anything outside the realm
of "normal" behavior, is, by definition "abnormal" (i.e., it is an
anomaly), and therefore constitutes an attack. Compared to misuse
detection, this method's advantage is the ability to recognize new attacks,
since they are defined as abnormal behavior. In addition, there is no need
to implement and maintain a database of attack patterns. Nonetheless,
anomaly detection comes with its own set of problems that significantly
impede its use in the commercial sector. Anomaly detection procedures must
first acquire knowledge of what constitutes "normal" behavior for a network
or computer system by creating user and system profiles. This phase alone
is an obstacle and could be exploited by an opponent who could teach the
IDS to classify attacks as normal behavior. Thus, in the future, the ID
system might then no longer recognize that type of attack as an
unauthorized intrusion. Another drawback is the high rate of false
positives triggered by disruptions of normal system activities that are not
actually attacks. Moreover, compared to misuse detection, the
implementation of anomaly detection is more difficult, since the latter
method involves more complex procedures.
Another method currently still in its infancy is based on a relatively
simple, yet effective approach and is known as burglar alarm, passive trap,
or strict anomaly detection. The premise is that anything that isn't "good"
must be "bad". Anomaly detection uses a similar approach, but the
difference is that recognition of attacks is based on pattern recognition,
just like misuse detection. This means that the system's normal, known
behavior is stored in a database as a signature. Any kind of system
activity that does not match a pattern in the database is classified as
abnormal behavior and indicates an attack. There are only a relatively
small number of patterns that need to be saved to the database. In contrast
to misuse detection, these patterns do not have to be implemented for every
new attack, but only in case of IT system changes. This translates into a
reduction of the administrative burden during operation compared to regular
misuse detection systems.
Below is a simple illustrative example:
The e-mail proxy server in a DMZ
network segment can send and receive TCP packets only through port 25. In
addition, these packets can be sent only to the Internet or the
secure/internal network. This is the normal behavior stored in a database
as a pattern. The IDS then checks the e-mail proxy server's packet flow
against the stored pattern. If the e-mail proxy server's behavior deviates
from this pattern, the system may be under attack. The patterns do not
necessarily have to be based on abstract network connections. They can also
reflect deeper levels, such as header information for the individual
network level or an operating system's system calls.
To ensure that the
error rate with respect to recognizing attack remains comparatively small,
anomalies that periodically affect a network must be explicitly defined as
exceptions or fall within a tolerance range. Thus, while false positives
are possible, there are significantly fewer of them than occur with anomaly
detection, and false negatives are virtually impossible. If attacks remain
below the tolerance threshold, the system does, of course, detect them.
Below are brief descriptions of the different analysis techniques for
misuse and anomaly detection. Additional information about strict anomaly
detection and other alternative analysis procedures (immune system, genetic
algorithms) is not included here, since these techniques are still too new
and have not yet been used in practice.
3.1 Misuse Detection
Expert Systems
Expert system capabilities turned out to be useful for data analysis at an
early stage. The following ID systems made use of expert systems:
- MIDAS
- IDES
- Next Gerneration IDES (NIDES)
- DIDS
- AID (uses RTworks)
- Emerald (uses a variation of P-BEST)
- CMDS (uses the open system CLIPS)
Users can easily program expert systems with if-then-else expressions. The
disadvantage of expert systems, however, is the fact that they cannot
quickly process a large number of these rules, since expert systems are
merely interpreters and thus are slow. In addition, the quality of an
expert system is always only as good as the individual who programmed it.
Changing the architecture of the rules is a difficult task, since doing so
affects the remaining rules. Due to the fact that expert systems use
if-then-else conditions exclusively, the systems cannot handle uncertainties
related to the attack analysis, as they are limited to "Yes" or "No"
statements only.
State Transition (State Transition Automata)
With misuse detection, optimized state transition devices for pattern
recognition provide a flexible and powerful means for spotting intruders.
STAT was the first ID system to use this technique, followed later on by
USTAT, NetSTAT and WinSTAT, all of which were developed at the University
of California, Santa Barbara. At the ACM Workshop 2000, three researchers
from the University of California, Santa Barbara introduced a flexible and
powerful description language named STAT for state transition-based ID
systems.
The above image illustrates the path from a start state (circle s1) through
transitions (arrows) to a possible final state (circle s3) that signals an
attempted attack. A system intrusion usually consists of multiple actions.
Every action is reflected by a transition. The states indicate the state of
the system, which can consist of, for example, user privileges, existing
files, or the current state of a network or system service.
The ID system
maintains a state transition diagram for every attack scenario. If there is
activity on the system, the analysis unit checks the activity against every
state transition diagram and changes every state device to the new state if
the activity matches the transition. This procedure makes it possible to
determine the probability and the course of an attack and to predict the
attacker's moves by checking the different state devices for their current
states. If the final state is reached - that is, in case of a successful
intrusion - the intrusion response unit (C-box) executes the corresponding
result. The state diagrams provide a very abstract picture of the attack
patterns, which means that they are easy to understand and independent of
audit data format and technical details. State devices constitute an
attack's minimum activity. That is why variations of an attack can be
covered with only a few small state devices. State transition-based ID
systems can even recognize highly coordinated and very slow attacks. An
improved variation of state transition analysis was implemented using the
so-called Colored Pertri (CP) network in IDIOT IDS.
Description / Interpreter Languages
Researchers have developed languages to describe attack scenarios in order
to facilitate the use of analysis units in ID systems even further. The
following examples are well known:
- RUSSEL
- STALKER
- N-Code from Network Flight Recorder (NFR)
- STATL
- Plus, of course, regular expressions as in SWatch or LogSurfer and others.
3.2 Anomaly Detection
Quantitative Analysis
There are different types of quantitative analyses. The most common one is
threshold detection, also known as "threshold and triggers". Threshold
detection uses counters to display system and user activities. The
counter's value is defined as normal behavior until it reaches a specific
threshold value. If this value is exceeded, the system reports a potential
attack. For example, imagine that a user has made three unsuccessful
attempts to log onto a system within the past two hours. In this case, the
threshold value is "3", and the IDS reacts by, for example, locking the
user's account.
Heuristic threshold analysis is a more advanced form of
threshold analysis. With heuristic threshold analysis, the threshold values
are dynamically adaptable instead of fixed values. In our example, the
threshold value would not consistently be set to 3 under all circumstances.
Instead, the value would be calculated based on the user's past login
behavior. For example, the system might calculate the threshold value based
on the past number of failed logins, plus a standard deviation.
Target-based integrity checks are another quantitative analysis technique.
This method watches for changes in system objects (files, programs,
hardware, etc.) that should, in fact, never change. Tripwire is the best
known example of a target-based integrity check. Tripwire monitors files
and programs by generating cryptographic hash values (MD5 algorithms) from
these files and programs. The program then saves these MD5 hash values to a
database and regularly compares them to the hash values of the system's
current files and programs. If the hash values do not match, the
corresponding file/program was modified. For example, an attacker might
have replaced the file/program with a Trojan horse or falsified the
contents of the password database. These quantitative methods, however, are
not only used to spot attacks, but also for the purpose of data reduction.
NADIR IDS exploits the ability to numerically express system attributes and
user behavior to reduce redundant and useless information among the audit
data. NADIR's developers used vectors to display user activities. These
profile vectors can then be utilized by expert systems and to perform
statistical analyses.
Statistical measurements
IDES and NIDES use statistical procedures and were the first functional
anomaly detection systems (actually, they are hybrid ID system, since they
use both state transition-based misuse detection as well as anomaly
detection).
The systems administer a large number of statistical profiles
that describe the normal behavior of the users and system components. The
profiles are updated periodically. If there is a severe deviation from the
normal profile, the system reports a security violation.
Haystack is
another IDS based on statistical measurements. Haystack uses two groups of
statistical profiles. The first group serves to compare user behavior with
attack behavior while the second group is used to detect deviations from
normal behavior. Haystack uses statistical processes to detect attackers
that appear to be regular users because the attackers utilize their
accounts. Since the attacker's behavior differs from that of the user whose
account is being compromised, the IDS takes note and informs the SO. In
addition, Haystack can spot previously unknown attack statistics and
methods that a system based on quantitative analysis cannot recognize. The
major disadvantage of many statistical procedures is their inability to
work in real-time, which makes it impossible to respond immediately to an
attack. At the time this analysis method was being developing, batch mode
operation was entirely sufficient, since the method was created to monitor
mainframes and today's widespread networks did not yet exist. The attempt
to use statistical analyses for real-time ID systems failed, since doing so
requires high-performance systems.
Moreover, statistical measurements can analyze one activity only, but not a
sequence of activities. This feature severely limits the usefulness of such
systems in terms of the number of attacks they can detect, since most
attacks consist of a sequence of actions.
The procedures described above are all based on assumptions about what is
normal system behavior and what is not. The threshold value used by the IDS
must not be too low in order to avoid false alarms, which lead to the IDS
no longer being taken seriously. On the other hand, the threshold value
must not be too high, or the system won't recognize attacks (false negatives).
The approaches described below do not require parameters for the threshold
value and strive to compensate for the drawbacks of parameter-based
analysis procedures.
Rule-based Systems
Generally, these systems work just like statistical systems. The only
difference is that rule-based systems use rules instead of statistics. For
example, Wisdom and Sense (W&S) is a rule-based IDS. With W&S, the rules
are either entered by the user (to implement the security policy), or W&S
can generate the rules from old audit data. The Time-Based Inductive
Machine (TIM) also uses rules, yet takes an entirely different approach
than other anomaly detection systems. It does not hunt for individual
events, but monitors their temporal sequence. An anomaly occurs when the
temporal sequence of system events is violated.
Neural Networks
Neural networks use adaptive learning techniques to recognize abnormal
behavior. That is why neural networks do not require user-specified
parameters. The neural network must first be trained with clean data, that
is, data not contaminated by attack activities. Due to their ability to
learn, neural networks are quite valuable when it comes to spotting
abnormalities. Unfortunately, neural networks cannot be used to determine
the cause of an abnormality. Thus, the SO receives information only about
the existence of a security violation and no information about its cause.
The effort to circumvent this problem has led to the development of neural
networks that are limited to only one type of attack. In other words, if
the neural network for SYN flooding fires, the SO knows which type of
attack he is dealing with.
References
- Books
- Bace, Rebecca Gurley (2000): Intrusion Detection. - MTP, Indianapolis.
- Proctor, Paul E. (2001): The Practical Intrusion Detection Handbook. - Prentice Hall, New Jersey.
- Papers
- Sundaram, Aurobindo: An Introduction to Intrusion Detection.
- von Helden, Josef: Grundlagen,
Forderungen und Marktübersicht für Intrusion Detection Systeme (IDS)
und Intrusion Response Systeme (IRS.
- Kumar, Sandeep & Eugene H. Spafford (1994): An
Application of Pattern Matching in Intrusion Detection. - Technical Report CSD-TR-94-013.
- Anderson, J. P.: Computer Security Threat
Monitoring and Surveillance.
- Crosbie, Mark & Gene Spafford: Active
Defense of a Computer System using Autonomous Agents.
- Ptacek, Thomas H. & Timothy N. Newsham (1998):
Evasion and Denial of Service: Eluding Network
Intrusion Detection. - Secure Network, Inc.
- Curry, David A., Herve Debar & Ming-Yuh Huang:
Intrusion Detection Message Exchange Format; http://www.silicondefense.com/idwg/draft-ietf-idwg-idmef-xml-02.txt
- Bishop, Matt (1995): Standrad Audit Trail Format. - National Information Systems Security Conference: 136-145
- Common Intrusion Specification Language
(CISL); http://www.gidos.org/drafts/language.txt
- Vern Paxons; Bro: A System for Detecting Network Intruders in RealTime
Thomas Biege has been working in the SUSE security team for the past two years. He studies general computer sciences at the technical college Dortmund. Presently he is busy with the development of a host-based intrusion detection system, which he will present in his diploma thesis.
|
|
|
Contact 
