Firewalls, Intrusion Detection and Anti-virus Scanners PDF

Preview

Summary

School of Computer Science and Information Technology University of Nottingham Jubilee Campus NOTTINGHAM NG8 1BB, UK Computer Science Technical Report No. NOTTCS-TR-2005-1 Firewalls, Intrusion Detection Systems and Anti-Virus Scanners Julie Greensmith and Uwe Aickelin First released: February 2005 ©...

Description

Accelerat ing t he world's research.

Firewalls, Intrusion Detection and Anti-virus Scanners Julie Greensmith Citeseer

Related papers

Download a PDF Pack of t he best relat ed papers 

A Cont ext Aware Scan Det ect ion Syst em Joel Scanlan

218-Research-Paper-FINAL Asgar Ali Improving net work int rusion det ect ion syst em performance t hrough qualit y of service configurat ion … Waleed Bulajoul, Mandeep Pannu

School of Computer Science and Information Technology University of Nottingham Jubilee Campus NOTTINGHAM NG8 1BB, UK

Computer Science Technical Report No. NOTTCS-TR-2005-1

Firewalls, Intrusion Detection Systems and Anti-Virus Scanners Julie Greensmith and Uwe Aickelin

First released: February 2005

© Copyright 2005 Julie Greensmith and Uwe Aickelin

In an attempt to ensure good-quality printouts of our technical reports, from the supplied PDF files, we process to PDF using Acrobat Distiller. We encourage our authors to use outline fonts coupled with embedding of the used subset of all fonts (in either Truetype or Type 1 formats) except for the standard Acrobat typeface families of Times, Helvetica (Arial), Courier and Symbol. In the case of papers prepared using TEX or LATEX we endeavour to use subsetted Type 1 fonts, supplied by Y&Y Inc., for the Computer Modern, Lucida Bright and Mathtime families, rather than the public-domain Computer Modern bitmapped fonts. Note that the Y&Y font subsets are embedded under a site license issued by Y&Y Inc. For further details of site licensing and purchase of these fonts visit http://www.yandy.com

Firewalls, Intrusion Detection and Anti-virus Scanners Julie Greensmith ASAP Group, University Of Nottingham, UK email: [email protected] June 21, 2004

1 Introduction While the sharing of resources and information in an interconnected communication network is essential, it is necessary to impose access restrictions. As a consequence, systems can be vulnerable to misuse by other users through access violation attempts. A number of tools have been developed to prevent this vulnerability including firewalls, intrusion detection systems and anti-virus software. However, the differences between these tools are not immediately obvious, but do exist and play a core role in securing systems. This article will examine the process involved in using each of the tools and will highlight the differences between the tools themselves and their subsequent deployment throughout a network of computers.

2 Securing Networks Security is needed throughout distributed systems (interconnected components forming a network) in order to build dependable and trusted computing platforms. During the design phase of a distributed system, security policies are developed which account for the measures taken to ensure both the confidentiality and integrity of the system, when it is necessary. Confidentiality in this context refers to access constraints on users, and is there to protect the data. The integrity refers to the correct running of the system and the data contained on the system. Additionally, the usability of the system must be preserved, which is tied in with preserving the integrity of the system so that it is still functioning at the use level. There are several ways in which a system can be compromised, as stated in [7].

¯ Interception can occur when an unauthorised user gains access to a service or to a resource, such as the illegal copying of data after breaking into a restricted file system. ¯ Interruption can occur when files are corrupted or erased, occurring as the result of denial of service attacks or from the action of a computer virus. 1

¯ Modification involves an unauthorised user or program making changes to data or system configuration, and can also include the modification of transmitted data, leading to a breakdown of trust between parties. ¯ Fabrication is where data or activities are generated which would not normally occur. An example of this would be the addition of information to a password file in order to compromise a system. To prevent such events from taking place within a system, a security policy must be put into place, and the necessary measures taken. Such measures can include the encryption of data, correct authentication and authorisation of users with respect to data access and command execution, and the conscientious audit of log files monitoring system activity. From these descriptions it is evident that potential abusers of these systems can be both external and internal to the system. Many tools and techniques exist with the purpose of ensuring the confidentiality and integrity of a system. The use and deployment of the tools (in this particular instance, firewalls, intrusion detection systems and anti-virus scanners) is dependent upon where in the system they are placed, and indeed, the architecture of the system itself. Therefore, I will briefly digress and discuss what is meant by ‘systems’ within this context. The system in question is a network of interconnected computers and servers, forming a local area network. This network could be used for example, in the inland revenue. A diagram of the connected components is represented in figure 1. This local network additionally needs to be connected to the external world i.e. the Internet. There are several security challenges that need to be addressed for this network. The data within the system must be protected: not all users within that local network need to have access to all files on the network or the external Internet environment. Similarly, external entities may need to access the web server within the network, for instance, to access a particular forum held on the web server. These functions must be available without compromising the integrity of confidentiality of the system, data or users. The level of security and methods of ensuring this are defined in a security policy. The type of tool used and the way in which it is implemented is dependent on the contents of the policy. For example, the policy would be used to define if incoming telnet connections were permitted. If so, there are various constraints and configurations that should be applied to the system to enforce this.

3 Security Measures 3.1 Firewalls Firewall systems are commonly implemented throughout computer networks. They act as a measure of control, enforcing the relevant components of the security policy. A firewall can be a number of different components such as a router or a collection of 2

Figure 1: A simple network structure

3

host machines. However, the basic function of a firewall is to protect the integrity of the network which is firewall controlled. There are different types of firewall that can be implemented, with the choice of firewall being dependent upon the security policy and the level of deployment in the system. 3.1.1 Packet Filtering Firewalls Packet filtering firewalls work at the transport layer of the seven layer model[8]. This means that they are commonly deployed on routers and act as a bottleneck between the local network and the external Internet. As the name suggests, a packet filtering firewall examines a packet passing through it, comparing it against a set of criteria for what is permissible either in or out of the network. The criteria for this is defined by the security policy. There are two ways in which packet filters operate; either accept all packets except those which are specified; or deny all packets except those which are specified. The advantage of the ‘accept’ method is that it gives legitimate users of the network greater flexibility. For example, a remote user of the system from a previously unseen IP address (e.g. in an Internet cafe) could log in to an office machine from a remotely connected laptop while working out of the office. However, it also increases the vulnerability of the network because not all attacks could come from rules which are already known: this is why the ‘denial’ paradigm is more likely to be used. This method should be deployed frequently but often isn’t due to a lack of understanding from persons responsible for the configuration of the firewall. Denying all that is unknown can give greater security, however, it can cause inconvenience to legitimate users. Packet filters can examine the following attributes of a packet:

¯ Source IP address ¯ Destination IP address ¯ TCP/UDP source port ¯ TCP/UDP destination port If in the example network an external user was trying to connect to port 23 of a machine on the local network, then it is likely that the external user is trying to TELNET into that machine. This operation is likely to not be authorised and therefore the firewall on the router would not permit the transmission of the packets into the network. 3.1.2 Circuit Level Gateways The situation could arise when an external user (not from the local area network) wishes to access information on a file server, behind at least one firewall. The security policy for the network would not permit a direct connection between the external user and the file server (as shown as part of the LAN in Figure 1) as this could leave the network vulnerable to attack. The solution to this is for the two parties to create a ‘tunnel’ between the two components, employing a method of encryption in the connection. The initial connection request is filtered (and is subject acceptance based on the security 4

policy) but all packets following are not, as it acts as a relay between the two entities. In this case is important in this case to explicitly state the use of circuit level gateways in order to avoid the exploitation of the network. 3.1.3 Application Gateways Application gateways, also known as proxies, are a commonly used firewall mechanism. It is feasible to want a particular component of a network, such as a publically available interface e.g. an online enquires form, to be available to entities outside of the local network. While remote access to other components of the network may not be allowed, the inclusion of components in a demilitarized zone ( in between the two firewalls) would allow access to components which were needed by the external network. Restricting the access to components via a DMZ, and through the use of a proxy server allows external users to perform functions on, for example a web server, but would not disclose the architectural details of the LAN. As with circuit level gateways, this proxy can act as a mediator between an external entity and a component behind the packet filtering firewall on the main router. However, unlike circuit gateways, application gateways can filter IP traffic. This is an advantage because it would not allow certain actions to be taken once a connection to the proxy has been made e.g. it can prevent anonymous FTP log-in to the system. Proxies can also act as caches for the local users accessing the Internet. This can be useful in the event of restricting access to certain blacklisted web-sites. For example, in corporate LANs, where common mail providers such as hotmail and Yahoo, cannot be accessed. Allowing employees to surf such sites is seen as a waste of resources, not to mention a breeding ground for viruses1 . Again, the pre-defined security policy, if adequately prepared, would define the access permitted to each individual user of the network. Additionally, application gateways can perform packet logging for a post hoc inspection of the traffic going both in and out of the network. The disadvantage with using an application gateway is that it requires a multi-stage handshake for the initialisation of a connection which could slow down the performance of that application considerably as opposed to making a direct connection. Due to the optional requirement for restricted commands execution, as in the case of FTP through a proxy, then modified clients may need to be installed, which is extra work for both the system administrators and the users. Hence, the transparency of the service to the users is affected. 3.1.4 Other Points to Note One feature of firewalls is that they should provide a high level of user transparency, meaning that the end user should be unaware of the action of the firewall, so quality of service is maintained as a result. Transparency is high for packet filtering firewalls as the user is not always aware of the firewall until a transmission is denied. Application gateways have a lower transparency as it often requires the users to use modified software clients in order to use the proxy’s service which could result in the user attempting to bypass the system entirely. 1 More

about this in a little while

5

Recently, Stateful Multilayer Inspection Firewalls have been deployed, at the application layer, transport layer and network layer, which combines the packet filter property with the packet sniffing capabilities of gateways. Stateful inspection can be used to prevent attacks such as the Loki or Smurf denial of service attack, as the firewall would be aware that the original packet was not sent as a broadcast message from a machine on the network [6]. However, experience has shown that these systems are difficult to manage due to the complexity of the rules and the processes involved, rendering them less secure than their separate counterparts. With respect to the actual hardware required in order to implement firewalls, there are two types, namely bridging firewalls and firewall routers. Bridging firewalls are software firewalls that can be run on a standard machine, using a firewall such as IPtables. Firewall routers are a specific piece of hardware designed to perform as a router and a firewall, and have been implemented as the first line of defence in many networks. Bridging firewalls are becoming prominent due to their ease of configuration, ease of initial installation, good performance (little computational overhead) and their ability to be stealthy and so are less likely to be attacked [15] .

3.2 Intrusion detection systems As previously stated, the majority of traffic on the network is not malicious, and users within a system do not set out to gain unauthorised access to information. However, the use of an intrusion detection system is becoming increasingly commonplace due to both the increase in complexity of attack and of the computer systems themselves. As with any complex system, emergent properties can arise unexpectedly. In the case of such systems, unexpected interactions between the various components can give rise to vulnerabilities which can be exploited. Additionally, the use of a firewall may not prevent internal abuse from an otherwise legitimate user of the system (either for breaches of confidentiality or for system integrity). When defining what intrusion detection systems are, it perhaps makes more sense to describe what they are not. IDS are not a preventive measure. They will not stop intruders breaking into a system. Neither will they prevent internal damage to a system. As the name clearly states, they are a detection system, thus implying that abuse of a system is reported as and when it happens. In essence, they are analogous to a burglar alarm in a house. Such an alarm can trigger an immediate response e.g. call the police, can be used to alert the owner that unauthorised behaviour is taking place, or simply to cause annoyance to the neighbours. As with firewalls, different types of intrusion detection system exist. There are two different ways of classifying an IDS. The first way is to classify based on the method of detection, in the form of either misuse detection or anomaly detection. An alternative way is to classify based on the position of deployment within a network. IDS can be either network based, host based or application based, depending on where they are deployed [9].

6

Irrespective of the specifics regarding implementation and deployment, IDS function in a generic way. Input data from a system is collected and processed into a manageable format. The data items are classified as a threat or harmless. If a threat is detected, then a response is produced, usually in the form of an alert to the system administrator. A more detailed explanation of the process is as follows: 1. Data has to be captured, often in the form of IP packets. 2. The data are decoded and transformed into a uniform format, through the process of feature extraction. 3. The data are then analysed in a manner which is specific to the individual IDS, and classified as threatening or not. 4. Alerts are generated if and when a threatening pattern is encountered. However, precautions must be taken to stealth this part of the system, so that an intruder can not spoof alerts (potentially leading to a denial of service attack). Various techniques are employed to produce correlations of the results; this can be done using an automated system, or manually. 3.2.1 IDS Classification based on style of detection Misuse Detection : This type of IDS can also be called a signature recognition system. Misuse detection systems rely on the accurate matching of system or network activity [19]. This method of detection is accurate for matching behaviour against a list of already documented patterns, known as signatures. An example of this type of IDS is a system known as Snort [4]. The means by which snort functions involves the use of software component processing information regarding network connections. Snort examines the network traffic at its position on the network in a passive manner: it sniffs the network. Examination of the headers and content of TCP packets is performed and matched against patterns contained in a signature database. If certain patterns of traffic are captured, then an alert is generated2. The use of only already known signatures means that the system will produce only a few false positives, or false alarms where an alert is generated yet there is not actual attack. There is a relatively high maintenance cost in that the signature base has to be kept up to date, else potential attacks could go unnoticed. Additionally, this type of system can miss highly novel attacks to which a signature does not yet exist, giving a higher rate of false negatives (where a real attack is not detected) than would be desired. Missing an actual attack is probably worse than being inundated with false alarms, though this is debatable. 2 Let me pose a question: is this really an intrusion detection system, or is it a TCP pattern detection system? This depends immensely on how you define an intrusion

7

Snort is an open source IDS which implements a range of pattern matching algorithms of the input data and produces alerts based on the matching of the input to a signature base. For example it is likely that multiple port-scans on a particular component would raise some sort of alarm. The advantage of the system being open source is that if a vulnerability is found, then it is likely to be posted on a user forum. The idea being that 1000 pairs of eyes are more likely to notice a vulnerability in the software than a select few hired ‘experts’. A recent example of this is a vulnerability found in the snort program itself, in which an integer overflow was discovered in one of the stream processors responsible for the calculation of the segment size for re-assembly. This could lead to a buffer overflow which could turn into a denial of service attack on the system itself, or even remote command execution of the host running the program (for examples, see [18]). Anomaly Detection : The goal of anomaly detection systems is to successfully classify user or network behaviour as normal or abnormal, based on a profile of information gathered during a training period. This is performed by taking into account the amount of background noise or user variation which is intrinsic to the system. The characterisation of what constitutes ‘normal’ behaviour is certainly a non-trivial issue. There have been many approaches used in order to perform this classification, including statistical models, Markov chains, neural net...