Extrusion Detection Systems; the art of network monitoring by: Ron DuFresne (c) 2001 I've watched over a number of years, as tools have developed to guide administrators in determining how, why, and from where their systems are attacked. Applications such as TCPD (TCP wrappers) was one of the first tools to help in limiting access to protocols and applications that have very insecure control means and are often used to compromise systems. Soon after came along some of the early firewalls like the now famous Firewall Toolkit. Since that time, many flavors and versions of firewalling products, protocol and application proxies have emerged, mostly to protect the perimeters of networks. To aid in perimeter protection many scanners have been developed, at first merely to demonstrate the insecurities and exposure systems and networks pose to companies and individuals alike. Not too many years ago a new set of buzzwords started to crop up, Intrusion Detection Systems (IDS's), that helped administrators monitor and watchout for intruders probing and prodding systems for exposure and weakness. And to this day the most popular, yet, probably most ill conceived placement of these systems is on the outside of the firewall or directly upon the same system. I've long been a proponent here that the better placement of such devices is behind the firewall. For these can be very noisy devices if placed outwards and exposed for all the black and grey hats to pound upon, let alone the large numbers of improper packets that many routers needing some gear replaced, and failing network cards, might strew about as they start to falter. Let's face it, systems and network admins are vastly overworked as it is, putting in far too many hours a day/week/month/year. We do not need to be woke every night at 2-5AM to deal with false positives and alerts that exposed IDS systems are prone to generate. We certainly don't have the time to be running back to look at the IDS and various system logs throughout the normal day even, to check into each and every stray packet such exposed IDS systems will 'discover'. Few, if any companies will provide the funds to pay for personnel to sit in front of a screen and continuously monitor the traffic generated by an IDS. I know there are many that might well argue the point, but, let's face it, an exposed IDS system is best suited to those folks conducting the honey-pot project and the others that like to peruse tons of logged information to study and report about Internet traffic trends. Monitoring the traffic from an exposed IDS can be near to impossible to do real time, it makes the eyes water and the brain hurt to say the least. I've seen enough postings to the various security lists and Usenet groups from very skilled persons asking for help in determining exactly what they are seeing in log extracts and network traces from their IDS systems and firewalls (these are probably the second most asked questions, the other being how to prevent users from violating network policies and block various ports and applications from the desktop outwards), to know that using these tools to help determine what new attacks are working their way into the flood of Internet traffic is very often beyond the means and understanding of most experts in the field whose job it is to maintain the routers and firewalls at the network edge. It takes very refined skills to know how to recognize new attack methods and modes within, most often very congested network streams of traffic. Muting the IDS system to not alert, but log for later analysis, again, seems to be more for the realm of those reporting trends, and defeats one of the finer aspects of the better IDS systems, near realtime proactivity. This really counts when something has bypassed the firewall and is headed at the network backbone. The argument that an external, exposed IDS is a great tool for qualifying security spending to management is, in our mind, a demonstration of overkill. First, this information should already exist in the logs of the perimeter devices maintaining the companies network security policies. Secondly, if the perimeter devices are not logging traffic attempts and flows, with matters as they are on the Internet, a laptop with a packet sniffer running for a short period would certainly suffice in demonstrating the need for security spending. Better justification comes from demonstrating the costs of a loss or even an outage of services across the company backbone. If specific tools need justification, one has ample evidence from the security related mailing list, Usenet groups, and security related trade journals and security related web sites, let alone the GAO reports on our own government's audit processes as well as the daily news. There is a plethora of information available to justify expenditures already. We are also unaware of any tools yet designed, capable of monitoring for new traffic and unknown packets that might emerge and be injected into network streams with wicked intent. Watching for something new and unknown is difficult, to say the least. The anti-virus tools available need constant updates to keep on top of the new viri and trojan variants constantly being unleashed. The veracity and nastiness of the newer viri and trojans has worked to ever increase the need for detections of such virulent code on desktops and servers. These newbreed nasties are now working with applications and network protocols to help themselves propagate and infect systems at tremendous rates. Quite often they can bring even large bandwidth networks to their knees merely with the overflow of packets they produce to further spread their destructive payloads. Often these sharp spikes in additional, seemingly legitimate, traffic are the only signs that something is amiss on the corporate LAN/WAN infrastructure. Even the various security scanners require updates and rewrites to deal with the new attack models discovered time and again. So the arguments for placement of an IDS on the external side of the boundry devices weakens under these observations. The variety and scope of the various IDS systems is already quite large, and, we'll not discuss the various attributes and abilities of those now available or those under construction here. Let it only be stated that we prefer IDS systems that silently monitor without injecting further packets into the network stream, as in my mind, those that require packet injection only serve to further add to already over-congested network flows, requiring that one parse out the systems particular packets in traffic analysis. I've always liked an IDS system to backup the firewall, to stand behind it and give notice when something has been able to cross the firewall and reaching the network boundary, becoming a potential intrusion to the systems behind the devices setup to guard the interior network from unwanted and malicious traffic. I've often described a properly placed IDS, monitoring the the internal network edge, as a silent warrior. A system that only wakes and shouts out alerts when traffic slips by the border devices, as a last alarm and guard, to warn that something is amiss on the perimeter, posturing a potential attack to what still amounts for the most part, on most corporate networks as the 'soft chewy center' of the corporate LAN/WAN. The ideal IDS sits silently, and functions as a tool that helps admins and network guru's feel warm and fuzzy that their router rules and firewalls are doing the job they have been designed to do, protect corporate assets. And yet, I have to admit this is a somewhat shortsighted description and use of IDS systems. Over the past few years a number of firewall wizards and networking guru's have come to understand the value of IDS systems in monitoring what is escaping from their networks, and even for monitoring for various application and service outages on the internal network structure. The detection of what traffic is departing and traversing the network can be "as", if not "more", important these days, in determining the effectiveness of efforts to protect the internal assets and structures that system and network administrators are hired to do. Most often the devices, routers and firewalls are very permissive in what they allow outbound. While these systems are extremely restrictive of what they will allow inbound, if that traffic originates within the perimeter systems are often designed to permit traffic flows back and forth across the network boundaries to facilitate the flow of information and work of the employees the network supports. And ever since computing networks have existed the concept of PEBKAC (problem exists between keyboard and chair) has been, and remains the biggest issue in the flow and progress of viri and trojans and various other forms of network compromise. This is one of the main reasons that many wizards and gurus have taken to installing a properly tuned IDS system facing inwards on the network, to set off alarms and alerts about desktops and servers that have been compromised in some manner and are spewing traffic not intended to flow out the corporate border. These IDS systems often function as additions to the anti-viri and trojan software installed on the network servers and desktops that need to be updated to the new nasty code that is released upon unwitting users almost daily. These systems further help the admins and network persons maintain the corporate network policies from within as well as without the network boundaries. Just as the external IDS is likened to the last line of defense and alert, the internal EDS systems often are front guards capable of saving a company from embarrassment and/or damaging information leak or internal system compromises. The tuning and taming of these internal monitoring "EDS" (Extrusion Detection Systems) is very crucial, so that false positives and alarms are kept to a minimum, the results should be mostly mirrors of the corporate network policies they are designed to maintain, perhaps with a focus upon trojan/worm based traffic. The more false positives one tends to respond to, the less attention, over time, administrators tend to pay to such systems' cries of alarm. Many of us like to layer a number of such tools and systems inside, as we do with our systems at the perimeters (routers and firewalls) for the reason that one system might miss, or false upon, something another will identify differently, as well as to compare what all are seeing on the network wire, in determining policy adherence and network/systems security being properly maintained. In other words, it allows for comparative analysis; the weeding out of false alarms and the finer tuning of the systems being used. It's never a good idea to place all ones eggs in one simple basket after all. Additionally, the detection of legitimate traffic helps function as an indicator that wanted and needed applications and tools are up and running, doing the jobs they are designed for. Further aiding administrators in watching for outages and other problems that can arise in complex network architectures. Further tuning of the filtering process, and the parsing of those results turns these tools into regular traffic monitors that can more proactivily identify potential areas of concern often before a full blown denial of service results. Recent additional supports for our ideas presented above: SANS NewsBites Vol. 3 Num. 51 19 Dec 2001 --14 December 2001 Intrusion Detection Swamps Users With False Alarms IDS vendors concede that false alarms and redundant alerts are a serious problem. Adding to the problem is the fact that companies buy IDSs but fail to provide adequately trained personnel to monitor the results. Special thanks to Jose Nazario and Fernando Montenegro for their help in critiquing and analyzing the ideas presented here.