Extrusion Detection Systems; the art of network monitoring
     				by:  Ron DuFresne
     				      (c) 2001
     I've watched over a number of years, as tools have developed to guide
     administrators in determining how, why, and from where their systems are
     attacked.  Applications such as TCPD (TCP wrappers) was one of the first
     tools to help in limiting access to protocols and applications that have
     very insecure control means and are often used to compromise systems.
     Soon after came along some of the early firewalls like the now famous
     Firewall Toolkit.  Since that time, many flavors and versions of
     firewalling products, protocol and application proxies have emerged,
     mostly to protect the perimeters of networks.  To aid in perimeter
     protection many scanners have been developed, at first merely to
     demonstrate the insecurities and exposure systems and networks pose to
     companies and individuals alike.  Not too many years ago a new set of
     buzzwords started to crop up, Intrusion Detection Systems (IDS's), that 
     helped administrators monitor and watchout for intruders probing and
     prodding systems for exposure and weakness.  And to this day the most
     popular, yet, probably most ill conceived placement of these systems is
     on the outside of the firewall or directly upon the same system.  I've
     long been a proponent here that the better placement of such devices is
     behind the firewall.  For these can be very noisy devices if placed
     outwards and exposed for all the black and grey hats to pound upon, let
     alone the large numbers of improper packets that many routers needing
     some gear replaced, and failing network cards, might strew about as they
     start to falter.  Let's face it, systems and network admins are vastly
     overworked as it is, putting in far too many hours a day/week/month/year.
     We do not need to be woke every night at 2-5AM to deal with false
     positives and alerts that exposed IDS systems are prone to generate.  We
     certainly don't have the time to be running back to look at the IDS and
     various system logs throughout the normal day even, to check into each and
     every stray packet such exposed IDS systems will 'discover'.  Few, if any
     companies will provide the funds to pay for personnel to sit in front
     of a screen and continuously monitor the traffic generated by an IDS.  I
     know there are many that might well argue the point, but, let's face it,
     an exposed IDS system is best suited to those folks conducting the
     honey-pot project and the others that like to peruse tons of logged
     information to study and report about Internet traffic trends.  
     Monitoring the traffic from an exposed IDS can be near to impossible to do
     real time, it makes the eyes water and the brain hurt to say the least.
     I've seen enough postings to the various security lists and Usenet groups
     from very skilled persons asking for help in determining exactly what they
     are seeing in log extracts and network traces from their IDS systems and
     firewalls (these are probably the second most asked questions, the other
     being how to prevent users from violating network policies and block
     various ports and applications from the desktop outwards), to know that
     using these tools to help determine what new attacks are working their way
     into the flood of Internet traffic is very often beyond the means and
     understanding of most experts in the field whose job it is to maintain the
     routers and firewalls at the network edge.  It takes very refined skills
     to know how to recognize new attack methods and modes within, most often
     very congested network streams of traffic.  
     Muting the IDS system to not alert, but log for later analysis, again,
     seems to be more for the realm of those reporting trends, and defeats one
     of the finer aspects of the better IDS systems, near realtime proactivity.
     This really counts when something has bypassed the firewall and is headed
     at the network backbone.
     The argument that an external, exposed IDS is a great tool for
     qualifying security spending to management is, in our mind, a demonstration
     of overkill.  First, this information should already exist in the logs
     of the perimeter devices maintaining the companies network security
     policies.  Secondly, if the perimeter devices are not logging traffic
     attempts and flows, with matters as they are on the Internet, a laptop
     with a packet sniffer running for a short period would certainly suffice
     in demonstrating the need for security spending.  Better justification
     comes from demonstrating the costs of a loss or even an outage of services
     across the company backbone.  If specific tools need justification, one has
     ample evidence from the security related mailing list, Usenet groups, and
     security related trade journals and security related web sites, let alone
     the GAO reports on our own government's audit processes as well as the
     daily news.  There is a plethora of information available to justify
     expenditures already.
     We are also unaware of any tools yet designed, capable of monitoring for
     new traffic and unknown packets that might emerge and be injected into
     network streams with wicked intent.  Watching for something new and
     unknown is difficult, to say the least.  The anti-virus tools available
     need constant updates to keep on top of the new viri and trojan variants
     constantly being unleashed.  The veracity and nastiness of the newer
     viri and trojans has worked to ever increase the need for detections
     of such virulent code on desktops and servers.  These newbreed
     nasties are now working with applications and network protocols to help
     themselves propagate and infect systems at tremendous rates.  Quite often
     they can bring even large bandwidth networks to their knees merely with
     the overflow of packets they produce to further spread their destructive
     payloads.  Often these sharp spikes in additional, seemingly legitimate, 
     traffic are the only signs that something is amiss on the corporate
     LAN/WAN infrastructure.   Even the various security scanners require
     updates and rewrites to deal with the new attack models discovered time
     and again.  So the arguments for placement of an IDS on the external side
     of the boundry devices weakens under these observations.
     The variety and scope of the various IDS systems is already quite large,
     and, we'll not discuss the various attributes and abilities of those now
     available or those under construction here.  Let it only be stated that we
     prefer IDS systems that silently monitor without injecting further packets
     into the network stream, as in my mind, those that require packet
     injection only serve to further add to already over-congested network
     flows, requiring that one parse out the systems particular packets in
     traffic analysis.
     I've always liked an IDS system to backup the firewall, to stand behind it
     and give notice when something has been able to cross the firewall and
     reaching the network boundary, becoming a potential intrusion to the
     systems behind the devices setup to guard the interior network from
     unwanted and malicious traffic.  I've often described a properly placed
     IDS, monitoring the the internal network edge, as a silent warrior.  A
     system that only wakes and shouts out alerts when traffic slips by the
     border devices, as a last alarm and guard, to warn that something is amiss
     on the perimeter, posturing a potential attack to what still amounts for
     the most part, on most corporate networks as the 'soft chewy center' of
     the corporate LAN/WAN.  The ideal IDS sits silently, and functions as a
     tool that helps admins and network guru's feel warm and fuzzy that their
     router rules and firewalls are doing the job they have been designed to
     do, protect corporate assets.  And yet, I have to admit this is a somewhat
     shortsighted description and use of IDS systems.
     Over the past few years a number of firewall wizards and networking guru's
     have come to understand the value of IDS systems in monitoring what is
     escaping from their networks, and even for monitoring for various
     application and service outages on the internal network structure.  The
     detection of what traffic is departing and traversing  the network can be
     "as", if not "more", important these days, in determining the
     effectiveness of efforts to protect the internal assets and structures
     that system and network administrators are hired to do.  Most often the
     devices, routers and firewalls are very permissive in what they allow
     outbound.  While these systems are extremely restrictive of what they will
     allow inbound, if that traffic originates within the perimeter systems are
     often designed to permit traffic flows back and forth across the network
     boundaries to facilitate the flow of information and work of the employees
     the network supports.  And ever since computing networks have existed the
     concept of PEBKAC (problem exists between keyboard and chair) has been, 
     and remains the biggest issue in the flow and progress of viri and trojans
     and various other forms of network compromise.  This is one of the main
     reasons that many wizards and gurus have taken to installing a properly
     tuned IDS system facing inwards on the network, to set off alarms and
     alerts about desktops and servers that have been compromised in some 
     manner and are spewing traffic not intended to flow out the corporate
     border.  These IDS systems often function as additions to the anti-viri
     and trojan software installed on the network servers and desktops that
     need to be updated to the new nasty code that is released upon unwitting
     users almost daily.  These systems further help the admins and network
     persons maintain the corporate network policies from within as well as
     without the network boundaries.  Just as the external IDS is likened to
     the last line of defense and alert, the internal EDS systems often are
     front guards capable of saving a company from embarrassment and/or
     damaging information leak or internal system compromises.
     The tuning and taming of these internal monitoring "EDS" (Extrusion
     Detection Systems) is very crucial, so that false positives and alarms
     are kept to a minimum, the results should be mostly mirrors of the
     corporate network policies they are designed to maintain, perhaps with a
     focus upon trojan/worm based traffic.  The more false positives one tends
     to respond to, the less attention, over time, administrators tend to pay
     to such systems' cries of alarm.
     Many of us like to layer a number of such tools and systems inside, as we
     do with our systems at the perimeters (routers and firewalls) for the
     reason that one system might miss, or false upon, something another will
     identify differently, as well as to compare what all are seeing on the
     network wire, in determining policy adherence and network/systems security
     being properly maintained.  In other words, it allows for comparative
     analysis; the weeding out of false alarms and the finer tuning of the
     systems being used. It's never a good idea to place all ones eggs in one
     simple basket after all.
     Additionally, the detection of legitimate traffic helps function as an
     indicator that wanted and needed applications and tools are up and
     running, doing the jobs they are designed for.   Further aiding
     administrators in watching for outages and other problems that can arise in
     complex network architectures.  Further tuning of the filtering process,
     and the parsing of those results turns these tools into regular traffic
     monitors that can more proactivily identify potential areas of concern
     often before a full blown denial of service results.  

	Recent additional supports for our ideas presented above:

		SANS NewsBites Vol. 3 Num. 51 19 Dec 2001

   --14 December 2001 Intrusion Detection Swamps Users With False Alarms
  IDS vendors concede that false alarms and redundant alerts are a
  serious problem. Adding to the problem is the fact that companies
  buy IDSs but fail to provide adequately trained personnel to monitor
  the results.

Special thanks to Jose Nazario and Fernando Montenegro for their help in
critiquing and analyzing the ideas presented here.

SANS: theregister article

Hosted by: Enter:  sysinfo.com

©copyright 1995-2002 sysinfo.com