  How to make Intrusion Detection Systems work better Every year in every large company the golden question is asked: “How much are we going to lose this year because of security failures?” The CITSO responds with:“Well, we need to find out what the likelihood is of things happening to us and we then need to check how much it will cost when those things happen to us. Once we have those figures, we can decide how much we have to spend and how much that spending will reduce the cost of these bad things happening to us” So the first question is, how do we find out what the likelihood is of bad things happening to us?
The most effective way to do this seem to be by measuring the daily activity on the network and seeing not only how many bad things happen on the networks, but also what their impact is on the organisation. This behaviour is then extrapolated and by correlating it with extraordinary global events (such as terrorist bombings), the “acidity” of the network can be predicted quite accurately. So herein lies the problem, how to identify bad things. Intrusion Detection Systems have specifically been designed to detect bad traffic.
In some cases, this means detecting traffic that contains data which is clearly identified as malicious, but in other cases, it also detects network behaviour that is out of the ordinary. But IDS's suffer from two problems, false positives when malicious behaviour is misdiagnosed and false negatives, when real attacks are missed. Examples of false positives are when signatures are triggered by themselves and the IDS then alerts on its own signature downloads while false negatives would be when an attacker substitute the NOOP sled (string of (sometimes) meaningless binary used to pad a buffer overflow until the function return value is located and overwritten) with binary data that is not detected by the IDS and the attack can therefore penetrate the environment undetected. The real problem with IDS is actually that it is normally focused on assuming that the volume of alerts is important and that a large number of a certain alerts is more dangerous than a small number of another alerts. This design feature, combined with the fact that IDS events are context free in the bigger scheme of things (they stand by themselves and are not correlated with other events) result in false positives and irrelevant positives (a dangerous payload that went to machine not sensitive to it). So, a whole new generation of Security Information Systems have been developed to collect data from all over your network and then correlate that data with your IDS data, thereby reducing false positives.
The broad idea is great, but it seems that most of these products will eventually be superseded by the capabilities of the IDS/IPS systems themselves for the following reasons. 1. All investigations start with the identification of anomalies and uses the rest of the information to qualify that anomaly, which means the IDS system is key to the whole design. 2. IDS systems are designed to deal with events and router logs, application logs and network banner grabbing produces events of a different kind that are still just events.
3. Large gains can be made by doing correlation of IDS events with themselves. Examples of this would be classes of alerts such as “attempted recon” that are then followed by “application exploit” are much more indicative of nefarious activity and raises the priority on clustered alerts that indicate a constructive effort to compromise a system. What is also interesting is that by collecting application logs and system logs in your IDS for correlation, you now have the ability to alert on error conditions which might have been brought about by a security event or not, but is still something that has to be dealt with since it is costing you money and security systems are there after all to save you money!
False negatives are difficult to detect since they were missed by the IDS in the first place, but one can assume that application logs will provide more detail in the case of attempted or successful exploits, thereby exposing false negatives. The bottom line is that visibility of network status is key to understanding risk and by correlating IDS alerts with application logs and other network information systems, false positives can be reduced thereby providing better information. 
