Early this year news broke of a clever but unnerving security breach at a “critical infrastructure company” in the United States. As originally detailed by the security firm who untangled the ruse, a U.S.-based software developer had outsourced his own coding responsibilities to a subcontractor in China.

Although paying the Chinese worker only a small fraction of his own substantial salary, the U.S. worker apparently spent most of his work hours simply fooling around online. The remote worker in China accessed the U.S. company’s VPN using credentials provided by the U.S. employee.

The enterprisingly lazy employee’s scheme went without notice for two years before it finally came to light. How did he get away with it for so long? It certainly helped that the Chinese subcontractor apparently did great work. The quality of code delivered raised no suspicions.


Beyond that, though, the two-year run can be explained by an even more banal fact: That is how long it took for the company to decide to monitor its log files.

The VPN log files had been recording the connections from China all along. But that’s the thing about log files – they aren’t much use unless you actually look at them.

As described in the Verizon Data Breach Investigations Report (DBIR) only 8 percent of compromises were discovered by log file review. Thus most log files sit around unloved while the key security holes they may reveal are hiding in plain sight.

Lonely Log Files

All major platforms can thoroughly log all facets of activity, from local application use to connections to and from local and external networks. In fact, most platforms generate so many logs, with so much data, that maintaining and archiving log files has become a specialization in its own right.

The sheer abundance of logged information actually tends to produce the opposite of the desired results because enterprises are tempted to ignore them. Because he information overload can seem overwhelming, log files accumulate silently, are automatically compressed and archived, and hardly ever pass by anyone’s eyes.

An all-too-common attitude about log files is that they are for post-game analysis. That is, you consult the log files after a problem is discovered. Maybe a hard drive failed. Maybe an application stopped working. Maybe a website was defaced.

In all these cases, the damage is already done. Using log files for investigative forensics is great, but organizations need to embrace a more pro-active attitude toward log files – to catch problems before or while they are happening.

Proactive Log File Analysis

Obviously it is not realistic to dedicate IT staff to literally read log files on a continual basis. This is the market for log analysis software. These applications review logs for known red flags, either specific error messages and warnings or patterns of behavior (known as “signatures”) which can signal malicious activity.

In the realm of security, intrusion detection systems describe software that continually monitors relevant log files and produces alerts when suspect conditions are discovered. IDS suites are typically broken down into two categories:

Host-based IDS: This type of software runs on one or more servers and/or workstations and continually analyzes operating system log files. When pre-determined signatures are detected, alerts are generated for further expert analysis.

Network-based IDS: This software runs at the outer edge of the network, directly analyzing traffic in real time before it reaches destination hosts in the local network. Rather than analyze logs, per se, Network IDS systems can generate their own logs and, of course, produce alerts when certain triggers are tripped.

In the case of the stealth outsourcing, the Chinese subcontractor was connecting to the corporate VPN at regular and routine times every day, just like a “real” employee. Both his origin and schedule would have produced a suspect signature and subsequent alert by either a host or network-based IDS, had the victimized company actually employed either.

Software Support

The open source software engine Snort is perhaps the most widely used IDS. With support for all major platforms, Snort applies a large set of intelligent and customizable rules to analyzing network traffic. But it is important to clarify that although Snort produces invaluable log files, the software itself is not a log analyzer. Many third-party products exist, both free and commercial, for analyzing Snort log files. For many enterprises, the combination of Snort plus a Snort log analysis package equals a powerful intrusion detection and protection system.

Enterprises looking for more turnkey solutions have a glut of options such as Sawmill, Splunk, and Sumo Logic.

Although many organizations greatly underutilize the value of log files, there is a silver lining. Much like physical exercise, doing anything is a big step up from doing nothing at all. Likewise, a pro-active approach will help keep your network defenses strong before failures occur.

Aaron Weiss is a technology writer and frequent contributor to eSecurity Planet and Wi-Fi Planet.