Traditional security techniques are no longer enough. By the time malware is spotted and added to antivirus (AV) signatures, the bad guys have either gotten in and wreaked havoc or buried themselves to gather data in order to stage a larger and more targeted attack. 

According to the Verizon 2015 Data Breach Investigations Report, 70 respondents reported 80,000 security incidents which led to more than 2,000 serious breaches in one year. Phishing is increasingly employed to gain discreet access, Verizon noted, and then malware takes its time learning passwords, account numbers and security defenses before an attack is staged.

"It is abundantly clear that traditional security solutions are increasingly ineffectual and that vendor assurances are often empty promises," said Charles King, an analyst at Pund-IT. "Passive security practices like setting and maintaining defensive security perimeters simply don’t work against highly aggressive and adaptable threat sources, including criminal organizations and rogue states."

As a result, a new field of IT security is emerging which is variously labeled as advanced threat detection, network security sandboxing or threat analysis and protection.

What Is Advanced Threat Detection?

As these tools take slightly different approaches, their definitions vary. They employ different combinations of technologies such as sandboxing, behavioral analysis, file integrity monitoring, telemetric heuristics, containerization, NetFlow analysis and threat intelligence in order to detect and block a malware attack or compromise. They look for subtle system process changes that may indicate hostile activity. Some are simply detect and alert, while others prevent damage by containing it within a sandbox.

Whatever you call it, Frost & Sullivan said the advanced threat detection space was a half billion dollar market in 2014 and will reach $3.5 billion by 2019. An average compound annual growth rate of 45.6 percent through 2019 makes it one of the hottest areas in all of IT.

Players in this space include market leader FireEye, as well as AhnLab, Fidelis, Fortinet, Cisco, Trend Micro, Damballa, ThreatTrack, Lancope and VEEDog.

AV products look at the ones and zeros of the files entering a network or a computer looking for certain patterns (signatures), explained Bruce Wiseman, vice president of Public Relations and Marketing at Veedog. In contrast the new wave of cyber security products set up a virtual machine (VM) replica of a network in order to open up suspect files and monitor them for malicious behavior. The focus is no longer on the malware or even the attack vector; instead these tools hunt for threats that have already bypassed traditional defenses.

"A network security sandbox is an analysis environment (often virtualized) in which a suspicious program is executed and the behavior of the program is observed, noted, and analyzed in an automated manner," said Frank Dickson, an analyst at Frost and Sullivan. "This approach is more effective than just looking at the appearance of the executable, because sandboxing goes beyond just the mere appearance of the binary and observes what the binary does; therefore it is much more conclusive in determining if an executable is malicious."

Short List of Advanced Threat Detection Solutions

FireEye provides a variety of network, endpoint and cloud-based security products. Its MVX virtual machine detection engine, which provides real-time threat protection to many large enterprises and governments, is at the heart of its solutions. The detection engine is complemented by threat intelligence, with the aim of identifying and block cyberattacks as they happen.

Damballa Failsafe is a network security monitoring system that provides evidence of threat-related activity needed to prevent data theft. It discovers criminal operators that have already bypassed perimeter defenses by using predictive behavioral analysis to uncover devices behaving in ways that indicate they are under a threat actor’s control. It doesn’t require prior knowledge of the threat and doesn’t focus on malware or the attack vector.

Its Threat Discovery Center monitors 15 percent of the world’s Internet activity covering more than half a billion devices, and analyzes the network communications of over 100,000 malware samples. This data feeds Damballa's machine learning systems, which produce behavioral models and threat intelligence that are automatically updated.

"Attackers may take time to reveal themselves and when they do, Damballa will expose them and initiate mitigation," said Stephen Newman, CTO of Damballa.

Lancope StealthWatch System analyzes flow data (such as NetFlow, IPFIX and sFlow), proxy records and authentication information to spot attacks such as distributed denial of service (DDoS) and insider threats. Combining continuous lateral monitoring across enterprise networks with user, device and application awareness, it is said to accelerate incident response, improve forensic investigations and reduce enterprise risk.

"Having user authentication information allows the behavioral detection model to identify a user account that is a threat versus just the IP address," said Tim Keanini, CTO of Lancope. "Behavioral security analytics are applied to look for changes in host behavior that could be indicative of an advanced attack, malware or an insider threat."  

Fidelis Cybersecurity traditionally focused on network-based threat monitoring but has now added endpoint visibility using agents to monitor and record system activity. Its Deep Session Inspection technology oversees application behavior within the network to see threats that would otherwise be invisible. It combines the ability to recognize malicious activity and content with being able to reconstruct what attackers may have done in the past, to prevent sensitive data or endpoints from being compromised.

"Both network and endpoint products are built to integrate with third-party security tools (e.g., SIEMs, next-generation firewalls or monitoring systems) so they fit seamlessly into the customer's environment and correlate activity across all of the customer's security investments," said Pete Lindstrom, an analyst at IDC.

Unlike the others, which tend to gravitate toward the large enterprise security space, VEEDog is purpose-designed for the SMB market. It monitors the network data flow, flags suspicious files and analyzes them for destructive or malicious intention, disables any file verified to be malware, and packages the file for submission as malware to the customer’s AV provider so they can write a recognition file and distribute to all their customers to neutralize future infection. It also reports to the network administrator the protection being carried out.

"We deliberately aim our virtual machine-based security technology at the small business market in terms of price point and ease of use," said Wiseman.