Buying Guide: Big Data Apps for Security

Companies are continually searching for new ways to get more value out of their data. Several months ago we published an article on how Big Data can be used to improve security, looking primarily at what the company KEYW was doing in this arena.

KEYW is far from the only company looking to capitalize on growing demand for Big Data-driven security solutions. Vendors of all types and sizes are getting into this game.

Here are four examples of what is currently available. Such products are offered by companies ranging from IBM, which has been around since 1911, to Fortscale, which opened its doors a century later.


The LogRhythm Security Intelligence Platform combines security information and event monitoring (SIEM), log management, file integrity monitoring and machine analytics with host and network forensics in a unified security intelligence platform.

“LogRhythm is designed to help prevent breaches before they happen by accurately detecting an extensive range of early indicators of compromise, enabling rapid response and mitigation,” says Seth Goldhammer, director of product management at LogRhythm.

LogRhythm’s AI Engine performs multiple analytical techniques across all incoming data in real-time, including pattern recognition, advanced correlation and generating behavioral and statistical baselines with anomaly detection. Analytical techniques can be linked together to corroborate events identifying the highest priority security events and compliance violations.

LogRhythm’s Network Monitor, released in September, provides independent collection of network session details needed for detecting and responding to threats, identifies more than 2,000 applications with layer7 deep packet inspection, provides access to rich session metadata and performs full packet capture for deeper analysis.

“While we still need firewalls, intrusion detection systems, endpoint security and other point specialized security products to block the drive-by attacks against known exploits, we need additional technology to recognize when someone has slipped by these defenses using unreported vulnerabilities or other methods,” says Goldhammer. “LogRhythm’s ability to combine Big Data analysis with real-time machine analytics provides an organization the right set of tools to truly recognize these indicators of threats and breaches.”


Fortscale 1.0 is comprised of two major layers: a Hadoop cluster that can be integrated with various Big Data repositories or SIEM systems; and a toolbox for cyber analysts. The Hadoop layer includes generic, nested or canned machine learning algorithms that profile behavior of users and entities across multiple log sources without the use of pre-defined rules, heuristics or thresholds. These dynamic algorithms produce a risk score indicating the potential risk of a given entity, based on automatic behavior analysis and peer analysis. The Analyst Interface layer includes visual reports, dashboards, unique query language and visual investigation of threats and attacks.  

“Fortscale’s goal is to enable enterprises to easily run Big Data analytics for cyber security, regardless of their technical know-how in machine learning or in advanced cyber warfare,” says Idan Tendler, CEO and co-founder of the company. “These analytics assist in exposing compromised and rogue users and investigating access of users to classified information.”

Fortscale uses machine learning techniques to identify suspicious behavior entities or discover new patterns, with no pre-defined rules, heuristics, signatures or thresholds. These algorithms are fine-tuned to the relevant security context.

”Advanced attacks usually leverage compromised user accounts and identities inside the enterprise,” says Tendler. “Our Big Data analytics profiles user behavior or access to sensitive information and automatically discovers abnormal behavior that may indicate a possible breach or even insider threats.”


EMC’s RSA has several products that leverage Big Data analytics in the security domain. These include RSA Silver Tail, which focuses on detecting Web-based fraud; RSA Vulnerability Risk Manager for managing technical vulnerabilities; and RSA Security Analytics for detection and investigation of security threats in an enterprise.

“RSA Security Analytics is a security solution that helps security analysts detect and investigate threats that are often missed by other security tools,” says Matthew Gardiner, senior manager, RSA. “By combining Big Data security data collection, management and analytic capabilities with full network and log-based visibility and automated threat intelligence enrichment and metadata creation, security analysts can better detect, investigate and understand threats that they could often not easily see or understand before.”

RSA Security Analytics uses centralized full network packet capture and session recreation, logs and events, together with threat intelligence and a Big Data analytics platform. It can ingest business and technical information to give the incident responder (security analyst) more context as to the assets involved in a given investigation to help prioritize and guide the investigation.

“The faster and more effectively a security issue can be detected and investigated, the more quickly and effectively the organization can execute its response, and the greater the probability that the security incident won’t lead to damage or loss of IP,” says Gardiner. “Big Data analytic techniques are key to providing this level of security anomaly detection and fast investigations.”


IBM Security Intelligence with Big Data is a purpose-built security intelligence solution which leverages advanced analytics to derive security insights. It is based on two components: the QRadar Security Intelligence Platform and the Infosphere BigInsights Hadoop-based solution for managing and analyzing massive volumes of structured and unstructured data.

“The traditional approach to cybersecurity placed significant reliance on the knowledge of an attack so that detection techniques and appropriate countermeasures can be deployed,” says Vijay Dheap, global product manager IBM Big Data Security Intelligence & Mobile Security. “This not only left organizations vulnerable until they gained awareness of an attack but also is completely blind to targeted attacks that are specific to the organization.”

Big Data solutions address this by providing greater intelligence on attack patterns, including advanced persistent threats (APTs), cyberattacks, fraud, hacktivism and insider threats. With these tools, administrators can now perform security analysis over much longer durations of time and can leverage non-traditional data sources as inputs for security analysis. The IBM solution standardizes the data flows from real-time security intelligence solution for offline analysis on an enterprise-ready Hadoop environment. It also provides a feedback loop back to the real-time security intelligence solution from the Hadoop environment, which enables automated monitoring of results of advanced analytics to filter out false positives.

“We are unique in that we not only can use Big Data to broaden visibility, but also use Big Data to help customers narrow down the data to the incidents and offenses they need to address,” Dheap says.

More Information

Once you get the right software in place, how do you use it to derive maximum value from it? Find out more with these Nine Tips on Using Big Data to Improve Security. And for even more information, check out this filled with lots of advice on evaluating and using Big Data analytics to improve security.

Drew Robb is a freelance writer specializing in technology and engineering. Currently living in California, he is originally from Scotland, where he received a degree in geology and geography from the University of Strathclyde. He is the author of Server Disk Management in a Windows Environment (CRC Press).

Drew Robb
Drew Robb has been a full-time professional writer and editor for more than twenty years. He currently works freelance for a number of IT publications, including ServerWatch and CIO Insight. He is also the editor-in-chief of an international engineering magazine.

Top Products

Related articles