By Sriram Ramachandran, Niara

The impact of a data breach has more significant consequences than ever before, costing organizations between $400 to $500 billion a year, partly due to the expansion of the Internet of Things and the evolving creativity of cyber criminals in today's complex threat landscape.

Being a victim of a data breach no longer results in a slap on the wrist. Instead it can lead to costly fines, job loss, physical damage and an organization's massive loss of reputation. Case in point: Target. Following its high-profile breach in late 2013, Target suffered large losses in market valuation and paid more than $100 million in damages.

As cybercriminals evolve their skills and tactics to perpetrate their crimes, and as sophisticated malware is increasingly able to bypass detection, the prevailing practice of "detect and prevent" to safeguard everything before it enters your network is no longer sufficient.

Threat Detection: Get More Proactive

It's important to recognize that the threats are already on the inside of the network, and therefore enterprises must adopt a more proactive "monitoring and response" strategy where successful attacks are detected and effectively investigated. Enterprises need to be able to pinpoint which aspects of their networks have been affected, and assess the impact to their networks.

So how does an organization or enterprise accomplish this? Through behavioral analytics, which uses machine learning techniques to detect anomalous behaviors for users, hosts and applications without requiring rules, signatures or configuration.

That said, this feat is easier said than done. The information needed to identify anomalies, even if available in a single repository, is contextually siloed and requires analysts to spend inordinate amounts of time to distill necessary intelligence out of the data. Machine learning-driven analytics may be needed to analyze the vast amounts of structured and unstructured data across the network and security infrastructure, to ferret out interesting anomalies associated with the entity (user or host).

Due to complexities related to storing huge amounts of data over time, most organizations aren't able to retain records for nearly long enough. This can make it impossible for analysts to perform forensic analysis on historical data to determine the indicators of compromise associated with a given attack. Further, with modern IT systems producing huge volumes of data with relevant security information embedded within, manually vetting that data is not practical. This is especially true given the shortage of data scientists and experienced cybersecurity talent in today's IT space.

So where does an organization begin? If possible, look to the skills of advanced security professionals who can use intuition, honed through experience, to propose and test hypotheses andhunt for threats.

How Security Analytics Can Help

Next, look to security analytics tools with Big Data architectures to ensure organizations can analyze the mountains of data emanating from their networks to provide insights into advanced attacks without rules or signatures. With the right platform, credible alerts can be raised for immediate attention as anomalies are detected and linked to malicious intent.

In addition, analytics-driven visibility can accelerate time-consuming historical incident investigations, going back months or years, and raise the productivity of the security teams.

Recently eSecurity Planet highlighted seven security analytics startups that utilize Big Data, machine learning and other technologies.

Organizations have vast quantities of log, alert, packet, flow, file and threat feed data that can yield intelligence for thwarting cyber criminals. Incorporating machine learning capabilities to behavioral analytics enables organizations to intelligently detect attacks, automatically analyze varied data from multiple perspectives and statefully correlate outliers across data sources over time. As a result, high fidelity anomalies are surfaced without adding to the alert white noise.

Moreover, integrating forensics and analytics allows for immediate access to contextually relevant evidence for the anomalies generated. This makes it efficient for the analyst to go from detection to triage to investigation.

Innovations like data reduction through metadata storage, coupled with an underlying Big Data architecture, enable the retention of essential supporting evidence for significantly longer time periods. As a result, figuring out who and what else may have been affected when you don't know exactly what you are looking for becomes easier, even when looking back in time.

Let's face it, given that the frequency of cyberattacks is at an all-time high, it's likely that your organization has been or will be the target of a hack in the future. Being able to identify an attack as it unfolds enables companies to react faster, limiting the damage a given attack can inflict and mitigating the post-breach impact.

Sriram Ramachandran is CEO and co-founder of Niara, which offers a security analytics platform that automates the detection of attacks that have bypassed an organization's perimeter defenses and reduces the time and skill needed to investigate and respond to security events.