There was a time when IT security was a very manual process, with humans required at nearly every stage. Those days are gone, as IT moves toward increased automation and Big Data analytics for security.
In a video interview with eSecurity Planet Symantec Chief Technology Officer Steve Trilling detailed his company's efforts in the Big Data space and the role of automation.
"In our world of security it has always been fundamentally about how do we take a process that is being done by humans and automate it," Trilling said.
Better Security Built by Humans
Most of Symantec's threat signatures that used to be handled by human beings are now almost all generated by automated systems, he said. Yet humans still have a place and a key role to fulfill, beyond just building automation engines for security.
"The goal of humans is to understand the threat landscape well enough to build the right automation," Trilling said.
Taking information from disparate sources of data and turning it into actionable intelligence is also an activity where humans can add value. "Our cycle has been to use human beings to get a depth of understanding about the threat landscape," Trilling explained. "And then we try and to find enough characteristics that are common to those attacks, so we can start to build an automated system and process them."
Big Data's Role in Security
Trilling noted that now attacks are so complex and coming from so many different places, that the challenge is about finding the needle in the massive haystack of data. Enterprises need to be able to figure out what the malware is, where it came from, how it got in and where it might still be on the network.
Big Data as an industry term is relatively new. Symantec's efforts to correlate and make use of large datasets predates the term Big Data. Trilling said Symantec's Big Data platform has evolved over the last three to five years.
The biggest change in recent years has been the amount of data that Symantec is collecting. The more data Symantec can inject, the great the fidelity of the security analysis.
"It's all about the fidelity and the accuracy that you can get, and that is ultimately driven by the datasets that you bring in," Trilling said.
From a technology perspective, the open source Hadoop project is often directly related to Big Data, but that's not necessarily the case for Symantec. Trilling noted that Symantec is not publicly disclosing what the company is using as its core Big Data back-end. That said, he indicated the system is something that Symantec built on its own from scratch and has integrated with some third-party components.
Hadoop is often operated in a batch processing mode with MapReduce queries. The Symantec approach is significantly more real time in its nature, he said.
"We have safety ratings for 3.6 billion files on the Internet today on over a hundred million websites, and we are constantly re-computing those because they may change over time," Trilling said. "We're doing that in real time with our home-grown system."
Symantec's Security Challenges
Symantec's CEO faces a number of key challenges. One of the big ones is the simple fact that IT security has an adversary.
"It's not just about building software, shipping it and putting it out there," Trilling said. "We have somebody that is every day trying to figure out how to get around us."
Watch the full video interview with Symantec CTO Steve Trilling below: