The Emergence of Big Data Security Intelligence

January 30, 2013

The Emergence of Big Data Security Intelligence

What’s Driving Big Data Security Intelligence?

There is a new generation of security offerings that can anticipate threats and prevent them in real-time.  This new generation of offerings is emerging because of an increase level of risk.  Traditionally, security systems have worked by collecting and analyzing logs and then setting alerts to trigger action when activity falls outside of a specified pattern.  This approach  no longer satisfies the security needs of many organizations.  There are an increasing number of  newer threats, such as Advanced Persistent Threats (APTs) that are designed to look like normal system traffic and often do not trigger traditional log monitoring systems.  Vendors in the market are beginning to address this emerging requirement.

While the idea of addressing this issue has been around for the last few years we are moving into a new phase of addressing the problem.  For example, last year when I attended the RSA Conference in San Francisco, many vendors touted Big Data solutions for security but few of these companies offered real solutions.  I was intrigued by one vendor that proudly displayed a big banner reading “Big Data Security.”  After a short conversation I realized that the company was simply offering an appliance that could pull logs from nearly any device, but could not make these logs actionable.  This product couldn’t address emerging threats.   If an organization using this product experienced a breach, it would have taken months to sort through the logs to hopefully identify the breach.  While looking at historical logs is useful, organizations expect security offerings to respond in real-time to current events.

This demand for proactive security offerings has spawned a new breed of security offerings based on Advanced Security Intelligence that look beyond machine and log data.   These offerings can collect data from hundreds of independent sources, both within and organization as well as security threats and events across the Web.  The rise of Advanced Security Intelligence platforms is being driven by:

1.  The advent of Big Data. Big Data and the ability to access and process huge amounts of data from thousands of sources and can help organizations overcome security threats more quickly. In addition, Big Data allows organizations to recognize patterns and trends that they didn’t even know to look for. With this type of Big Data analytics , seemingly independent events can now be correlated to determine the source of a problem and allow the company to take preventive actions.  In the field of security, Big Data allows organizations to aggregate everything from logs, to tweets, known malicious IP addresses, emails, information on other cyber attacks, and third party research, just to name a few and feed this information into a Big Data platform.

2.  The rise of Advanced Persistent Threats (APTs) and other highly sophisticated and targeted threats.  Cyber attacks are increasingly being perpetrated by criminal rings who are motivated by profit, foreign governments, corporate espionage and “hacktivists” driven by political or social motives.  These groups don’t just seek the low hanging fruit and instead focus their efforts on specific companies and often specific individuals within a company.  For example, they might use social engineering for pre-texting or to launch a phishing attack.

3.  Advanced analytics and predictive analytics have become operationalized.  These technologies are no longer used for looking back over quarterly results and trends, but can be fed streaming data and give real-time recommendations.  Security vendors have taken note of this trend and have scooped up a number of security companies that used analytics for security.

In the second part of this blog, I will provide some information about how vendors are responding to this market opportunity with new innovations and new acquisitions.  The landscape of security services is changing dramatically, which will help customers be safer and more secure.

Dan Kirsch , , , , , , ,
About Dan Kirsch

Dan’s research focus is on how compliance, governance, security and privacy are impacting the software industry and customer requirements. Additionally Dan is looking at mobile market.

Leave a Reply

Your email address will not be published.