Proactive Hunting for Hidden Threats using Metadata

Any company that falls victim to a cyber attack wants to be able to detect it as quickly as possible and to understand what’s going on – fast!  For a comprehensive investigation, however, scrutinizing the historical data is essential.  While some visibility can be achieved through logs and netflow data, this isn’t enough to trace threats back to their source. If a business is to stand any chance of pinpointing who the culprit was, what was compromised and what else is at risk, it needs metadata.  

Simply put, metadata is data that describes other data. For example, if you had a phone conversation you could listen to every word said, but, if you had an easily searchable description of everything that was said you could get almost the same value in a format that was much easier to consume. Metadata provides insights and analytics that can transform the way critical security threats are investigated and understood.

The Data Problem

Increasingly, detection – and especially detecting attackers at each stage of the attack lifecycle – is becoming a data problem.  Traditionally, the only way organizations could capture high-fidelity data about what was happening on their network was to invest in a full packet capture system.  This creates a huge amount of data, however, without efficient methods of sifting through it, requires an army of forensic analysts to make sense of it.  What’s more, not only does this send storage fees into the millions of Euros but full packet capture systems were never intended to facilitate the detection or investigation of advanced threat actors.

Automated Hunting Using Metadata

Metadata can be tapped in place of full packet capture systems.  Metadata contains all the necessary descriptors of the data itself to create an index which can be searched in real time, and at less than twenty percent of the cost of a full packet capture system.

Organizations can therefore search the data at a much faster rate, pinpointing anomalies that could be potential threats.  By creating this early warning system of threats and allowing teams to begin investigations quickly, damage is mitigated and the full implications of an attack can be understood quickly.

For example, metadata can reveal:

• How, why and when the network was compromised
• If a company has been compromised in the past
• Whether the company is part of a multi-vector attack
• What is actually going on in the network?

It is now possible to capture metadata about every document and communication protocol.  For example, information from inside a web session can be gathered, including the source and destination IP. IT teams can also investigate the data in terms of whether a document or executable has been transmitted before, who authored the document and when, as well as information about tags and attachments. This is in addition to being able to scrutinize common attacker tactics, such as SQL injection, web shells, content staging and cross-site scripting, irrespective of whether malware has been used, which differentiates it from other solutions.

Ultimately, collecting more data does not make you more likely to detect a threat. Data has to be indexed and operationalized to add value, or else an enterprise is reliant on a costly forensic analysis team sifting through hoards of data that may or may not indicate a threat. The analysis of metadata allows you to understand the past and present activity on a network, in context, which ultimately helps secure the future.

Please note: The opinions expressed in Industry Insights published by dotmagazine are the author's own and do not reflect the view of the publisher, eco – Association of the Internet Industry.