Threat Analyst Insights: Life Without Data

In the context of scoping out research for reports over the years, I’ve occasionally heard the phrase, “I can’t find any data regarding ‘X’.” The person making this statement usually offers it in a spirit of resignation, as if the lack of data signifies a deficiency in their toolset, their methodology, or the research request itself.

My response is always, unironically, “Good to know!” That’s because the lack of data about any event is as analytically meaningful as the presence of data. From personal experience, constant access to a tool as powerful as Recorded Future for finding information makes the job of a cybersecurity analyst much, much easier, but it also sometimes lulls people into thinking that truly successful analysis for a research request has to end with a link to a suspicious domain, a criminal conversation, or an unreported attack.

A better approach to analysis comes from trying to correctly understand whatever data is available, even if the only data point is, “There is no evidence of ‘X’.” Put in a different way, out of the two statements, “I know nothing” and “I know that I have found nothing,” only the first statement shows ignorance. Of course, an analyst needs to be able to research effectively, but in my opinion, that isn’t very useful if the analyst does not know how to frame the results.

How should an analyst move forward if the result of their initial research is, “I have no data”? The following three general methods can be helpful.

1. Change Perspective

Coming up short on information can happen because a researcher looked in the wrong place, didn’t look hard enough in the right place, or (very uncommon) didn’t look at all and called it a day. Regardless of the reason for not finding a lot of data to answer a research question, the knee-jerk response should be to reset and look at the issue from a different angle. Always assume there are other ways to find what you need until you’ve exhausted every option.

A recent report I wrote for our Weekly Threat Landscape product involved a compromise of point-of-sale (POS) software for a restaurant chain. Initial research turned up underground forum posts about laundering money via the chain’s gift cards. This was great contextual information, but it didn’t reveal what software the chain used or how a criminal might have exploited it. Not satisfied with a generic analytic comment about criminal interest in payment card data, and not finding anything quickly about what software the chain used, I switched up my search and dug for evidence of employees or franchise owners who needed help with installing or using the software. That search not only pulled up the software name, but also an old legal document with information about how the software could remotely access POS terminals. This let me write a comment that could more specifically warn about risks associated with POS software and the likely attack vector behind the compromise.

2. Don’t Jump to Conclusions

An early and excellent episode of the TV show “The Boondocks” involves a couple of privileged posers trying to pull a citizen’s arrest on the owner of a gas station because they suspect he is a terrorist (he isn’t). To justify an increasingly ridiculous situation, one of the two keeps repeating, “The absence of evidence is not the evidence of absence!” In context, this is completely misused — in real life, it’s completely accurate. It counters a common, logical fallacy of assuming that something doesn’t exist simply because one can’t find evidence for it.

Good analysts are careful about what a lack of data does and doesn’t mean. For example, Recorded Future’s dark web and underground forum collections are an incomparable resource which I use almost every day, but they shouldn’t be relied on as the ultimate reflection of cybercriminal intent. If I can’t find criminal chatter about a particular company, product, vulnerability, or malware, it doesn’t follow that no malicious actors are interested in that item. Maybe they are interested and are just keeping quiet. This is why it’s important to set up, review, and narrowly tailor alerts so that new data where there used to be none can be caught immediately.

3. Own It

Even after researching as much as possible and confirming no logical inconsistencies, there are often times where all an analyst can say is, “I can find no evidence of ‘X’.” When that is the case, the analyst should present it simply and confidently, and shouldn’t try to shoehorn irrelevant data into the response as if otherwise, their audience will question their intelligence.

Standing by one’s inability to find expected data is also just part of being honest, and in the long run, this pays off. In a previous life, my team received a request from a company to look into a video online that appeared to contain a threat of violence against them. The initial assessment of the video fit with significant threats of the time, but we couldn’t find any data to support a connection. Further digging (see the first method above) convinced us that the video was a gag and didn’t warrant further concern, which we told them in a short response. About a week later, the company thanked us not just for the analysis, but also for not abusing an opportunity to hype up the need for cybersecurity research in a dangerous world. We found out that at least one other vendor wrote a report about the rise of violent threats online and how to mitigate them without any information proving how the video fit with this analysis.

Conclusion

Analysts should not have to artificially fill reports when they can’t locate certain kinds of data. On the other hand, analysts should also not have to manually run down every avenue of investigation to confirm whether or not relevant data exists — that’s an unsustainable and outdated process. The power of a platform like Recorded Future is that even a statement like “I have no evidence of ‘X’” can be based on a data collection capability that is vast and constantly improving.