Recap: Spring 2014 Product Improvements

Now that summer is here, let’s recap the Recorded Future product improvements we rolled out over the last three months. With our SaaS approach, we deliver improvements in weekly increments, along with support documentation and examples. Periodically, it’s good to take a step back from the “trees” and refocus on the bigger picture of the “forest.”

Sources and Data

The data work of mapping new public sources literally never stops. Here are two notable milestones from Spring 2014.

Cumulatively, we’ve now identified over 500,000 public sources. A few prolific sources publish millions of new event reports on a busy day, although most are much quieter. Here’s a view of the breadth of top sources reporting on HeartBleed this spring.

Click image for larger view

Having mapped sources, the next step is to harvest event reports and match this information against “ontologies” of the things people write about on the web. This spring our Malware Signature ontology, a more recent IT security addition, reached the “million plus” scale of some of our stalwarts like Companies, Organizations, Technologies, and Geolocations.

General Purpose Analysis of Events

Fundamentally, Recorded Future is a system for capturing event reports from the public web – the past, current, and anticipated events that people around the world are concerned about today – and making that information continually useful for analysis. Most product features are designed to support open source intelligence (OSINT) analysis for many different intelligence questions.

Often, the analyst’s attention is triggered by an alert. Analysts can now receive alerts for newly report events only in highly-discussed topics. Analyst teams have new tools for shared alert management and multiple alert subscribers so every analyst can benefit from the team’s best alert configurations.

Search is essential for interactive investigation and discovery. We delivered several usability tweaks, and we overhauled support for mixing-and-matching entities and text keywords in the search builder. Experienced analysts will appreciate how a point improvement to search toolset can have huge leverage for precise searches and alert rules.

Data visualizations got major attention this spring. Analysts can inspect a dense Timeline with more precision. The Network has new features for “hairball” avoidance: focused view presets, and incremental reveal of additional node types. New features in the Table support analysis of social media by author name, reporting location, hashtags, and profile attributes. We revamped the “pivot questions” offered as suggested navigations in all views; the three views noted above, plus the Geomap,Feed, and Treemap.

Reporting features also got attention. We added a new Dossier feature, which lets an analyst collect key event reports from any step in their investigation, for further analysis and reporting. For flash reports, the Table view now offers a print style to grab the key events using copy/paste. For in depth reports, we overhauled the layout support in the Network. The analyst can pause real-time rendering, zoom in, repel or drag-and-drop a dense area, and hide irrelevant entities and types to enhance their work product with a tailored network visualization.

Cyber-Specific Analysis

Let’s wrap up by highlighting improvements focused on the IT security problem set.

In alerting, we added control over email subject lines for Cyber Alerts, to catch up with general-purpose Event Alerts. Cyber Alerts then moved ahead with a visual layout and content redesign. Similar improvements for general purpose alerts are coming soon!

Our Dashboards visualize the latest Cyber Attack event signals, and how these signals interrelate attackers, methods, targets, and operations. For monitoring, we added highlighting for watchlist entities with no current signal. Here also we overhauled the “pivot questions” offered as suggested navigation paths for each threat signal entity.

Several improvements targeted drill down into more detailed analysis of IT security events. We renovated our ontologies, breaking out Vulnerability and Malware Signature entity types. We added view presets for general-purpose visualizations like the Network, to shift these views into a cyber-focused configuration. A new search option for any Information Security-related events makes it simpler for threat intelligence analysts to run a broader search that includes reporting on vulnerabilities, malware and threat actors, not specifically cyber attacks and incidents.

The timeline below shows that broader recall, again using Heartbleed as the topic.

Click image for larger view

Last, for reporting we added a specific export feature for telltales and IOCs – essential details for an IT security threat report.