Raising Your Own APT: Purple Team Exercises to Drive Security Program Maturity

As President George Washington wrote in 1799, “...offensive operations, oftentimes, is the surest, if not the only … Means of defence.” This could not be more true in today’s cyber battleground as organizations work to defend themselves from attackers they cannot see, with tactics they may not be aware of, and with motives that are not favorable.

Security intelligence and defensive measures seek to narrow the playing field and prepare defenders for a real attack. However, without testing your capabilities, responding when it actually hits the fan is cumbersome. Enter purple team exercises.

About Purple Teaming

Purple teaming allows your organization to run scenarios pitting your blue team (defenders) against a red team (penetration testers or pen test software) to identify breakdowns in detective and preventive controls, processes during incidents, and procedures. Pen testing, of course, is nothing new to information security teams, but the potential for conducting pen tests in conjunction with a smart, focused intelligence-driven defense will yield far more information about how ready your organization is.

MITRE ATT&CK has become a buzzword of sorts, but positioning intelligence at the heart of asset- and organization-focused risk management approaches can help drive a proactive security program — and center its mission and results. Pairing MITRE with streamlined security intelligence workflows can push the needle forward or provide a blueprint for organizations to drive toward. Being able to focus not on just threat actors and their TTPs, but also trends, relevance, and the context intelligence provides can dramatically increase value from these exercises.

Organizations that understand their assets, know which threat actors are relevant to their business and industry, and have identified which TTPs are mitigated by their controls, are in a unique position to test their skills and technology in a fire-drill environment. Recorded Future makes it much easier for you to identify attack vectors and exposure points by providing the data you need to support your controls and mitigate risk.

Build out your program and test their connections, both for data and people.

Being the Bad Guy: Designing Your Exercise

In order to conduct a purple team exercise, you must push to identify what you are ultimately trying to assess. A strong starting point is to review your priority intelligence requirements (PIRs) and determine how you validate those currently. Additionally, keep the scope straightforward. Start by answering these questions:

• Is there a gap or something critical you missed in a previous audit, assessment, or model?
• What are the goals for the exercise? (Are you trying to validate that controls actually work? Are you testing your IR team’s response capabilities and time? Are your assumptions of understood risks true?)
• Finally, the major question: What are you testing (people, process, technology, or everything at once)?

Once you identify the scope of the exercise, you can determine the type of assessment and attributes to test and evaluate. The next question should be, “What type of assessment will support your hypothesis?” For those familiar with secure development and testing, the following concepts will seem familiar. You could try one of the following or a mix:

• Black Box Testing: Pen testers have no knowledge of the application being evaluated
• White Box Testing: Pen testers have full knowledge of the application being evaluated
• Targeted Testing: Third-party or internal pen testers have knowledge about the organization and scope to simulate certain attack types or scenarios
• Double Blind Testing: Neither the red nor the blue team have knowledge about the organization to test event identification and response times.

Each of the above can be coordinated to review organizational readiness, controls validation, and application or system hardening.

Identifying Relevant Threats With Security Intelligence

A key part of the exercise design process is to review your threat models and identify who might attack your organization or industry. Again, these are typically defined by your models and validated through your PIRs. Make sure to include not just external threats, but also internal threats to get a full view of defensive capabilities.

Recorded Future can help you identify relevant threats through research in the UI and by utilizing your “methods” and “attackers” watch lists. You can also use Recorded Future Intelligence Cards to identify and research the context of relevant MITRE tags and TTPs that your organization has identified as threats to operations.

Focusing on remote code execution on Windows Machines vulnerable to BlueKeep could help detect gaps in controls coverage. (Source: Recorded Future)

Reviewing Available Security Controls: What’s in the Rack?

Once the threats and TTPs to evaluate are identified, the next step is to determine what controls are available to you internally. Another way of asking this is, “Which people, processes, and/or technology (PPT) have been put in place for us to defend with?” Identifying your gaps and control maturity ahead of time will go a long way to refine hypotheses or test and follow up tasks after the actual exercise.

‘Threats Are Great, But What Are We Actually Testing It On?’

Once you have an understanding of your scope, the controls in place, and you have identified what they cover, you can conduct your exercise. The purpose is to search for vulnerabilities or flaws in PPT that can be attributed to the threat actors, TTPs, and motivations you have modeled in the design and risk management processes.

It’s very important to document and report on the outcomes of the test to continue tracking maturity growth over time. Were you successful? If so, how do you grade that? If not, what new exposure areas did you identify?

Prioritizing and Measuring New Risks

No successful test is complete without a follow up of refining priorities (both business and security), as it relates to your organization’s ability to defend itself. Well-defined threat models rely on prioritization to keep the focus of security teams and prevent a frantic “threat du jour” approach.

For each threat and risk, revisit your risk assessment methodologies and evaluations to better define your critical areas of exposure and operation. Such an approach provides the opportunity to craft metrics around controls improvement, process improvement, and maturity growth. A few examples could be:

• Mean time to respond
• Mean time to escalate
• Control efficacy (blocks or allows)
• Downtime allowed versus downtime experienced
• IAM and access control efficacy evaluations

Reaffirm Investment in Your Controls

Ultimately, your organization will not have achieved anything out of scheduled purple team exercises and intelligence driven pen tests if countermeasures to reduce the risks observed during these exercises are not refined. There are several examples of countermeasures and adjustments, including:

• Better logging of endpoint and network data
• Applying threat data via API to relevant security controls
• Improving correlation detections
• Supplying block-grade data to host and network security controls or applications to stop threats before they become incidents
• Workflow improvement to reduce response times

Whatever the outcome, always think about how you will use the results of the exercise to improve your detective and preventive controls.

Building Upwards: Adding Another Pillar to the Foundation of Security Teams

Monitoring alerts and vulnerabilities for the sake of monitoring doesn’t do anyone any good — neither does conducting CTI research for tracking threat actor appetites and TTP trends if you dont review controls to battle them. However, neither are valuable if you do not test and validate your organization’s security and risk mitigation controls.

By combining reactive and proactive investigation and intelligence techniques with strategic risk management frameworks, you can evaluate program efficacy and improvement across strategic and operational lines. Being able to evaluate those results in a qualitative and/or quantitative way further allows you to respond in an agile and adaptive fashion — improving consistency while being able to respond to changes in threat landscape quickly and efficiently.

When a new year begins, many people set resolutions and claim the ever-cliche "new year, new me." Whether your objective is to start developing an intelligence-led security program from scratch or you are trying to take that next step to improve yours, using security intelligence in conjunction with broader risk management and controls validation efforts can bring everything together.

Using Recorded Future to support your intelligence and security program development can help you actually move toward achieving your resolutions, which is more than can be said for the expensive gym membership your coworker across the aisle bought.

Are you ready to make the shift to a risk-based approach to cybersecurity? Download your free copy of “The Risk Business: What CISOs Need to Know About Risk-Based Cybersecurity” to get started.