But what do you specifically want to know? And how are the findings relevant to the threat landscape you face at this moment?
"Decide what you want to know and then choose the best tool for the job."
Security teams typically use several different testing tools to evaluate infrastructure. According to SANS, 69.9% of security teams use vendor-provided testing tools, 60.2% use pen-testing tools, and 59.7% use homegrown tools and scripts.
While vendor-provided tools test a specific security solution—whether it's a web application firewall (WAF), EDR solution, or something else—pen testing is frequently used to verify that controls meet compliance requirements, such as PCI DSS regulations, and by red teams as part of broader testing assessments and exercises.
Automated pen tests help answer the question, "can an attacker get in?" They can help identify vulnerable or high-risk pathways into an environment, but they usually don't cover the entire kill chain. They can emulate multiple threat actor techniques and even different payloads, but they typically don't replicate and fully automate the full Tactics, Techniques, and Procedures (TTPs) of a real threat actor.
Automated pen tests rely on skilled human pen testers with varying levels of expertise, making it difficult to gain consistent data over time. The sheer variety of pen-testing tools and approaches can actually complicate testing. For example, different attack vectors require different testing tools. These tools also tend to be weak at recognizing vulnerabilities in business logic, which can skew results.
For organizations, pen testing is costly and requires significant advance planning, which often limits its use to annual or semi-annual testing. And even with automation, pen-testing takes time to scope, conduct, and analyze, slowing the organization's ability to respond accurately to immediate threats.
The SANS poll found that most respondents test their controls quarterly at best. However, the real-world threat landscape evolves daily, leaving a lot of time for threats to exploit any gaps or weaknesses between scheduled assessments. If you want visibility into the effectiveness of security controls—right now—you'll have additional questions that pen testing cannot easily answer:
- Are your controls working as they are supposed to work, and as you expect?
- Are interdependent controls correctly generating and delivering the right data? For example, are your web gateway, firewall, and behavior-based tools correctly alerting the SIEM when they detect suspicious activity?
- Have configurations drifted over time or been set incorrectly? For instance, are controls actively detecting threats, or were they left in monitoring mode?
- If you have rolled out new technology or settings, how have they affected your security posture?
- Are controls able to defend against the newest threats and variants?
- Does your security defend against the latest stealth techniques, such as living off the land (LOTL) fileless attacks by sophisticated attackers?
- Do you have visibility into security outcomes that require both human processes and technology?
- Is your blue team able to identify and respond effectively to alerts?
Automated Breach and Attack Simulation (BAS) tools enable you to answer these questions. BAS complements point-in-time testing to continually challenge, measure, and optimize the effectiveness of security controls. BAS is automated, allowing you to test as needed, and the best solutions assess controls based on the latest malware strains and threat actor TTPs—without having to assemble teams of security experts. Organizations are using BAS to:
- Simulate attacks without jeopardizing production environments
- Simulate attacks across the full kill chain against all threats, including the latest attacker TTPs
- Test continuously with the flexibility to target specific vectors, infrastructure, and internal teams for awareness against the latest threats
- Automate simulations for repeatability and consistency
- Conduct testing at any time interval—hourly, daily, weekly, or ad hoc with results in minutes
- Identify gaps and evaluate controls against the MITRE ATT&CK framework
- Remediate security posture and the company's exposure using actionable insights
When cyber adversaries continue to up their games, you and your executive team need assurance that controls across the kill chain are indeed delivering the protection you need—every day, every hour, or every moment. For a growing number of organizations, BAS is delivering the continuous security control and cyber risk assessment data needed to achieve that goal.
For more information, visit Cymulate and sign up for a free trial.
Have something to say about this article? Comment below or share it with us on Facebook, Twitter or our LinkedIn Group.