CISO Commentary: How Often Should You Pen Test?

Recently, Core Security released the 2024 Penetration Testing Report, which shares the results from an annual survey of cybersecurity professionals on their experiences with offensive security strategies and solutions. In this series, we’ll take a deeper dive into some of the most noteworthy findings from the survey, with expert insights from Fortra’s CISO, Chris Reffkin, and Lead Product Manager for Infrastructure Protection, Pablo Zurro. In this first installment, we’ll look into frequency of pen testing. Is there a golden rule about the number of tests an organization needs to run? Read on to find out.

How Often Do Other Organizations Pen Test?

The 2024 Penetration Testing Report reveals a notable trend in the frequency of pen testing. 43% of respondents to the 2024 Pen Testing Survey indicated they test one-two times a year, making this the most common cadence. Other organizations opted for a more frequent schedule, with 23% performing monthly to quarterly tests. A smaller group tested on an even more regular basis, with 17% conducting daily to weekly pen tests. The prevalence of annual to bi-annual testing suggests a schedule that aligns with requirements for compliance standards. Despite a clear majority, it is still evident that there is no one-size-fits-all approach to pen testing frequency. But what factors should be considered to determine the right number?

What to Consider When Developing a Pen Testing Plan

Reffkin recommends starting with a few fundamental questions to determine how many pen tests your organization will need. What are the objectives? What do you want to get out of a pen test?

If you have a long list of objectives, you’ll likely need to run multiple tests. Having too many objectives for one assessment can make the test too broad, leading to a superficial evaluation of vulnerabilities that miss critical security issues that a focused approach would uncover. By concentrating on a narrower set of objectives, the test can provide deeper insights and yield valuable, actionable information that can be used to strengthen the organization's security posture. Additionally, identifying finite objectives can help create a more definitive priority list, which will also help determine how many tests are needed to achieve the most critical goals.

Determining Your Risk Profile

One myth that Reffkin says needs busting is that the size of the organization is the biggest factor in determining how many pen tests you will need. In fact, your risk profile is actually much more important. Smaller organizations oftentimes have a higher risk profile than larger organizations because they do not have a dedicated security team.

What else contributes to your profile? Other attributes to consider include:

Types of Data | The more personal identifying information an organization must store, the greater the risk. For example, hospitals house electronic health records (EHRs) that can contain names, social security numbers, addresses, phone numbers, insurance information, payment information, medical record numbers, driver’s license information, and more. On the other hand, manufacturers often don’t directly interact with customers, reducing the need for personal information.

Exposure | The accessibility of an organization’s systems, services, or data also increases risk as it expands the attack surface and provides multiple points of entry for attackers. While most organizations have some level of online presence, some industries require more than others. Financial institutions with online banking applications have a higher risk, as do healthcare organizations, whereas other companies may only have informational websites.

Business Operations | How an organization operates can also affect risk. For example, working with supply chains and third-party services can introduce risk that can be difficult to manage. Additionally, where an organization operates can play a role, as working in unstable regions with fewer regulations can leave a company less protected.

Security Controls | An organization’s current security posture is the element over which they have the most control. This may include its policies, procedures, and technical controls. A strong security posture can mitigate many risks, while gaps in security can increase the likelihood of a damaging breach.

The Optimal Frequency of Pen Tests

Determining the optimal frequency for conducting penetration tests is not a matter of adhering to a fixed rule or a universally applicable number. Instead, it hinges on several factors unique to your organization. With 17% of organizations reporting that they never pen test, it should be stated that “zero” is rarely, if ever, the “right” number. Pen testing at any frequency not only provides visibility into the effectiveness of your security, it also lays out a path for closing security gaps before they’re exploited.