One of Core Security's most dedicated customers offers his opinion on why penetration testing is so valuable when used in cooperation with vulnerability scanners and other proactive security practices.

Customer Testiment on Penetration Testing

We’ve noted on many occasions that Pennsylvania CISO Bob Maley is one of our marquee customers as he’s been such an outspoken advocate for the use of penetration testing as part of the state’s Commonwealth Application Certification and Accreditation (CA)2 Process. Unlike many others (including certain executives at companies in the vulnerability management space) who confuse automated penetration testing as a replacement for other processes, including vulnerability scanning and source code analysis, Maley truly appreciates the value provided by our CORE IMPACT solutions within a larger context that includes those security practices and many others. That’s probably why the (CA)2 initiative, which he architected and has been adopted as a requirement across all of Pennsylvania’s applications development operations, includes many of these different proactive processes to ensure that the Commonwealth’s systems are being tested for security flaws both during and after they are brought online. Vulnerability scans are very valuable, the CISO points out, but used in cooperation with penetration tests the scanner results become significantly more useful in calibrating issues of risk – allowing for significant savings in time and investment previously dedicated to sorting through scanner output. In a new podcast conducted by CSO online, and hosted by the esteemed IT security journalist Bill Brenner, Maley highlights his approach to vulnerability management, and details why using CORE IMPACT Pro to test the results of PA’s scanning efforts has proven so helpful in isolating the state’s most ominous points of exposure – and in handling issues of staffing bandwidth limitations in executing security and compliance management programs. The podcast is a follow-up to a great story that Brenner did on the entire (CA)2 initiative. I encourage anyone reading this to check out the entire CSO podcast (found here), but below are a few select quotes that we at Core Security feel really get to the heart of the matter in illustrating the complementary nature that penetration tests play in relation to other vulnerability management practices.

On the matter of source code analysis versus penetration tests:

“Static source code analysis is also a critical part of (CA)2  early on in phase one of the requirements we have… and that helps us tremendously, but applications vulnerabilities aren’t the only thing that we’re looking at in our pen testing. I’d agree that both are critical to our security posture and risk mitigation and I don’t see one replacing another; as I see it they enhance each other.”

On complying with PCI DSS:

“With PCI, the primary thing that [auditors] are looking at are vulnerability scans, and anyone in the industry knows, vulnerability scans are great and we need to do those, but, with the mountain of information it returns, the numbers of false positives that come back… that makes it a very difficult piece of our vulnerability management program.”

On using vulnerability scanners alone:

“If we do a vulnerability scan on a subnet with some piece of critical infrastructure on it, say 50-to-100 servers , that can bring back reports of hundreds of pages. If I deliver that to server admins, that’s overwhelming, and like many others we’re at a critical stage with the budget crisis, we have no new hiring going on and we’re not replacing any staff who leave, so, it’s a difficult thing to be put in that position. For me to dump vulnerability scans and expect the IT department to remediate all those… typically responses to that are less than favorable even though there may be real problems.”

On the value of using penetration testing alongside vulnerability scanners:

“An automated pen testing tool allows me to go through and review scans in real time on systems and see what are real vulnerabilities, where are the holes that can be reached. We may have 100 vulnerabilities and if 95 percent are irrelevant and pose no real risk, the penetration testing tool gives me the ability to identify real high-risk vulnerabilities, the 5 percent that are real risks, and get significant buy-in from IT administrators. We get a positive response when we go in that way.” So there you have it, Bob makes a tremendous case for the value of using automated penetration testing alongside vulnerability scanners and other elements of the vulnerability management ecosystem. We think his voice, and those of our other customers, resonates much farther and deeper than anything we could cook-up ourselves.