Nearly every large IT security group makes some use of penetration testing these days, whether they do so in-house using open source tools or outsource it IT service firms. Either way, it's happening. And the trouble here lies not so much in the testing itself, but in how the results are interpreted and used.
To start with, penetration test (or pen test) results can be interpreted on at least two levels -- technical and procedural. All too many organizations focus on the technical findings of a pen test as a measurement of their security posture. This leaves the organization with, at best, a false sense of security (or insecurity if the test results are egregiously bad).
You see, fault testing, by its very nature, is only capable of testing for the presence of flaws, not their absence. Further exacerbating this weakness is the fact that the crop of available tools, whether commercial or open source, typically only tests for an, arguably, small and known set of vulnerabilities.
If there is any value to be found in pen tests, and I believe that there is, then it lies in an area that few people pay attention to when they look at their pen test findings -- the procedural aspects. By that I mean that the pen test findings can point to problems in systems administration processes and procedures.
After all, the flaws discovered during a pen test are really just symptomatic of procedural shortcomings in the systems management processes.
A laundry list of a hundred or so system vulnerabilities unearthed in the latest pen test is of marginal value in and of itself. Or, to put it another way, using that laundry list as a checklist of things to fix is of nominal value. But using the pen test findings to address and remedy procedural problems can be a healthy way of improving your systems management processes.
Instead of divvying out the findings list to the IT staff for them to correct, why not start by looking at how the vulnerabilities came to be in your data processing environment in the first place?
That's where the gold is.
For example, are your productions servers not being adequately configured and ''hardened'' during the setup phase? Or, did the configuration management team neglect a critical flaw and its associated patch for an unacceptably long period of time? Was a network service inadvertently opened up during a maintenance cycle or upgrade on a network device? If so, why wasn't the device's configuration verified before it was restored into service on the production network?
Those are the types of questions that you should be addressing. That's not to say that you should ignore the actual vulnerabilities that were discovered during the pen test, but you should be looking at the somewhat bigger picture, as well.
Now, let's re-visit the technical aspects of pen testing for a moment.
As I said, the tools that we have available today, at least for the most part, test for some set of known vulnerabilities, in much the same way that an anti-virus scanner examines a system for a known set of viruses, Trojans, and other malicious software. That's not entirely useless information to have, to be sure, but it is, by nature, a reactive approach that only finds problems after the mistakes have already been made.
If we are ever to get serious about software security, then we have to start our security efforts much earlier in the software development process, but that's a topic for another column.
My point for now is that pen testing is not in any way an acceptable measure of the security of software.
If you're going to use pen testing, then it's important to be cognizant of its strengths, as well as of its limitations. Use pen testing to assess your IT operations processes and methodologies, not to measure how secure you are or to test the security of any software that you develop in house. Both of those aims require very different processes than a simple pen test can ever provide.
Kenneth van Wyk, a 19-year veteran of IT security, is the prinicpal consultant for KRvW Associates, LLC. The co-author of two security-related books, he has worked at CERT, as well as at the U.S. Department of Defense.