Finding the Elusive Value in Penetration Testing

Share it on Twitter  
Share it on Facebook  
Share it on Google+
Share it on Linked in  
One of the most common security practices today also happens to lead toone of the most common mistakes made.

Nearly every large IT security group makes some use of penetrationtesting these days, whether they do so in-house using open source toolsor outsource it IT service firms. Either way, it's happening. And thetrouble here lies not so much in the testing itself, but in how theresults are interpreted and used.

To start with, penetration test (or pen test) results can be interpretedon at least two levels -- technical and procedural. All too manyorganizations focus on the technical findings of a pen test as ameasurement of their security posture. This leaves the organizationwith, at best, a false sense of security (or insecurity if the testresults are egregiously bad).

You see, fault testing, by its very nature, is only capable of testingfor the presence of flaws, not their absence. Further exacerbating thisweakness is the fact that the crop of available tools, whethercommercial or open source, typically only tests for an, arguably, smalland known set of vulnerabilities.

So, at its most fundamental level, a pen test does not test yoursecurity. It partially tests your insecurity. That's hardly the sort of''warm and fuzzy'' feedback that I'd be comfortable with taking to theboard of directors.

If there is any value to be found in pen tests, and I believe that thereis, then it lies in an area that few people pay attention to when theylook at their pen test findings -- the procedural aspects. By that Imean that the pen test findings can point to problems in systemsadministration processes and procedures.

After all, the flaws discovered during a pen test are really justsymptomatic of procedural shortcomings in the systems managementprocesses.

A laundry list of a hundred or so system vulnerabilities unearthed inthe latest pen test is of marginal value in and of itself. Or, to put itanother way, using that laundry list as a checklist of things to fix isof nominal value. But using the pen test findings to address and remedyprocedural problems can be a healthy way of improving your systemsmanagement processes.

Instead of divvying out the findings list to the IT staff for them tocorrect, why not start by looking at how the vulnerabilities came to bein your data processing environment in the first place?

That's where the gold is.

For example, are your productions servers not being adequatelyconfigured and ''hardened'' during the setup phase? Or, did theconfiguration management team neglect a critical flaw and its associatedpatch for an unacceptably long period of time? Was a network serviceinadvertently opened up during a maintenance cycle or upgrade on anetwork device? If so, why wasn't the device's configuration verifiedbefore it was restored into service on the production network?

Those are the types of questions that you should be addressing. That'snot to say that you should ignore the actual vulnerabilities that werediscovered during the pen test, but you should be looking at thesomewhat bigger picture, as well.

Now, let's re-visit the technical aspects of pen testing for a moment.

As I said, the tools that we have available today, at least for the mostpart, test for some set of known vulnerabilities, in much the same waythat an anti-virus scanner examines a system for a known set of viruses,Trojans, and other malicious software. That's not entirely uselessinformation to have, to be sure, but it is, by nature, a reactiveapproach that only finds problems after the mistakes have already beenmade.

If we are ever to get serious about software security, then we have tostart our security efforts much earlier in the software developmentprocess, but that's a topic for another column.

My point for now is that pen testing is not in any way an acceptablemeasure of the security of software.

If you're going to use pen testing, then it's important to be cognizantof its strengths, as well as of its limitations. Use pen testing toassess your IT operations processes and methodologies, not to measurehow secure you are or to test the security of any software that youdevelop in house. Both of those aims require very different processesthan a simple pen test can ever provide.

Kenneth van Wyk, a 19-year veteran of IT security, is the prinicpalconsultant for KRvW Associates, LLC. The co-author of twosecurity-related books, he has worked at CERT, as well as at the U.S.Department of Defense.

Submit a Comment

Loading Comments...