Fuzzing Tools

Since Miller's first fuzzing tool, countless tools have been developed and used by security professionals. According to TippingPoint security researcher Pedram Amini, fuzzing tools can be broken into two categories, general purpose and target-specific.

Amini and his team at Tipping Point recently came up with their own fuzzing framework called Sulley, which can be customized to audit a wide range of targets. On the other hand, to better secure its own browser, Mozilla developed a fuzz testing tool called jsfunfuzz that is targeted at testing JavaScript inputs.

Browser vendors certainly have a particular need for using fuzz testing. Last year, security researcher H.D. Moore filled his Month of Browser Bugs effort with a number of bugs that came out of his own fuzzing efforts.

In addition to simply trying to crash a program, Amini noted that fuzz testing could help researchers find all kinds of problems: memory leaks, performance degradation, denial of service, format string bugs and integer handling issues.

"Fuzz testing can expose the same classes of risk that a human analyst can -- the trick is detecting the discoveries," Amini told InternetNews.com.

What Developers Should Do

While there are a number of reasons why fuzzing works in the first place, a large cause is that software developers simply introduce errors. Consequently, software can be better protected against the kinds of vulnerabilities that fuzzing exposes if programmers just code properly.

"In more general terms, the No. 1 security mistake that programmers make is to cut corners -- or altogether omit -- input validation," Fortify's Chess told InternetNews.com. "Good input validation makes fuzzing go limp."

Yet using that technique doesn't mean that fuzzing still won't turn up additional errors. Tipping Point's Amini said that getting zero results from a fuzzing test is a tall order, and likely unachievable.

"Developers can certainly cover a lot of ground, however, by applying fuzz testing on their own code during the development lifecycle," Amini said. "Fuzz testing does not require as specialized a skill set as line-by-line source or assembly auditing, which makes it an attractive testing methodology to involve developers in."

The More Things Change, The More They Stay The Same

Taking the long view of fuzzing following the nearly 20 years since he developed the technique, Miller said he has seen some disturbing trends. Even though his research at the University of Wisconsin has been available to the public developer community, the fuzzing studies he has done over the years have found worsening problems in software coding.

Miller said that in 1995, he could hang or crash a quarter of X Window applications on Unix. In 2000, he could hang or crash at least 45 percent of the Microsoft Windows applications he tested. In 2006, he tested Mac OS X and found that he could crash over 70 percent of the items studied.

The reason for these increasing problems? A simple matter of economics, according to Miller.

"There is a constant notion that you have to release software that makes people want to buy the next version," Miller said. "People assume that bug fixes are things you get for free as updates and features are things you buy. So long as we have that business model, people will only make software as reliable as the market demands and no more."

Fortunately for developers, fuzzing remains a useful tool to combat bugs even 18 years after its advent. On that fateful night back in 1989, Miller was using technology that many would today consider to be Stone Age by today's standards, yet the techniques he developed still work.

"The way we're programming today isn't all that different from the way we did 25 years ago. We're still using languages that are error-prone," Miller said. "We're writing software that is more complex and the tools are only incrementally better. I don't know if we're overall keeping up or if we're slowly drifting behind."

This article was first published on InternetNews.com.