Van Wyk on the 'Top 25 Programming Errors'
A noted security expert talks about the larger meaning of the recently released list of top programming errors.
Ta dah! Wow, that ta dah didnt have the public impact I expected. So then, whats the big deal about the world producing yet another top-N list? Ill tell you why this one is different.
I can certainly understand if you werent bowled over by the announcement, though. Its a natural (non)reaction. After all, the world already has the SANS Top-20 Vulnerabilities list and the OWASP Top-10. Do we really need another list that points out our problems? Lets consider a few things here.
So then lets consider the OWASP Top-10 list. They get closer to the root causes by addressing software application problems like cross-site scripting and SQL injection. And, to be fair, their documentation does a superb job at describing these weaknesses and pointing to effective remediation steps that can be taken, right down to code examples. Excellent stuff, but it too falls short of truly pointing a finger directly at the underlying problems themselves.
I should also point out MITREs own CVE and CWE efforts. The Common Vulnerabilities and Exposures (CVE) project documents (and makes searchable) a collection of software vulnerabilities, patches, etc. Its counterpart, the Common Weakness Enumeration (CWE) project is a dictionary of underlying software weakness types. Both of these are excellent resources to information security as well as software development staff.
Now do you see whats missing here? Or perhaps you think you could probably gleam the vast majority of the information in the CWE/SANS Top-25 list by poring through the CWE information? Thats true, but only to a point.
From where I sit, I see two things missing. For one, the SANS Top-20 list is hugely advertised and well known. Of course its not a comprehensive list of all the security problems on the Net, but it is arguably the most well known list of problems, and that fact by itself is not a bad thing. (It can become a bad thing if we only address these flaws, but more on that later.)
The second thing missing is that until now we didnt have a list of the biggest, baddest, nastiest programming security defects. We didnt have a counterpart, if you will, to the OWASP list that speaks specifically to the programming mistakes and not just the vulnerabilities. After all, an XSS vulnerability can look a lot different in different contexts.
Until we draw attention to the programmatic problems that lead to XSS vulnerabilities, were only talking about the symptoms and not the problems. Thats the real big deal here.
And just like the other Top-N lists, this one isnt comprehensive. There are many other programming mistakes that can be made, of course. We all understand that, right? Right?
Well, theres a danger there as well. Weve seen the SANS list being adopted by many a security auditor as a mere checklist of things to look for. If the CWE/SANS list gets similarly adopted, were guilty of a very bad thing indeed: negative validation.
Negative validation happens when we evaluate something against a set of known bad things and assume it to be safe if we dont find them. Positive validation, on the other hand, evaluates something against a set of accepted good attributes and presumes it to be dangerous if it doesnt conform. (Remember me mentioning that in my column here just a few short months back?)
The danger of negative validation here is that we must not focus solely on these 25 bad things. That said, raising awareness to these bad things is positive and valuable. This list has already been widely cited in just about every trade publication Im aware of. Thats great news for us all.
OK, so this list is different. How should we work with it in our day-to-day work? For starters, every software developer should be exposed to the CWE/SANS Top-25 list. They should all understand the issues and how to avoid them. But dont stop there.
We all need to also ensure that software developers understand the underlying sound engineering principles that are implicitly referenced in the list. Things like the principle of least privilege, compartmentalization, and so onthink Salzer and Schroeder circa 1975. You know, the things that instantly and irrevocably cure insomnia among software developers. Well, you can use the Top-25 list as a way of drawing attention to those principles in an interesting and engaging way.
So yes, there is real value to the CWE/SANS Top-25 list. Use it to raise peoples attention. Use it to make change. Just dont get so hung up on the list that you miss the underlying messages. Sound engineering principles are the foundation that we need to build a strong and reliable infrastructure, and we mustnt ever lose sight of that.