Many cybersecurity audits now ask whether penetration testing is conducted and how vulnerabilities are detected and tracked. These questions ask IT teams to consider how frequently security is tested from the outside via penetration testing and from the inside via vulnerability testing.
That right there – inside vs. outside – should give you a good idea of a key difference between the tests, but for those who don’t spend their days performing compliance audits, the difference between these two tests can be hard to understand. Ultimately, while penetration testing is more commonly required, the addition of vulnerability testing leads to a more secure organization.
At A Glance: Penetration tests vs. vulnerability tests
- When to use penetration testing: To find external vulnerabilities in things like firewalls, web applications and gateways
- When to use vulnerability testing: To find vulnerabilities inside your network — important for stopping lateral movement by hackers who have penetrated your network
See our guides to the best penetration testing and vulnerability scanning tools
What is a Penetration Test?
Penetration tests of IT systems probe external security protections to find holes and weaknesses from the outside of the organization without any advanced knowledge of its systems.
Penetration tests include the use of vulnerability scanning tools and will generally be applied against external security devices and applications including, but not limited to, firewalls, web servers, web applications, gateways, and VPN servers. Penetration testing can also involve common hacking techniques such as social engineering, phishing attacks, dropped USB drive attacks, etc.
For many compliance regimes, an annual penetration test is required, but some organizations may be required to test more frequently. All organizations are encouraged to perform a penetration test after any significant change to their IT infrastructure or security, but not everyone has the time.
Also read: Penetration Testing: How to Start a Pentesting Program
What is a Vulnerability Test?
Penetration tests provide critical information about externally visible vulnerabilities; however, vulnerability testing may be even more valuable. Most companies do a good job with basic security, and penetration testing often will fail to find anything of interest.
However, all it takes is one bad click on a phishing campaign, and suddenly attackers will be looking at an organization from the inside. Vulnerability tests of IT systems seek to look for these security holes from inside the organization.
Often called vulnerability scans, vulnerability testing is about finding vulnerabilities and prioritizing them in order of their threat level and likelihood of exploitation.
Vulnerability tests start with knowledge of the internal systems and, like penetration testing, can use vulnerability scanning tools with internal permissions to test internal IT systems. These tests will often expand upon the devices tested by penetration tests and include:
- Internal networking equipment, such as switches and routers
- File servers
- Network accessible storage (NAS) devices
- Individual computers
- Peripheral devices like printers and scanners
- Internet of Things (IoT) devices connected to the network, such as security cameras, TVs, etc.
- Critical applications and internal processes, such as Active Directory (AD); Domain Name System (DNS); and accounting, banking, or operations management software
Vulnerability testing will often start with a basic user’s level of security to attack the organization. This will simulate how an attacker might exploit access provided by a user that fell for a phishing attack and had their credentials stolen.
Lastly, for companies that create their own applications, vulnerability scanning can involve scanning their software libraries and their supply chain for known vulnerabilities. Recent research by Veracode finds that more frequent vulnerability scanning has reduced the typical number of vulnerabilities by two-thirds and decreased the time to fix vulnerabilities by more than 30%.
See the top code security and debugging tools
Vulnerability testing is encouraged to be conducted regularly, but is only required by some compliance regulations. As an example, the PCI Data Security Standard (PCI DSS) requires vulnerability testing to be conducted at least quarterly and after every significant IT or security change.
Penetration vs. Vulnerability Testing
Ultimately, both penetration testing and vulnerability testing provide great value to any organization. So why do we seem to hear so much more about penetration testing than vulnerability testing? There are three reasons: popularity, minimum requirements, and overwhelming workloads.
Popularity
Penetration testing is seen as being cooler than vulnerability testing. Penetration tests, also known as red team attacks, can be more fun for IT security investigators, so it always seems like there are many more classes and certifications devoted to red team techniques than to the defensive blue team techniques. “White hat” and ethical hacking certifications are just cool to have.
However, this perspective also is skewed by the reality that a penetration tester only needs to find one way in to be successful and a blue team defender needs to be skilled against every possible red team tactic. The number of classes is also quite skewed, since every class that is not specifically labeled as a red team or penetration testing class is essentially a vulnerability testing class.
A large part of vulnerability testing is checking to see if IT teams did their job correctly in building out a company’s infrastructure and setting up its security. Vulnerability testing seems more like homework than hacking.
Minimum Requirements
While most regulations will have requirements for both penetration testing and vulnerability testing, many third-party compliance questionnaires only ask for penetration testing and vulnerability management.
Vulnerability management can be checked off by performing patch management or using an update and patch management service. Many businesses only seek to meet the minimum requirements and will simply stop once those boxes are checked.
However, this will likely be an evolving condition over the next few years. New York State already requires both penetration testing and vulnerability assessments for its financial institutions, and both PCI DSS and the National Institute of Standards and Technology (NIST) cybersecurity framework (800-115) have vulnerability scans as basic requirements.
We should expect other laws and regulations to begin to meet these framework standards in the near future as cybersecurity attacks continue.
Overwhelming Workloads
Sometimes vulnerability testing uncovers more vulnerabilities than can be fixed. Healthcare facilities notoriously find many issues since many medical imaging devices run on operating systems that no longer receive updates (see Three Ways to Protect Unfixable Security Risks).
Then, there are the thousands of vulnerabilities with low Common Vulnerability Scoring System (CVSS) ratings because the vulnerabilities can be difficult to exploit. Many vulnerability scans will produce a list of ratings that may not seem very important.
For example, an organization may find that a marketing server uses iText V.7.1.17 to create and manipulate PDF files in Java. This software contains vulnerability CVE-2022-24198 that allows a specially crafted PDF to cause a distributed denial-of-service (DDoS) attack. If marketing is not using this server all the time, then the issue can likely wait to be fixed to prioritize more pressing vulnerabilities.
Why Both Are Important
When an IT team struggles to keep up with its workload, performing vulnerability scans and then tracking low severity vulnerabilities seems like more trouble than they’re worth. It is easier to perform the penetration scan and claim the organization is secure.
However, a huge number of attacks start through social media or through phishing. With one bad click, the attacker is inside the organization and behind the wall of security tested in the penetration test.
Vulnerability testing looks at the organization from the perspective of an attacker that has successfully breached the organization. It reveals the easiest targets and fastest approaches an attacker might deploy.
Vulnerabilities force organizations to perform a critical task: risk assessment. While such assessments do not need to be formal, organizations with mature security processes will maintain a prioritized risk register.
Critical risks obviously need to be addressed right away and the register will track the date and the method used to address the vulnerability (updates, patches, controls, asset replacement). Lower-rated risks can be prioritized based on the value of the asset, the severity of the vulnerability, and the control.
Both forms of testing catch mistakes and give IT teams a chance to fix them. Verification of secondary layers of defense against an attack on the inside can be extremely valuable. Companies that really want to avoid the headaches of a breach will want to run both penetration testing and vulnerability testing.
Further reading:
