Top Metrics to Demonstrate the Need to Expand an Application Security Program

You’ve started an application security initiative, yet you know you need to do more. But how do you prove the need to do more? Whether you’re making the case to executives or developers, we’ve found it’s hard to argue with numbers. Collecting a few key metrics will create a clear picture of where you are falling short, and where you need to expand your program.

Every organization has unique priorities and initiatives, and will need to collect its own customized set of metrics, but the following set is a good place to start. By measuring these key areas you can show progress as well as the need to expand your activities.

Your flaw density

What it reports:

Flaw density reports on where the most code flaws are seen. For instance, you could see more flaws in:

• Code developed by particular development groups
• Third-party code
• Applications developed in a particular language

How to use this information:

If your flaw density numbers reveal:

• More flaws in one language type: Make the case for adding developer training to your program. Each programming language has idiosyncrasies that make it more susceptible to different kinds of vulnerabilities. For example, a report released by Veracode in the fall of 2015 found that applications written in web scripting languages have a higher prevalence rate of vulnerability classes like SQL injection and Cross-Site Scripting than applications written in .Net or Java. Your flaw density results might indicate the need to train developers on avoiding flaws in their particular programming language.
• More flaws by one development group: This metric also makes the case for developer training. Research done by Veracode has found that implementing an eLearning program has a big impact on vulnerability remediation, as well as on reduction in overall flaw density. Specifically, we found that development organizations that leverage eLearning see a 30 percent improvement in fix rate.
• Flaws in third-party applications: This number justifies expanding your program to include third-party software.
• Many flaws remaining in production applications: You can use this number to promote the need for additional testing methodologies (for instance, add DAST or runtime protection) to reduce the number of vulnerabilities reaching the production phase.

Your fix rate

What it reports:

Your fix rate illuminates your window of exposure, or how long it takes your organization to fix vulnerabilities. In other words, how long did known vulnerabilities stay open, especially serious vulnerabilities?

How to use this information:

If your window of exposure is unacceptable, you could use this metric to make the case for:

• Runtime protection: This technology will protect applications in production while you work to remediate vulnerabilities.
• Developer training: Recent Veracode research reveals that organizations using our remediation coaching services improve code security by a factor of 2.5x compared to those that choose to do it on their own.
• Additional assessment types: Our research has found that vulnerabilities discovered through the use of static analysis have a 28 percent higher fix rate than those found by dynamic analysis. While no single assessment technology can secure an application alone, understanding a technology’s strengths and weaknesses in terms of fixing — not just finding — vulnerabilities, will have a direct impact on your application security program.

Your rank in application security maturity models

What it reports:

Application security maturity models, like OpenSAMM, are a compilation of objective recommendations on best practices for application security from industry professionals. Leverage application security program maturity models such as OpenSAMM7 or BSIMM8 to understand, measure and compare your application security initiatives against best practices and industry leaders.

How to use this information:

• Illuminate gaps: Use this model to point out gaps in your program, and make the case for filling those gaps based on the lessons learned and best practices of those who have gone before you. If others are finding success and reducing risk by mapping to a particular model, make the case that this is what you should strive for.
• Compete better: Make a case for expansion based on what your competition is doing. If the maturity model reveals that your program is far from the industry standard, you could be giving your competitors an edge. Customers are increasingly asking security questions when purchasing software, and if you are far behind the competition in terms of security, you could be losing customers to a more security-minded organization.

Your compliance with industry regulations

What it reports:

This metric reveals where you are meeting industry regulations, such as PCI-DSS, SOX, HIPAA or NIST 800-53, and where you are falling short.

How to use this information:

Compliance is a major driver in many application security initiatives, and not meeting regulations is a big impetus for change. It’s also a signal that your organization is at risk. According to a recent Verizon PCI Compliance Report, 84% of organizations that suffered a data breach were out of compliance with application-layer security controls (Requirement 6).

If you have gaps in your controls or assessments that you need to add in order to be compliant, you’ve got a strong case for adding these elements into your program. For example, some regulations require:

• Code reviews be built into the SDLC
• Use of both manual and automated assessment methods
• Developer training on secure coding
• Controls around third-party software

Your compliance with internal policies

What it reports:

These numbers reflect how well you are meeting the requirements in your own internal policies. These policies frequently contain requirements like remediate OWASP Top 10 vulnerabilities (see image above), or reduce our flaw density by a certain percentage.

How to use this information:

If you aren’t meeting your policy requirements, your application security program is not effectively reducing your risk. Use this metric to make the case for expanding your program to include elements that would address these shortcomings.

For example, if your flaw density number is not going down, or OWASP Top 10 vulnerabilities are emerging in production applications, you’ve got a strong case for:

• Incorporating developer training
• Adding assessment methods
• A push to get assessment methods integrated into the SDLC

For further details on explaining the need for an expanded application security program, see our new guide, Top 6 Tips for Explaining Why Your Application Security Journey Is Just Beginning.