Focus on Fixing, Not Just Finding, Vulnerabilities

When investing in an application security (AppSec) program, you expect to see a return on your investment. But in order to recognize a return, your organization needs to determine what success looks like and find a way to measure and prove that the program is meeting your definition of success.

For those just starting on their AppSec journey, success might be eliminating OWASP Top 10 vulnerabilities or lowering flaw density. But as you begin to mature your program and work toward continuous improvements, you should start measuring your program against key performance indicators (KPIs) like fix rate. Fix rate is used to indicate how fast your organization is closing – or remediating – flaws. The formula for fix rate is the number of findings closed divided by the number of findings open. As you can see in the diagram below, of the 6,609 flaws, 2,581 flaws are open and 4,028 are closed. This means that flaws are remediated at a rate of 16 percent. The faster your organization fixes flaws, the lower the chances of an exploit. For the sake of continuous improvement, you should be finding that your organization is improving its fix rate by remediating flaws faster year over year.

Using Veracode Analytics to examine fix rate and prove AppSec success.

Using Veracode Analytics custom dashboards, you can examine your total fix rate or break it out by application, scrum team, business unit, or geographical location. These dashboards can be shared with stakeholders and executives to show areas where your fix rate is improving or areas that need additional attention and resources.

When examining fix rate across applications, you should be finding that your more critical applications have a better fix rate. If that’s not the case, you need to be examining the application security policies you have in place for fixing flaws. High-severity and highly exploitable flaws should be prioritized over low-severity flaws with a lower chance of exploitability. The same logic applies to applications: High-risk applications storing large amounts of sensitive data should be prioritized.

When examining the fix rate across scrum teams and locations, you should find that teams and geographical locations are continuously improving their fix rate. If not, you should use the data to tailor future security trainings or to ask stakeholders and executives for additional resources.

How does fix rate impact return on investment?

By remediating flaws faster, you are reducing the chance of an exploit which could cost your business thousands – even millions – to resolve. For example, Capital One had a third-party vulnerability that was not remediated, and it led to a massive data breach which exposed its customer’s social security numbers and bank account numbers. It cost Capital One approximately 150 million dollars to resolve the matter.

Faster time to remediation also means faster time to production. Once developers fix all of the flaws defined in their policy, code can be moved to production. If code is moved to production at a faster rate, an organization – and its customers – can start recognizing value from the application sooner.

For additional methods on proving AppSec success, check out our recent video, How to Use Analytics to Measure AppSec ROI.