"We Don't Sell It? Then It's Not Important"

[UPDATE: Since there seems to be some confusion, the "We" in the title of this post is NOT "Veracode". The expression is a generic one intended to illustrate the attitude exhibited by many companies who like to downplay the value and/or effectiveness of technologies that they themselves do not sell. I can't believe I am having to explain this.] Fair warning, this is a bit of a rant. Back in my consulting days (early 2000, I'm getting old), we delighted in the fact that our web application penetration testing methodology didn't rely on automated tools. This was completely true; we did everything manually, and we were among the best in the industry. Many so-called security consultants of the day would run a commercial web scanner and repackage the results as a high dollar "penetration test" -- what a ripoff! What we didn't acknowledge to our customers is that those web scanners, even in their immature state, were probably capable of detecting some of the low hanging fruit that we didn't want to spend our time looking for. Oh, we'd find a few "representative examples" of XSS and SQL injection, but then we'd get bored and move on to the more interesting and complex attack vectors. In our naivete, we figured developers would be inspired to revisit their entire input validation and/or output encoding practices, as opposed to just fixing the proof-of-concept examples we found. Meanwhile, the commercial web scanner vendors were always downplaying the value of manual testing! "Why would you want to pay for an expensive penetration test when you can just run this less expensive tool and find the same vulnerabilities?" They'd gloss over all the technical challenges of automated web scanning and conveniently forget to mention how it was impossible for them to find authorization issues, cryptographic weaknesses, business logic flaws, and so on.

What's my point?

Using multiple testing methodologies is crucial. Sure, there may be some overlap, but ultimately they are complementary to one another. That's why at Veracode, we've never positioned automated static analysis (SAST) as a complete solution. That's why we integrated both automated web scanning (DAST) and manual penetration testing into our service offerings less than a year after launching the company, even though SAST is our patented bread-and-butter technology. This meant we could always be completely honest about the strengths and weaknesses of each technique. I've had a slide titled "There Is No Silver Bullet" in my corporate slide deck since the very beginning.

Our silver bullet is better than yours

Meanwhile, it's been amusing to watch other companies -- who only had a single offering -- having to espouse the tactic of downplaying any testing approach that wasn't in their service portfolio.

• • Over at Fortify, Brian Chess famously predicted that 2009 would mark the end of penetration testing.
  • Over at WhiteHat, Jeremiah Grossman often downplays the value of writing secure code and testing code quality.
  • Even as recently as last week, we have Errata Security (a consultancy) claiming that automated tools are useless and doomed to fail. Welcome back to 1999.

I'm only picking on these guys because they're visible, well-respected practitioners in the application security space. Of course Brian knows source code scanning is an incomplete solution, and now that Fortify and WebInspect are part of the same parent company, I suspect he's adjusted his message. I'm certain Jeremiah knows there's value in writing secure code during the SDLC, which is why WhiteHat is now trying to get into the SAST market by acquiring some technology. And I'm pretty sure Dave Maynor knows automation does provide real value. How else can a big company -- spooked by all the recent breaches -- quickly hunt for SQL injection vulnerabilities across 5,000 websites without the benefit of automation? How does one look for issues in the 150 third-party libraries you use, where only the binary is available? Do you hire Mark Dowd to spend a month looking at each one?

Building trust

We all know a few sales reps that jump from one company to another, changing their pitch as they go no matter how much it conflicts with things they've said in the past. First a service-based approach is best, but suddenly an on-premise tool is better. Source code scanning used to be pointless, but now it's the best thing since sliced bread! It's no surprise these guys don't experience more success -- they lack credibility. The most successful account reps I've seen are the ones who build trust with their customers over time by being honest about what they are selling, even when hopping from one company to the next. Look, it's no big secret why people talk up their own stuff and imply everything else stinks. It's part of the sales and marketing machine and by no means is it unique to the security industry. Even so, can't we make an effort -- as practitioners -- to cut back on the rhetoric a little bit and be more honest with our customers? Customers look to us as experts to help them build their security programs, and what do we do? We oversell them on an approach that has huge gaps we pretend don't exist. If you're really looking out for your customers, start being more honest, and stop handing out kool-aid. Here's another approach: Instead of outright dismissing an effective technology or methodology just because you don’t sell it, sometimes it's worth thinking about partnering, or even building something better. That's why at Veracode we designed our service platform around the idea of technology integration. There is no silver bullet and there never will be.

Veracode Security Solutions

Security Threat Guides