What About the Testing You Can't Automate?

The shift to DevSecOps is altering the security role in some fundamental ways. We’ve seen this new environment changing not only the security team’s tasks and responsibilities, but also their mindset. Specifically, the security team has had to shift from thinking like a “breaker” to thinking like a “builder.” Rather than focusing on auditing the code at the end of the development cycle, they now need to focus on building security into the SDLC. And that builder mentality requires working closely with those “building” the code. In DevSecOps, the security team enables developers to test for security while they’re writing the code, rather than actually doing the testing themselves after the fact. And the key to this transition? Automate, automate, automate. DevOps is all about automating, and nowhere is that more true than in security testing. If security testing is “shifting left,” it can’t slow developers down, but needs to be integrated into their processes and tools and automated so that it requires little human intervention. But then a question emerges: what about the security testing I can’t automate? Some testing requires security experts, and simply can’t be automated, like threat modeling and manual penetration testing. How do they fit into this new DevOps-driven, automated world?

Non-automated testing needs to shift left too

First, it’s important to think about any required manual processes out of band of the normal pipeline – you don’t want to create any gates that hold up the build. Early in the process – ideally during planning, before any coding gets underway – work to identify if and when threat modeling and manual testing has to happen. Then conduct this testing as early as possible as well. For instance, do threat modeling as soon as you have some design, but before you have any code. For manual penetration testing, you need to have some code written, but it doesn’t have to be 100 percent complete to start this type of test.

Batching is key

Another important component to manual testing in a DevSecOps world: batching. The best way to fit manual testing into a DevSecOps environment is by doing it in small batch sizes. For example, think about threat modeling a piece of the software that is being planned now, and is going to be built over the next few days. Threat modeling a small piece of code will take hours or a day, compared to the typical threat modeling exercise, which takes several days. In addition, when you work this way, you’re completing tasks in parallel, rather than holding things up. In terms of manual penetration testing, the standard model is to wait for the work of multiple scrum teams to come together into a lot of functionality and then, right before release, pen testing it. Instead, do pen testing on small pieces of code, for example, one feature identified as needing an additional layer of testing, or that simply can’t be assessed with automated testing alone. For instance, if you are updating a password rest mechanism – which clearly includes some security-critical business logic – identify that in that planning process as a piece that needs manual testing, and schedule it.

Don’t let manual processes slow DevOps

With this type of planning and batching, you can conduct critical manual testing and not slow down software development processes. In addition, you are using resources more effectively by stretching expert security resources over many projects.

Continue the conversation

We talk more about integrating security into fast and incremental development processes, and about security’s changing role in this new environment in our guide, The Security Professional’s Role in a DevSecOps World.