Anti-Debugging Series - Part II

Welcome back to the series on anti-debugging. Hopefully you have your debugger and development environment handy as we are about to dive into the first round of anti-debugging code. In the first post to this series we discussed six different types of anti-debugging techniques that are in common use today. To refresh, the classifications buckets that we chose to use are:

• API Based Anti-Debugging
• Exception Based Anti-Debugging
• Process and Thread Block Anti-Debugging
• Modified Code Anti-Debugging
• Hardware and Register Based Anti-Debugging
• Timing and Latency Anti-Debugging

Basic API Anti-Debugging We'll continue this series of posts by going into a bit more depth on the easiest of API based anti-debugging techniques. An application programming interface (API) is used to support requests made from other applications for resources or functionality within a target service or library. In our case we will be primarily focused on the Microsoft Windows operating system API. There are a number of calls built directly into the operating system API that make detection of a debugger possible. Minor differences in thread and process meta-data is present when processes are run within a debugger. These calls typically facilitate a process or thread examination technique in order to determine if the target thread has a debugger attached. When learning about anti-debugging, a developer will typically first be introduced to the IsDebuggerPresent() function. This function analyzes the process block of a target process to determine if the processes is running under the context of a debugging session. We'll save the details of how this actually works for a later article, however suffice it to say that the target process has a flag that will contain a non-zero value if the process is being debugged. This flag is queried and returned when IsDebuggerPresent() is called. A very basic debugging detection routine would be to call this function and execute different code paths based on the response.

Prototype: BOOL WINAPI IsDebuggerPresent(void); 

if (IsDebuggerPresent()) {
    //Debugger Detected - Do Something Here
} else {
    //No Debugger Detected - Continue
}

We could also use the API function CheckRemoteDebuggerPresent(). Contrary to first thought, this function does not target a process on a remote machine, nor does it even require that it target a process remote to itself. The call can use a parameter pointing to itself to determine if it is running inside of a debugger. In the example below we pass in a handle to our current process by calling the GetCurrentProcess() function along with a variable to hold the return value from the CheckRemoteDebuggerPresent() call.

Prototype: BOOL WINAPI CheckRemoteDebuggerPresent(__in HANDLE hProcess,
           __inout PBOOL pbDebuggerPresent);

BOOL pbIsPresent = FALSE;
CheckRemoteDebuggerPresent(GetCurrentProcess(), &pbIsPresent);
if (pbIsPresent) {
    //Debugger Detected - Do Something Here
} else {
    //No Debugger Detected - Continue
}

While these two methods are probably the easiest and most straightforward methods of anti-debugging, they are also the most likely to be understood by a person wishing to bypass them. We can mix it up a bit and use a call to OutputDebugString() instead. OutputDebugString() is typically used to output a string value to the debugging data stream. This string is then displayed in the debugger. Due to this fact, the function OutputDebugString() acts differently based on the existence of a debugger on the running process. If a debugger is attached to the process, the function will execute normally and no error state will be registered; however if there is no debugger attached, LastError will be set by the process letting us know that we are debugger free. To execute this method we set LastError to an arbitrary value of our choosing and then call OutputDebugString(). We then check GetLastError() and if our error code remains, we know we are debugger free.

Prototype: void WINAPI OutputDebugString(__in_opt  LPCTSTR lpOutputString);

DWORD Val = 123;
SetLastError(Val);
OutputDebugString(L"random");
if(GetLastError() == Val) {
    //Debugger Detected - Do Something Here
} else {
    //No Debugger Detected - Continue
}

These three methods are the basic starting point for a developer wishing to implement anti-debugging into their code base. The methods are so simple they could even be implemented as macros making a call quick and easy. Numerous other API based detection methods exist with a vast array of complexity. In the next post in this series we will discuss slightly more advanced API anti-debugging techniques that will make reverse engineering and debugging even more difficult.

Veracode Security Solutions

Security Threat Guides