For those that don't know, anti-debugging is the implementation of one or more techniques within computer code that hinders attempts at reverse engineering or debugging a target process. Typically this is achieved by detecting minute differences in memory, operating system, process information, latency, etc. that occur when a process is started in or attached to by a debugger compared to when it is not. Most research into anti-debugging has been conducted from the vantage point of a reverse engineer attempting to bypass the techniques that have been implemented. Limited data has been presented that demonstrates anti-debugging methods in a high level language that the average developer can understand. It is with this in mind that I hope to begin a series of posts that present some of the methods of anti-debugging in a clear, concise, and well documented fashion. The end goal of this series is to arm developers with the techniques and knowledge that will allow them to add a layer of protection to their software while simultaneous educating reverse engineers in some of the anti-debugging methods used by malware authors today.
Before we delve into the intricacies of individual methods of anti-debugging let's use this post to define the classes of anti-debugging that we will be discussing. While other classes may exist, the definition of these classes is an attempt to include the majority of anti-debugging methods in use today. There is some overlap between classifications and we may have left out some methods due to limited exposure or effectiveness.
API Based Anti-Debugging
API based anti-debugging is the most straightforward and possibly the easiest to understand for a typical developer. Using both documented and undocumented API calls, these methods query process and system information to determine the existence or operation of a debugger. From single line calls such as IsDebuggerPresent() and CheckRemoteDebugger() to slightly more complex methods including debugger detaching and CloseHandle() checks. These methods are generally trivial to add to an existing code base and many can even be implemented in as few as two or three lines.
Exception Based Anti-Debugging
Exception based anti-debugging is slightly different than your basic API based techniques. Many times when a debugger is attached to a process, exceptions are trapped and handled by the debugger without regard to passing the exception back to the application for continued execution. Occasionally these exceptions can even crash or terminate a process when run under a debugger and be handled gracefully when running clean. It is these discrepancies that makes exception based anti-debugging techniques possible.
Process and Thread Block Anti-Debugging
Some of the API based anti-debugging methods use published functions to query information from within the process and thread blocks for our running code. Many API based detections can be subverted within a debugger by hooking the API call and returning values that indicate a clean process. One way around this subversion is to directly query the process and thread blocks, bypassing the API calls. Direct analysis of the process and thread blocks, while more complex, can lead to a more accurate and high assurance result.
Modified Code Anti-Debugging
One of the methods that a debugger uses to signal a breakpoint is to insert a break byte into the running code at the location that it wishes to stop execution. The process execution breaks when this value is seen, giving control to the debugger. When the program is resumed, the breakpoint value is removed and replaced with the original byte, the execution backed up one byte, and the program is resumed. Detection of software based breakpoints can be achieved by analyzing the process for modifications from the expected norm.
Hardware and Register Based Anti-Debugging
A second way that a debugger can break the execution of a process is by using a hardware breakpoint. A hardware breakpoint relies upon CPU registers to store the pertinent information and to detect when the target break addresses are seen on the bus. A break interrupt is triggered at the appropriate time based on these register values. Reading or modifying the hardware can allow for the detection of a debugger.
Timing and Latency Anti-Debugging
Finally timing and latency can be used as an effective anti-debugging method. When executing a program within a debugger, specifically when single stepping, a much larger latency occurs between execution of instructions. This latency can be detected and compared against a reasonable threshold to detect the existence of a debugger attached to our process.
Each of the classes of anti-debugging outlined above has merit when used individually to protect a process. While none of them can be assured to ever protect a program from a determined reverse engineer or debugger, implementation of these techniques (or many of them if appropriate) can sufficiently slow down the debugging process and hopefully make the attacker spend his time on other, easier, ventures. In the remainder of this series on anti-debugging we will review in depth some of the more interesting methods of each of the above classes. So bring along your debugger and your development environment and let the games begin.
