Anti-Debugging Series - Part III

It's time for part three in the Anti-Debugging Series. With this post we will stay in the realm of "API based" anti-debugging techniques but go a bit deeper into some techniques that are more complex and significantly more interesting. Today we will analyze one method of detecting an attached debugger, and a second method that can be used to detach a debugger from our running process.

Advanced API Based Anti-Debugging

There are a number of functions and API calls within the Windows operating system that are considered internal to the operating system and thus not documented well for the average developer. Many of these functions have undergone extensive research and reverse engineering to be able to understand how they operate and what can be achieved using them. One such poorly documented API function is the NtQueryInformationProcess function which is used to retrieve information about a target process. The function prototype looks like the following:

NTSTATUS WINAPI NtQueryInformationProcess(
    __in HANDLE ProcessHandle,
    __in PROCESSINFOCLASS ProcessInformationClass, 
    __out PVOID ProcessInformation,
    __in ULONG ProcessInformationLength,
    __out_opt PULONG ReturnLength
);

This function resides within the Ntdll.dll file and is not an exported function. Because of this, we must use run-time dynamic linking to gain access to the functionality. Run-time dynamic linking is the dynamic loading of a library and mapping of functions within the library to a function pointer allowing it to be called and executed. To load our function, "NtQueryInformationProcess", we first call LoadLibrary("ntdll.dll") and then execute GetProcAddress(HMODULE, "NtQueryInformationProcess") to receive a pointer to our required function.

HMODULE hmod;
FARPROC _NtQueryInformationProcess;
hmod = LoadLibrary(L"ntdll.dll");
_NtQueryInformationProcess = GetProcAddress(hmod, "NtQueryInformationProcess");

Note: The dynamic linking method is slightly different when using C++ due to declaration differences.

Once we have a function pointer to the NtQueryInformationProcess function, we can call the API. The function call takes five parameters, the first two of which are the most interesting to our anti-debugging efforts. The first parameter is a HANDLE to the target process that we wish to interrogate. Since we are trying to determine information about our own process, we will use a HANDLE that points to ourselves. By default, a HANDLE value of -1 will instruct the function to use the current process as the target. The second parameter is a value indicating what type of information is being requested from the target process. In the Microsoft MSDN documentation there are four documented values for this parameter ProcessBasicInformation (0), ProcessDebugPort (7), ProcessWow64Information (26), and ProcessImageFileName (27). There are other undocumented values that can be passed in, some of which allow for interesting anti-debugging techniques, however we will focus on the ProcessDebugPort (7) value. This value, when used as the second parameter in the NtQueryInformationProcess function will return a DWORD, returned in the address of the third parameter, indicating the DebugPort that is currently available for the process. If a non-zero value is returned, indicating that a DebugPort exists, we can be sure that a debugger is attached and act accordingly.

status = (_NtQueryInformationProcess) (-1, 0x07, &retVal, 4, NULL);

if (retVal != 0) {
    //Debugger Detected - Do Something Here
} else {
   //No Debugger Detected - Continue
}

The second anti-debugging method we will look at today also uses run-time dynamic linking of the Ntdll.dll library along with GetProcAddress() to gain access to the NtSetInformationThread function. This function's primary purpose is to modify thread specific data for a targeted thread.

NTSTATUS NTAPI NtSetInformationThread(
    __in HANDLE ThreadHandle, 
    __ in THREAD_INFORMATION_CLASS ThreadInformationClass,
    __in PVOID ThreadInformation
    __in ULONG ThreadInformationLength
)

For the anti-debugging use of this function we are again only interested in two particular parameters. The first parameter is an identifier to the thread we wish to target and the second parameter is the particular information we wish to modify on the target thread. To get a pointer to our current thread we will use a call to GetCurrentThread(). We will submit that as the first parameter and the enum value for ThreadHideFromDebugger, 0x11, as the second parameter. If a debugger is attached and we pass in 0x11 to NtSetInformationProcess, our process will immediately detach any attached debugger and terminate the process.

lib = LoadLibrary(L"ntdll.dll");
_NtSetInformationThread = GetProcAddress(lib, "NtSetInformationThread");

(_NtSetInformationThread) (GetCurrentThread(), 0x11, 0, 0);

MessageBox(NULL, L"Debugger Detached", L"Debugger Detached", MB_OK);

Many of the Microsoft API calls are intentionally not well documented to discourage their use/abuse. In this case we can make calls to two non-exported functions within the Ntdll.dll library to achieve our goals of detecting or detaching a debugger from our process. There are a number of other methods of API based anti-debugging, feel free to comment about them below.

Stay tuned for our next installment as we touch on process and thread block anti-debugging.