Skip to content

Fuzzing the Windows API for AV Evasion

What is emulation?

Malware Detection Systems (MDSs) use a technique called emulation as perhaps their most effective weapon against novel malware threats. Emulation does not rely on the static structure or signature of a file, but instead, executes the suspicious file for you. Of course, the MDS will not run the file on your computer, but rather in some artificial environment that emulates a real operating system. Based on heuristic analysis of the execution, the MDS will conclude whether the behavior of the file was benign or malicious and will issue a detection accordingly. MDS emulation environments have come a long way in the past decade, but are still plagued by an axiomatic fallacy ripe for exploitation.

Emulation is Fundamentally Flawed

Emulation is just that — an imperfect replica of a legitimate operating system (OS). For every novel malicious file to be analyzed, there must be an accurate, complete, and unique instance of the OS running in a virtual environment. The program must then be executed in full, meaning that all possible conditional branches must be taken, and every possible outcome variant logged and analyzed. Many emulated environments implemented by MDSs are well constructed and adept at masking their shortcomings. But no MDS implements a carbon copy replica of each Windows OS. Bypassing MDS emulators is nothing new and utilizes many strategies aimed at tricking or exhausting the emulator. Common techniques involve poking around for environmental artifacts present on a real operating system that are not correctly implemented in the emulated one. Other strategies include placing malicious code behind very computationally intensive cycles, which must be emulated for malicious code to be executed. Emulated environments are orders of magnitude slower than an OS running on bare-metal. I am proposing a generic and completely blackbox approach to automating finding of such artifacts.

Conceptualizing the Attack

On my Windows 10 machine, I searched my System directory and found over 4,000 DLL files. Many of these DLL files make up the Windows API, the set of functions that expose the functionality of Windows for programs to use. Each of these DLLs contain functions with implementation and behavior that can differ with OS version. The premise is as follows. Since no emulator, with the real-world limitations of a consumer environment, can be as complex as the real deal, one could, distinguish between emulated and non-emulated functions by analyzing the artifacts from the unique behavior of each function.

The Windows API follows the stdcall calling convention, where the function called is not responsible for cleaning up the EAX, ECX, and EDX registers. These leftover values can be anything. EAX will most often contain return values in the case of successful execution, or error codes in the case of failure. However, ECX and EDX are undocumented, and may contain anything: internal state information, error codes, module/function addresses, etc. Before continuing, it is important to acknowledge previous research done by the VX community and threat researchers. Importantly, let’s not forget that AV companies have been aware of this for over 10 years and still have not fixed this type of vulnerability in their emulators.

Previous Research

The concept of attacking emulators using register artifacts is not new and in fact, is well over a decade old. Throughout this project, I reference famous VX’er SPTH (Second Part To Hell) and his work entitled Dynamic Anti-Emulation using Blackbox Analysis, which can be found in full here. In addition, there are two VirusBulletin articles that contain pertinent information. The first, Is our viruses learning?, authored by the distinguished VX researcher Peter Ferrie describes a write up of what seems to be SPTH’s technique in the wild (although the payload is messagebox). The second is, Okay, so you are a Win32 emulator…, by Gabor Szappanos, which analyzes in-the-wild malware using a number of attacks against emulators, which are in concept, still used successfully nearly a decade later.

SPTH’s research formed the basis of my interest, but the article and code provided are proof-of-concept level. Therefore, I decided to improve this base and release a tool that can help automate the gathering of register artifacts.

Attack Description

This attack follows very closely what was outlined in SPTH’s article. Here’s how it works.

Finding Arbitrary Parameter Count

First, load a module and walk the exported (by name only here) functions, compiling a list of names, in memory addresses, and position relative to the imagebase. Once all function addresses are found, one must know the appropriate number of parameters it takes. I spent considerable time trying to come up with alternative solutions to the one SPTH described, including using disassembler engines, trying to see which parameters were accessed in memory by protecting with PAGE_GUARD, and even scraping MSDN. I saw some promise with disassembly, but it was very inconsistent and proved to be too complex to implement in a blackbox approach. Here is the solution SPTH described that I also used. (Pseudo-code explanation).

DWORD espRestore;

__asm {
	mov espRestore, esp
	push 0
	push 0
	push 0
	push 0
	push 0
	push 0
	push 0
	push 0
	push 0
	push 0
	push 0
	push 0
	push 0
	push 0
	push 0
	push 0
	mov [fn->esp_1], esp
	call eax
	mov [fn->esp_2], esp
	mov esp, espRestore
}

fn->argLength = (fn->esp_2 - fn->esp_1) / 4;

Essentially, save esp, push 16 arguments onto the stack, preserve esp, call arbitrary function, preserve esp directly after call, and then restore the original esp value. Subtract the esp value saved from after the function executes by the esp value from before. Finally, divide by four (32-bit word size). This works, about 75% of the time on any arbitrary function, because, inevitably, calling an arbitrary function with improper parameters and stack will cause some pretty nasty exceptions. So, before we can find out the number of parameters an arbitrary function takes, we have to setup some vectored exception handling.

Exception Handling

So, right now, we’ve got the functions we want to call, and a way to find out the number of parameters that each one takes. But in doing so, we have to call the function, blindly, and with an improper stack. We need some serious exception handling, enter Vectored Exception Handling (VEH). Structured exception handling (SEH) does not provide the level of control we need to accomplish our goals here, specifically, because vectored handlers aren’t tied to a specific function nor are they tied to a stack frame and they are called before SEH. The entire point of setting up this error handling is to be able to fuzz these functions in such a way as to not have to restart the program (i.e. continuously). In order to do this, one cannot assume anything about the state of the exception. Instead, upon handling of the exception, EIP must be pointed to a safe termination procedure and execution continued. I approached this by allocating each function a thread, performing the totally illegal behavior, and if an exception is raised, pointing EIP to a function which exits the thread. This allows for handling of the most extreme exceptions. One could also save the registers from the exception context record.

VOID __stdcall ExitThreadProc() {
	PVOID pExitThreadProc =
		GetProcAddress(GetModuleHandle(TEXT("kernel32.dll")), "ExitThread");
	__asm {
		push 0
		call pExitThreadProc
	}
}

LONG __stdcall VectoredHandler(_EXCEPTION_POINTERS* ExceptionInfo)
{
	PVOID pExitThreadProc =
		GetProcAddress(GetModuleHandle(TEXT("kernel32.dll")), "ExitThread");

	ExceptionInfo->ContextRecord->Eip = (DWORD)&ExitThreadProc;

	return EXCEPTION_CONTINUE_EXECUTION;
}

Function Fuzzing

Okay, so now we can execute some really dangerous and illegal code, relatively safely. We combine the previous two tactics in order to find the parameter count of an arbitrary function — blackbox style! Knowing the legal parameter count, it’s time to look for execution artifacts leftover in the registers not cleaned up by the __stdcall functions. We can call the function in question in almost the same way as we did to obtain the parameter count. There are a few differences. One, the parameter count should be legal. Second, the parameter values are not zero, but rather random parameters. And last, we want to save the values of the EAX, ECX, and EDX registers for analysis. In order to provide a somewhat consistent approach for analysis, we want at least two calls to the function with different random parameters. Since some of these values could be related to the imagebase, some amount of human interaction is required to pick a “good candidate” for anti-emulation purposes. This is something I plan to look into further. Run the function twice, with unique random parameters. Continue to the next function if an exception is found, otherwise, record and compare register values. If they are equal — congratulations you just found a newly discovered bypass for every commercial AV emulator!

Results & Source Code

To demonstrate the efficacy of this technique let’s run an experiment.

Pick an infrequently used DLL, in this case I will be using iphlpapi.dll. Run my tool on it twice, and compare the output. In this case I found over a hundred artifacts on this one dll. Let’s try DeletePersistentTcpPortReservation, a function I’m sure none of you are familiar with. That’s the point, there are tons of obscure functions that don’t get attention from emulators because they are not “watched” functions. Here’s the found artifact.

DWORD _eax = 0;
DeletePersistentTcpPortReservation(0x910234, 0x123849);

// EAX = 5
__asm {	
     mov _eax, eax
}

if (_eax != 5) {
     ExitProcess(0);
}

I’m testing this technique, simply by creating an EXE which drops the Eicar test file. Of course, there are static detections on this file because AVs are awful at accurately differentiating between small, benign programs, and packed stubs.

Scan No Bypass

Scan AV Bypass (Anti-Emulation Inserted)

Disregard all the AI/ML and generic detections, because they are just detecting the file structure. Keep in mind that these are false positives, as there is nothing malicious about this file. Pay attention to the EICAR detections from: Avast, AVG, DrWeb, Kaspersky, VBA32, VIPRE, and Check Point’s ZoneAlarm. What these detections mean, is that the emulator has successfully emulated the code and observed that the EICAR test file has been written to disk. Now, observe in the bypass scan, there is not a single instance of an “EICAR” detection — in other words, no malware detection system correctly emulated this automatically generated emulation bypass.

The source code is available on my Github.

Some final thoughts… Please don’t search for artifacts on your actual machine. Blindly calling Windows API functions with both null and random parameters carries an infinitesimally small, but very real chance of causing damage to your OS. So, please run this in a virtual machine!

The code does not address some concerns such as ASLR and different Windows versions, but these seem to be straightforward fixes (i.e. future releases).