simulating memory allocation failures

Mon, 05 Jan 2009 14:14:36 -0600

I've written a debugger script that tests dlls for memory allocation failures by hooking ntdll.RtlAllocateHeap

full doc here:
http://www.bintest.com/m/malloc.html

API Fuzzing with PaiMei

Sat, 16 Dec 2006 06:54:32 -0600

I keep myself idle lately coding an API Fuzzer.

The idea is simple:
      -init:
          -load program
          -set hooks
          -load fuzz testcases in address space
      -fuzzing loop:
        on api call entry:
          first pass
            -save process snapshot ( thread context, memory)
            -get arguments
            -monitor mem access
        on api call return
            -restore memory
            -restore thread context (=> eip on api entry)
            -fuzz api call arguments

e.g.:

main.cpp:
---------



extern "C" __declspec(dllimport) void myfun ( int * a,int b,int c);



void main(void)

{

 int a = 6;

 myfun(&a,7,10);

}

dll.cpp:
--------



#include <iostream>

using namespace std;

extern "C" __declspec(dllexport) void myfun(int * a,int b, int c)

{

  cout << "a: " << *a << " | b: " << b << " | c: " << c << "\n";

}

compile and link:



$ cl -LD dll.cpp && cl main.cpp /LINK dll.lib



$ ./main

a: 6 | b: 7 | c: 10

this is the testcase:
hook myfun when called from main and fuzz the arguments...



$ python fuzztest.py | grep a:  # grep to ommit debug prints

a: 6 | b: 7 | c: 10

a: 6 | b: 256 | c: 32

a: 6 | b: 4096 | c: 0

a: 6 | b: 64 | c: 16

great... as b and c are integers they are fuzzed in the 3 fuzzing loops.
To get some more interesting results we have to fuzz all the arguments, i.e. pointers. This appears to be more difficult than expected: I have to monitor which memory locations are
accessed, then fuzz those locations. I thought putting memory breakpoints on the stack would help do the trick but pydbg doesn't support stack memory breakpoints. I'm trying to work around this by setting PAGE_NOACCESS on the stack pages and an access violation callback but this isn't finished yet. I do have a start for a wxpython gui:

OpenRCE: Blog

simulating memory allocation failures

API Fuzzing with PaiMei