OpenRCE

About Articles Book Store Distributed RCE Downloads Event Calendar Forums Live Discussion Reference Library RSS Feeds Search Store Users What's New

Customize Theme

Flag: Tornado! Hurricane!

Created: Wednesday, August 6 2008 18:07.16 CDT

Printer Friendly ...

Part 1: Bytecode and IR

Author: RolfRolles

# Views: 4616

The approach I took with ReWolf's x86 Virtualizer is also applicable here, although a more sophisticated compiler is required. What follows is some preliminary notes on the design and implementation of such a component. These are not complete details on breaking the protection; I confess to having only looked at a few samples, and I am not sure which protection options were enabled.

As before, we begin by constructing a disassembler for the interpreter. This is immediately problematic, since the bytecode language is polymorphic. I have created an IDA plugin that automatically constructs OCaml source code for a bytecode disassembler. In a production-quality implementation, this should be implemented as a standalone component that returns a closure.

The generated disassembler, then, looks like this:

let disassemble bytearray index =

  match (bytearray.(index) land 0xff) with

    0x0  -> (VM__Handler0__PopIntoRegister(0),[index+1])

  | 0x1  -> (VM__Handler1__PushDwordFromRegister(0),[index+1])

  | 0x2  -> (VM__Handler2__AddWords,[index+1])

  | 0x3  -> (VM__Handler3__StoreByteIntoRegister(bytearray.(index+1)),[index+2])

  | 0x4  -> (VM__Handler0__PopIntoRegister(4),[index+1])

  | 0x5  -> (VM__Handler1__PushDwordFromRegister(4),[index+1])

  | 0x6  -> (VM__Handler4__ShrDword,[index+1])

  | 0x7  -> (VM__Handler5__ReadDword__FromStackSegment,[index+1])

  | ...  -> ...

Were we to work with the instructions individually in their natural granularity, depicted above, the bookkeeping on the semantics of each would likely prove tedious. For illustration, compare and contrast handlers #02 and #04. Both have the same basic pattern: pop two values (words vs. dwords), perform a binary operation (add vs. shr), push the result, then push the flags. The current representation of instructions does not express these, or any, similarities.

Handler #02:                Handler #04:

mov   ax, [ebp+0]           mov   eax, [ebp+0]     

sub   ebp, 2                mov   cl, [ebp+4]      

add   [ebp+4], ax           sub   ebp, 2           

pushf                       shr   eax, cl          

pop   dword ptr [ebp+0]     mov   [ebp+4], eax     

                            pushf                  

                            pop   dword ptr [ebp+0]

Therefore, we pull a standard compiler-writer's trick and translate the VMProtect instructions into a simpler, "intermediate" language (hereinafter "IR") which resembles the pseudocode snippets atop the handlers in part zero. Below is a fragment of that language's abstract syntax.

type size = B | W | D | Q

type temp = int * size

type seg  = Scratch | SS | FS | Regular

type irbinop  = Add | And | Shl | Shr | MakeQword

type irunop  = Neg | MakeByte | TakeHighDword | Flags

type irexpr = Reg of register 

            | Temp of int 

            | Const of const 

            | Deref of seg * irexpr * size 

            | Binop of irexpr * irbinop * irexpr 

            | Unop of irexpr * irunop

            

type ir = 

  DeclareTemps of temp list

| Assign of irexpr * irexpr

| Push of irexpr

| Pop of irexpr

| Return

A portion of the VMProtect -> IR translator follows; compare the translation for handlers #02 and #04.

let make_microcode = function

  VM__Handler0__PopIntoRegister(b)       -> [Pop(Deref(Scratch, Const(Dword(zero_extend_byte_dword(b land 0x3C))), D))]

| VM__Handler2__AddWords                 -> [DeclareTemps([(0, W);(1, W);(2, W)]);

                                             Pop(Temp(0));

                                             Pop(Temp(1));

                                             Assign(Temp(2), Binop(Temp(0), Add, Temp(1)));

                                             Push(Temp(2));

                                             Push(Unop(Temp(2), Flags))]

| VM__Handler4__ShrDword                 -> [DeclareTemps([(0, D);(1, W);(2, D)]);

                                             Pop(Temp(0));

                                             Pop(Temp(1));

                                             Assign(Temp(2), Binop(Temp(0), Shr, Temp(1)));

                                             Push(Temp(2));

                                             Push(Unop(Temp(2), Flags))]

| VM__Handler7__PushESP                  -> [Push(Reg(Esp))]

| VM__Handler23__WriteDwordIntoFSSegment -> [DeclareTemps([(0, D);(1, D)]);

                                             Pop(Temp(0));

                                             Pop(Temp(1));

                                             Assign(Deref(FS, Temp(0), D), Temp(1))]

| ...                                    -> ...

To summarize the process, below is a listing of VMProtect instructions, followed by the assembly code that is executed for each, and to the right is the IR translation.

VM__Handler1__PushDwordFromRegister 32



and   al, 3Ch ; al = 32

mov   edx, [edi+eax]

sub   ebp, 4

mov   [ebp+0], edx        Push (Deref (Scratch, Const (Dword 32l), D));



VM__Handler7__PushESP



mov   eax, ebp

sub   ebp, 4

mov   [ebp+0], eax        Push (Reg Esp);



VM__Handler0__PopIntoRegister 40



and   al, 3Ch

mov   edx, [ebp+0]

add   ebp, 4

mov   [edi+eax], edx      Pop (Deref (Scratch, Const (Dword 40l), D));



VM__Handler19__PushSignedByteAsDword (-1l)



movzx eax, byte ptr [esi] ; *esi = -1

sub   esi, 0FFFFFFFFh

cbw

cwde

sub   ebp, 4

mov   [ebp+0], eax        Push (Const (Dword (-1l))); 



VM__Handler9__PushDword 4525664l



mov   eax, [esi] ; *esi = 4525664l

add   esi, 4

sub   ebp, 4

mov   [ebp+0], eax        Push (Const (Dword 4525664l));



VM__Handler9__PushDword 4362952l};



mov   eax, [esi] ; *esi = 4362952l

add   esi, 4

sub   ebp, 4

mov   [ebp+0], eax        Push (Const (Dword 4362952l));



VM__Handler19__PushSignedByteAsDword 0l};



movzx eax, byte ptr [esi] ; *esi = 0

sub   esi, 0FFFFFFFFh

cbw

cwde

sub   ebp, 4

mov   [ebp+0], eax        Push (Const (Dword (0l))); 



VM__Handler42__ReadDwordFromFSSegment};



mov   eax, [ebp+0]        DeclareTemps([(0,D)]);  Pop (Temp 0);

mov   eax, fs:[eax]

mov   [ebp+0], eax        Push (Deref (FS, Temp 0, D));

Active in Last 5 Minutes
	halsten

There are 21,880 total registered users.

Recently Created Topics
Searching freelist[0...	Sep/05
How to fix this in o...	Sep/03
Trouble linking plug...	Sep/02
PyEmu error when cal...	Sep/02
Restore Themida/Winl...	Sep/02
Anti-olly technique	Aug/30
RAR Password	Aug/29
Heap protection on W...	Aug/23
Why Inline asm in C+...	Aug/20
Bypassing OllyAdvance	Aug/17

Recent Forum Posts
Trouble linking plug...	jduck
Trouble linking plug...	timtoady
Trouble linking plug...	jduck
Trouble linking plug...	timtoady
Trouble linking plug...	jduck
reverse engineering ...	raiden56
pydbg, memory breakp...	Researc...
RAR Password	Ineedhelp
RAR Password	cod
Heap protection on W...	voila

Recent Blog Entries
	waleedassar	Sep/06
svchost from A to zinc part4
	waleedassar	Sep/04
svchost from A to Zinc part3
	waleedassar	Sep/04
svchost from A to Zinc part2
	Mcstyle	Sep/03
Cheap Pegeout Partner Tepee...
	waleedassar	Sep/03
svchost from A to Zinc
More ...

Recent Blog Comments
	convik on:	Sep/04
Is it legal??
	djnemo on:	Sep/04
Gunpack (God's Unpacker) - ...
	frozenrain on:	Sep/02
Restore Themida/Winlicense ...
	tosanjay on:	Sep/02
PyEmu 0.0.2
	GynvaelColdwind on:	Sep/01
Is it legal??
More ...

Imagery

SoySauce Blueprint
Jun 6, 2008

[+] expand

View Gallery (11) / Submit