Flag: Tornado! Hurricane!


Article Abstract In this article I will cover the basics of reverse-engineering binary file formats, starting with the simple archive format used in the popular game EverQuest II known as VPK. This is the first of many articles I'll be writing on the subject as time goes on. You can find VPK files in any EverQuest II installation, and any will do for the purposes of this article. For the purposes of this article, I will assume the reader has knowledge of programming in C. The architecture being used is x86, and thus, everything is unsigned and little-endian. The tools you will need to follow along with this article are a decent hex editor (I recommend "HexEditor" for Mac OS X) and a copy of Python.

Full Article ...    Printer Friendly ...

Article Comments
drew Posted: Tuesday, September 6 2005 18:45.26 CDT
For file format RE, my favorite hex editor is Hex Workshop:
http://www.hexworkshop.com/

It has tons of features for REing file formats.  Simply highlight some bytes and it will instantly tell you info about the bytes.  It lets you easily calculate checksums of all sorts, int/uint values, digests, byte lengths, etc.

However Hex Workshop won't let the user comment locations of the file, nor does it have any features to deal directly with compressed data.

0xebfe Posted: Tuesday, September 6 2005 19:57.01 CDT
So basically, generically speaking, you looked for zlib blocks, and length:data like structures. Cool.. do post any attempts at structured network protocol reversing..

hoglund Posted: Monday, October 10 2005 01:12.42 CDT
What is your favorite feature of HexWorkshop?

luis Posted: Monday, October 10 2005 22:51.38 CDT
Hi hoglund. Personally my favorite feature of HexWorkshop is the structure editor. It is very useful in reversing file format, as it also serves as your notes. HexWorshop includes some samples like pkzip and others.

I do wish it had some kind of api or scripting mechanism such as IDA.

rfreeman Posted: Wednesday, October 12 2005 17:42.19 CDT
Two I have used extensively:

1) The CopyAs function. Output options include C array.
2) Resyncronizing compare.

drew Posted: Thursday, October 13 2005 03:47.55 CDT
The ability to highlight a chunk of bytes and quickly get info on them is simple but wonderful.  Being able to quickly see the length, casted values, CRC, MD5, etc of a chunk of bytes definitely speeds up my file format reversing.

drew Posted: Friday, January 20 2006 08:54.49 CST
Someone mentioned to me that the 010 editor was also good for file format RE:
http://www.sweetscape.com/010editor/

klondikemarlen Posted: Thursday, September 30 2010 20:12.57 CDT
Would the following code do the same thing with bzip2 as your code did with zlib?

import bz2, sys

def main(args):
    f = file(args[0], 'rb').read()
    size = len(f)

    i = 0
    while i < size:
        try:
            de = bz2.decompress(f[i:])
        except IOError:
            i += 1
            continue

        print 'Found a compressed block starting at', i

        i += 1

if __name__=='__main__':
    sys.exit(main(sys.argv[1:]))



Donner2011 Posted: Wednesday, December 21 2011 04:08.22 CST
This was precisely the answers I��d been searching for. Amazing blog. Incredibly maternity wedding dresses
Pregnancy wedding dresses
maternity dresses for weddings
chiffon maternity wedding dresses
short maternity wedding dresses
plus size maternity wedding dresses
christmas costumes inspirational! Your posts are so helpful and detailed. The links you feature are also very useful too. Thanks a lot

wedikes Posted: Thursday, November 7 2013 02:46.26 CST
It is interesting article and what about RE any binary file? That is, given a binary file which we do not have any clue using what tool it has been created, how we could reverse engineer it?


Add New Comment
Comment:










There are 31,313 total registered users.


Recently Created Topics
[help] Unpacking VMP...
Mar/12
Reverse Engineering ...
Jul/06
hi!
Jul/01
let 'IDAPython' impo...
Sep/24
set 'IDAPython' as t...
Sep/24
GuessType return une...
Sep/20
About retrieving the...
Sep/07
How to find specific...
Aug/15
How to get data depe...
Jul/07
Identify RVA data in...
May/06


Recent Forum Posts
Finding the procedur...
rolEYder
Question about debbu...
rolEYder
Identify RVA data in...
sohlow
let 'IDAPython' impo...
sohlow
How to find specific...
hackgreti
Problem with ollydbg
sh3dow
How can I write olly...
sh3dow
New LoadMAP plugin v...
mefisto...
Intel pin in loaded ...
djnemo
OOP_RE tool available?
Bl4ckm4n


Recent Blog Entries
halsten
Mar/14
Breaking IonCUBE VM

oleavr
Oct/24
Anatomy of a code tracer

hasherezade
Sep/24
IAT Patcher - new tool for ...

oleavr
Aug/27
CryptoShark: code tracer ba...

oleavr
Jun/25
Build a debugger in 5 minutes

More ...


Recent Blog Comments
nieo on:
Mar/22
IAT Patcher - new tool for ...

djnemo on:
Nov/17
Kernel debugger vs user mod...

acel on:
Nov/14
Kernel debugger vs user mod...

pedram on:
Dec/21
frida.github.io: scriptable...

capadleman on:
Jun/19
Using NtCreateThreadEx for ...

More ...


Imagery
SoySauce Blueprint
Jun 6, 2008

[+] expand

View Gallery (11) / Submit