OpenREF

Tue, 13 Sep 2005 04:57:36 -0500

I want to take some time to talk about a project which is currently taking up all my time aside from work - OpenREF.  OpenREF is the Open Reverse-Engineering Framework.

The goal of OpenREF is simple: Build a framework which can be used to develop all sorts of tools for analyzing binaries.
Another less general statement is that OpenREF will allow you to build applications like IDA Pro as well as standalone tools and everything in between.

OpenREF is being designed to be fully threadsafe and open-ended.  Far too often you run into issues where the design can only scale to a certain point before you run out of ways to extend that, and the object model of OpenREF should solve these issues.

What follows is a simple explanation of how you would use OpenREF.

First you create an ORDatabase object.  After this, you pass the database to your file loader, which populates the database with file segments.  Next you run your processor module on the segments you wish you analyze.  After this you might run a flow analysis module on the instruction objects generated by the processor module.  Finally, you might run a decompiler on the total output.

The structure of the system is that all object types extend ORObject.  I'm defining base objects like ORDatabase, ORSegment, ORInstruction, ORFunction, ORJump, ORConditional, etc that should be used, but the power is that you can define your own objects and use them how you like.

OpenREF will most certainly be released LGPL, and I'm hoping to have it at a point where loaders and such can start to be written within the next month or two.

If you'd like to ask any questions or have any feedback, shoot me a message, post on the forum, or get in touch with me in the IRC channel.

Thanks,
Cody Brocious

ArchShadow

Sun, 28 Aug 2005 01:48:11 -0500

Well, I'm now in the process of rewriting a large bit of the core of my decompiler ArchShadow, now nearing its 3rd year of development (though not on the same codebase)

ArchShadow originally started as a proof of concept decompiler that was fully standalone.  It was written in PHP (please don't martyr me for that) and used its own, very poorly written, disassembler.  It actually could decompile a number of large test binaries, but it was gcc-specific and very specific to certain ways of using constructs.  It was a hack, by any definition of the word.

From there it grew into a pure C project, still implementing its own disassembler.  This implementation didn't last long, as it simply wasn't worth the hastle for the limited return.  It did make the disassembler core a decent bit cleaner, though.

After that, I worked on an implementation in Python, still using my own disassembler.  This lasted for a while and let me get a lot of the SSA theory down. Eventually it was elimited when the analysis work on the disassembler side (especially function detection) got to be too big to handle.

All of this taught me a big, very important issue.  If the option to use an existing disassembler is there, USE IT.

ArchShadow is now in Python, sitting on top of IDA.  It reads in a good bit of information from the DB and caches it allowing you to run it away from IDA as long as you don't change the code to the point that the information read from the DB is different.  I'll eventually make it pull the entire IDA DB so that that's not such a big deal, but that's going to be a while in coming.  The current version works good enough for now.

The problem with the current revision is that my SSA support for variables (used for detecting the modifications to different things over the course of a given function) is simply poor.  It works, but to change names from the SSA name (var_#) I have to do a string replace which simply feels like a hack.  I'm going to address this in my partial rewrite.

Anyway, enough history.

I'm considering building a system that can export data from an IDA database and then be used in an external interface.  The main reason being that the interface for IDA on Linux is very poor, and there's absolutely no way for me to run any sort of IDA interface natively on OS X as it stands.  One other option is writing a network layer where tvision would currently stand in the linux version and building a GUI that works with that, but I'm not sure of what is exposed ot tvision.

Well, enough blogging for now.  Let me know if you have any ideas/questions/comments.

Take care.
Cody Brocious

PS.  Stop by #openrce on irc.freenode.net if you're so inclined.  I'd love to start a nice sized channel for reverse-engineering of all sorts, and I think it's easily doable if I can just get the word out a bit :)

OpenRCE: Blog

OpenREF

ArchShadow