📚 OpenRCE is preserved as a read-only archive. Launched at RECon Montreal in 2005. Registration and posting are disabled.








Flag: Tornado! Hurricane!

 Forums >>  Brainstorms - General  >>  FLOW: Engineering vs. Reverse Engineering

Topic created on: July 6, 2005 07:23 CDT by JCRoberts .

Have you ever noticed the similarities between the engineering process and the reverse engineering process?

The reason I'm asking is simple: framework?

I get to do a lot of development work in the EDA tools world and work closely with the people engineering the latest and greatest hardware as well as all the muckety mucks selling EDA software tools. Whether EDA tool producer or consumer, everybody talks about "Flow" or more accurately the combination of tools and methods used to get the job of designing new hardware done in the most effective manner. In spite of the explicit abuse of the buzzword, flows are never really defined, documented or formalized and the required correct operation of interaction/interfacing/integration between various tools is basically a roll of the dice. It's a real mess because there is no solid way to manage the flow, know what tools/methods work with other tools/methods, know what has been done and know what still needs to be done.

I'm sure everyone here knows what it's like to have an idea and just start writing code. Some of the best programs start life that way but more often than not, you also waste a lot of time recoding a poorly thought out implementation. Documentation, if it ever gets written, is not much more than an after thought and it's seldom accurate. The short answer is you're making stuff up as you go along and it shows in the quality of your work. -Such is the model for poor software engineering and any one who can code is guilty of doing it at least once. ;-)

Reverse engineering often starts out just like the poor software development, with just an idea and a lot of determination. Often you get the one line dictate from the powers that be of "Can you make it work?" in the integration/compatibility field, or "Is it safe/secure?" in the vulnerability assement field, or "What damage has it done?" in the threat assement field.

Sure, there are how-to's, documented methodologies (ISECCOM/OSSTMM etc.) and similar things out there (both public and private) but as far as I know there really is no "Flow Management" program for the world of reverse engineering; a flexible framework of what should be done, how it should be done, along with the results of doing it and ways to pass such data from one part/tool in the flow to another. A place where you can load a default framework for doing a particular kind of work and then tweak it to include what ever you want.

Though my understanding of it is limited, I'll try to put it in terms of the related infosec reverse engineering fields; Have you ever been doing vunlerability analysis and forgotten something simple like failing to look for strcmp coding errors? race conditions? any of the other countless things that can be done wrong in coding?

Would it be nice to have a map that lets you know where you are, where you need to go and what you need to do to get there as well as a way to document your entire journey?

This is the reason why Ero's work on REML (Reverse Engineering Markup Language) sparked my interest. OK, I can admit I'm unabashedly biased in favor of Lisp S-Expressions over XML but that's besides the point. It seems Ero built REML mainly for interfacing with IDA. My thought is one step beyond; namely, defining tool/process/method flow, keeping documentation and providing a neutral interface between tools.

Maybe it's just a wild idea, but I think having the ability to organize your game plan and keeping the framework flexible enough to plug new tools/methods into the flow as they become available, seems like a lot better idea than keeping the whole thing in your head or in a static document and hoping you remembered everything.

If built in an agnostic fashion, a game plan manager could be used as the base for both engineering and reverse engineering. Do you think an open source game plan manager would be useful?

  BillyBoBob   July 6, 2005 09:18.31 CDT
Different people have different goals and difficulties, but for me juggling the amount of data and referencing the data afterword is a major problem. I generate thousands of little, somewhat independent looking facts and attempting to hold them togather is always a big problems, especially if the effort stretches into weeks. I use text files, IDA (heavy use of comments), heavy use of search (in my own stuff), scribbles on paper near me, 'keywords', etc. all in an attempt to hold it togather. Stuff always gets lost. OTOH, when you follow a piece of code and trip over your own comment from weeks ago, it is wonderful to see things connecting.

Methodology? Great idea, but I think it is mostly personal; I don't think standardized process offers that much in most cases (though certainly publishing your own can help other people). But organization of data would be nice. I am leaning towards voice annotation right now to add the ability to narrate stuff at speed, especially when single stepping through something where long pauses can't be done...but I know it won't work, cause I tried it before (admittedly probably 10 yrs ago)

  JCRoberts     July 6, 2005 10:25.50 CDT
Flexibility would is the key to a program of this sort. The goal is to *NOT* lock someone into a rigid set of predefined steps or methods. Though you could load a predefined "template" of sorts, there would be no requirement to follow it or leave it as is. You could modify it on the fly to suite your whims/needs and still have good place to store all your results, data and documentation. In this mode it would help you keep track of what you've already done and still give you the freedom to do whatever else you desire. If you've got your own personal way of doing things, you can save the flow framework and use it as a template for your next excursion.

If you've ever seen one of those outliner/thinker/planer/organizer programs, you've got the basic idea. Take that and add the ability to stuff predefined outline formats (flows) into it then modify the result, you have an idea of where I'm heading with all this.

JCR

  2GooD     July 7, 2005 08:56.07 CDT
Not actually process-oriented but a possible parallel to the difference between engineering and reverse engineering: One of my experiences from developing Desquirr was that decompilation is, somewhat simplified, just like compilation but with binary code input and source code output.

About a (reverse) engineering process framework: I suspect that one hard part is to make it both easy to adopt (to make people start using it) and powerful (so that advanced users do not feel limited by it).

But anyway, is this assumed to be a framework for a single person or for a team?

On the lighter side, maybe we should have "agile reverse engineering", inspired by http://www.agilealliance.com/? :-)

  JCRoberts     July 15, 2005 04:11.33 CDT

Well, I'm still tinkering with things over here and have traded a few emails/messages with Pedram and others. I've been doing a lot of reading on Operations Research, Management Science, Human Performance / Learning Modeling, Project Planing/Management and of couse flow/concept mapping... -I can't say it really accounts for anything one could mistake as progress but I'm getting a better idea of how to scope the whole thing.

If any of you have ideas for organization of tools and methods in reverse engineering or even forward engineering, please post them.

If necessity is the mother of invention, annoyance is the magnifier. If you can just name some annoyances that you have to deal with regularly, maybe even stuff you've written tools/plug-ins to solve, it would be helpful.

Thanks,
JCR

  nummish     July 15, 2005 09:04.42 CDT
I'm getting in here late and my google-fu is coming up short. As far as REML goes is there anything other than Ero's tools as far as a definition of it goes?

  ero     July 15, 2005 14:14.34 CDT
REML was created when I started writting the first versions of idb2reml (originally a monster IDC). It did grow to a level were I felt comfortable enough with releasing.

Although it would surely benefit from user feedback. I believe the format is pretty obvious. It's nearly a dump of the data presented by IDA. If suggestions are made probable it would make sense to update it (info on stack and structures sure would be nice) and define a XML Schema for it, so it would be standarized.

The format itself is more intended to be used through pyreml than anything else, just came as a brainfart in order to have something intermediate, but the advantage of having something like a XML IDB which can be accessible form outside IDA is something people would sure appreciate, mass-data mining anyone? :)

If anybody has specific questions about the format just shoot.

Note: Registration is required to post to the forums.

There are 31,328 total registered users.


Recently Created Topics
[help] Unpacking VMP...
Mar/12
Reverse Engineering ...
Jul/06
let 'IDAPython' impo...
Sep/24
set 'IDAPython' as t...
Sep/24
GuessType return une...
Sep/20
About retrieving the...
Sep/07
How to find specific...
Aug/15
How to get data depe...
Jul/07
Identify RVA data in...
May/06
Question about memor...
Dec/12


Recent Forum Posts
Finding the procedur...
rolEYder
Question about debbu...
rolEYder
Identify RVA data in...
sohlow
let 'IDAPython' impo...
sohlow
How to find specific...
hackgreti
Problem with ollydbg
sh3dow
How can I write olly...
sh3dow
New LoadMAP plugin v...
mefisto...
Intel pin in loaded ...
djnemo
OOP_RE tool available?
Bl4ckm4n


Recent Blog Entries
halsten
Mar/14
Breaking IonCUBE VM

oleavr
Oct/24
Anatomy of a code tracer

hasherezade
Sep/24
IAT Patcher - new tool for ...

oleavr
Aug/27
CryptoShark: code tracer ba...

oleavr
Jun/25
Build a debugger in 5 minutes

More ...


Recent Blog Comments
nieo on:
Mar/22
IAT Patcher - new tool for ...

djnemo on:
Nov/17
Kernel debugger vs user mod...

acel on:
Nov/14
Kernel debugger vs user mod...

pedram on:
Dec/21
frida.github.io: scriptable...

capadleman on:
Jun/19
Using NtCreateThreadEx for ...

More ...


Imagery
SoySauce Blueprint
Jun 6, 2008

[+] expand

View Gallery (11) / Submit