📚 OpenRCE is preserved as a read-only archive. Launched at RECon Montreal in 2005. Registration and posting are disabled.








Flag: Tornado! Hurricane!

 Forums >>  Brainstorms - General  >>  REML and Databases

Topic created on: July 28, 2005 06:03 CDT by stevem .

After discovering ero's cool REML (Reverse Engineering Meta Language) and associated tools, I got thinking about some applications for this handy way of representing reversed code. Specifically, I was wondering if anyone had considered the application of SQL databases to storing and analysing reversed code, or REML (or both)?

I guess the main advantage of using an SQL database is the ability to use a rich (mostly standard) language for querying the data and doing analysis. Scripts, tool-sets and even web front-ends could be written to analyse and present the data from the database.

Thinking further ahead, IDA can directly interface with these SQL databases through plug-ins. This should allow for better multi-user collaboration and provide an open methodology for analysis.

I've been piecing together some ideas for a schema and have had a breif discussion with ero about some ways of utilising REML, but I was hoping to find out if anyone else has considered this before?

  drew     July 28, 2005 15:02.54 CDT
Thinking further ahead, IDA can directly interface with these SQL databases through plug-ins. This should allow for better multi-user collaboration and provide an open methodology for analysis.

Have you looked at ida_sync?  Eventually ida_sync will most likely use XML for data transfer b/w the client and server.  Perhaps it will even use REML.

  stevem     July 28, 2005 15:49.26 CDT
Yep, the database concept I'm talking about would work _with_ the ida_sync "central server", as the central server could potentially be able to store the data in an SQL database (as well as IDB).

If you consider that some people use OllyDbg, ohers use WinDbg or whatever. Down the track, if plug-ins are written for these tools so that they can write to the same SQL Database that the IDA central server is using, then you can have people using their disassembler of choice, but all collaborating off the same database backend.

I know that this is thinking way far ahead, and I didn't want to focus too much on the collaboration side of things initially, as there's a lot of thought that needs to go into it. I wanted to focus more on providing a standard format for storing reversed code, and using an open platform with a flexible language for doing analysis.

  drew     July 28, 2005 16:41.06 CDT
If you consider that some people use OllyDbg, ohers use WinDbg or whatever. Down the track, if plug-ins are written for these tools so that they can write to the same SQL Database that the IDA central server is using, then you can have people using their disassembler of choice, but all collaborating off the same database backend.

For an example of this, there's a tool I wrote -- olly_sync.  It enables ollydbg to share info with the ida_sync central server.

What would you like to do with a SQL interface that you can't do with the current ida_sync protocol?

  stevem     July 28, 2005 18:39.36 CDT
For an example of this, there's a tool I wrote -- olly_sync.  It enables ollydbg to share info with the ida_sync central server.

Ah, ok, I wasn't aware of such a thing, that's awesome. Looks like the collaboration side of it is covered then. As an aside, I'm just browsing through the ida_sync code, and from what I can tell, only comments and names are stored in the ida_sync db. Am I right in saying this? The approach I was hoping to take was to store either the whole disassembly (from REML format), or "summary" information (call trees, etc.) for analysis.

What would you like to do with a SQL interface that you can't do with the current ida_sync protocol?

I guess the main advantage is providing an open, standard way to access, manipulate and query data, the rest is up to the users imagination (the "if we build it, they will come" approach :) There are a lot of data mining tools that operate on SQL databases, not to mention (for popular databases), APIs available in most languages.

  pedram     July 29, 2005 16:20.28 CDT
On the post Vegas agenda is adding features to this site and then re-working IDA Sync. There have been some issues associated with the current DB choice and more so with the "protocol". Communication will probably be worked around REML with storage still up in the air. Chris Eagle shared some great ideas with me on feature thoughts for IDA Sync and suggested looking into SQL Lite. I will do some research and poll interested parties for feedback through the site before starting any development.

Look for a new and improved syncing framework to be released at some point in the not-so-distant future.

  stevem     July 29, 2005 19:14.11 CDT
On the post Vegas agenda is adding features to this site and then re-working IDA Sync. There have been some issues associated with the current DB choice and more so with the "protocol". Communication will probably be worked around REML with storage still up in the air. Chris Eagle shared some great ideas with me on feature thoughts for IDA Sync and suggested looking into SQL Lite. I will do some research and poll interested parties for feedback through the site before starting any development.

I've got a semi-working schema design, which I've tested so far on MySQL (should be easy++ to convert to SQL Lite). This design however is for storing an entire disassembly, derived from a REML dump, so it might not be applicable to what IDA Sync is used for (only storing comments and names from what I can tell).

Drop me a line if you're interested in collaborating on this. In the meantime, I'm writing some scripts to parse REML and import into an SQL database, based on the schema I've got. I'm also happy to submit a patch for the existing IDA sync for storing the stuff in an SQL Lite database. Just let me know.

Note: Registration is required to post to the forums.

There are 31,328 total registered users.


Recently Created Topics
[help] Unpacking VMP...
Mar/12
Reverse Engineering ...
Jul/06
let 'IDAPython' impo...
Sep/24
set 'IDAPython' as t...
Sep/24
GuessType return une...
Sep/20
About retrieving the...
Sep/07
How to find specific...
Aug/15
How to get data depe...
Jul/07
Identify RVA data in...
May/06
Question about memor...
Dec/12


Recent Forum Posts
Finding the procedur...
rolEYder
Question about debbu...
rolEYder
Identify RVA data in...
sohlow
let 'IDAPython' impo...
sohlow
How to find specific...
hackgreti
Problem with ollydbg
sh3dow
How can I write olly...
sh3dow
New LoadMAP plugin v...
mefisto...
Intel pin in loaded ...
djnemo
OOP_RE tool available?
Bl4ckm4n


Recent Blog Entries
halsten
Mar/14
Breaking IonCUBE VM

oleavr
Oct/24
Anatomy of a code tracer

hasherezade
Sep/24
IAT Patcher - new tool for ...

oleavr
Aug/27
CryptoShark: code tracer ba...

oleavr
Jun/25
Build a debugger in 5 minutes

More ...


Recent Blog Comments
nieo on:
Mar/22
IAT Patcher - new tool for ...

djnemo on:
Nov/17
Kernel debugger vs user mod...

acel on:
Nov/14
Kernel debugger vs user mod...

pedram on:
Dec/21
frida.github.io: scriptable...

capadleman on:
Jun/19
Using NtCreateThreadEx for ...

More ...


Imagery
SoySauce Blueprint
Jun 6, 2008

[+] expand

View Gallery (11) / Submit