OpenRCE: Blog

radare2 porting hackaton

Sun, 12 Apr 2009 17:13:08 -0500

I'm going to invoke a hackaton on radare2 for the next sunday (2009-04-19).

Previous hackatons were organized without any defined objectives, but I
think we need to expose some points to do, to centralize the efforts and
stabilize the code for the release which is planned for 2009-04-25.

The main points for this hackaton (feel free to do whatever else ;) are:

- Porting radare2 to windows, freebsd, netbsd, openbsd, solaris
- Porting to sparc, mips, arm, powerpc, m68k
- Porting to other compilers (TCC, MSVC, MINGW32, ICC, Forte, ...)
- Fixing build issues for packaging on debian/gentoo/arch/nix/..
- Fixing gcc warnings

The work on these points can probably expose endian issues, makefile
design bugs, non-portable/standard C code, casting problems, etc..

---

Lot of work has been done these days without many mails in the
list, and if you are not following the commits this mail will give
you feedback about the current state.

elektranox is working on making radare1 to be included in the
Debian linux distribution. We are working together to fix the
lintian issues reported to avoid installation issues.

As I have already told, radare1 is currently in a frozen state. All
the current work is done in the radare2 repository and only
minor fixups and build issues are going to change in radare1.

New releases for both versions will be released next week during
the BS3C congress in Bilbo:

http://www.blindsec.com/jornadas/index.html

There will be two talks about radare. One about practical debugging
sessions and another about binary manipulation using r_bin, r_asm
and r_anal (this one by Nibble). They will be in spanish.

The talk I did a month ago in the Fiberparty (Barcelona) was recorded
and here's the media files to be downloaded (I would like to thank Graz
for the mirror).

http://mirror.radare.org/get/

There is also a wmv encoded clip, but the theora one is at best resolution.
This talk is in spanish too.

Nibble has been working in the analysis and binary modules of radare2
(r_anal and r_bin). The Java classfiles, ELF32/64 and PE32/64 binary formats
are now more stable and functional.

r_asm can now assemble multiple opcodes in a single line and it has been
integrated together with some basic directive commands (.byte , ..) in
rasm2 and radare2. You will have to quote the wa command to concatenate
the opcodes in a single line:

> "wa nop;mov eax,33;int 0x80"

The r_anal module now handles the ascii-art reflines at the left of the disasm
output using the 'pd' radare2 command.

We now have a new library called r_bininfo that implements an interface for
getting debugging information from binaries, The plugins will use libdwarf
and so on, to extract information from the r_io file.

From my side I am working on r_bp that interacts with r_debug and its plugins
to manage a generic breakpoint database independently from the debugger
backend, so we will be able to setup multiple breakpoints for software/hw,
for read-write-exec, for a certain length at a memory offset. And then move
the process from the dbg-ptrace backend to the dbg-qemu and keep the
breakpoints working. Breakpoint expressions will be also handled from the
same module.

The r_bp module will provide a plugin interface for extending software breakpoint
instructions for multiple architectures, endians, etc..

The r_asm module now rsolves an available assemble() plugin if the current plugin
used only support disassemble(). Each asm plugin now exports information
about the architecture string suported and the supported register sizes.

The Visual mode in radare2 is now more functional and usable, cursor usage can be
now used with the 'a' and 'w' commands to write assembled opcodes or hexpair
strings like in r1.

We now have some basic metadata support in radare2, so we can add comments.

I have added a basic binary tree implementation and it is currently used in r_flag
to accelerate the resolution of flags by offset.

configure script accepts --with-compiler and --with-ostype flags to choose compiler
profiles and force operating system. I have fixed some basic issues for the mingw32
porting.

There are more things done, but these are the most interesting ones.

Enjoy

radare 1.2 and libr refactoring

Mon, 26 Jan 2009 09:53:21 -0600

This is an stabilization release after the new year's 1.1 one. It fixes some
installation issues noticed by the gentoo maintainer.

Dont expect too much new features in this version, We (nibble and me) have been
spending most of the time working on 'libr'. The full refactoring of radare into
a set of libraries reducing LOCs, adding test cases, removing warnings, adding
language bindings and the vala api.

As always:

  http://radare.org/get/radare-1.2.tar.gz
  http://radare.org/get/changelog-1.2-short

We plan to make a hackaton for libr in about two weeks. I will ping the mailing
again :)

The reason for libr is that after two years of scratchy development we think the
base is well defined and now the task is a full rewrite from scratch trying to
refactor everything to keep the core simple and minimalistic but bypassing all the
limitations and ugly hacks we have faced during the development of the old 1.0 core.

In a month we have refactored most of the old code into 20 libraries which can be
easily detached from each other to use in your own projects. Expect some licensing
changes for this library..GPL is not designed for libraries and LGPL is probably a
good option but we have to discuss about it. Because the base should be Free.

We have managed to write some minimal implementations of the radare1 programs (rax,
rabin, rasm, radare...) as test programs inside libr (see src/libr fmi) they are
now 10-200 LOC =). Obviously they are incomplete and didnt support all the options
of the old ones, but this is just a question of time. radare2 will be built on top
of libr.

Our plan is to continue working in the refactoring to provide a simpler user interface
but at the same time providing more features in less lines of code. Some basic
optimizations has been done, but major ones will be done when finished.

Actually it's just a toy, but reusing the old code is pretty simple and the development
goes fast :) as always ;)

You can find some vala code examples in src/libr/vapi/t like this one:

--------------------------------------------------
using Radare;

public class AsmExample
{
        public static void main(string[] args)
        {
                Asm.State st = new Asm.State();
                st.set_arch( Asm.Arch.X86 );
                st.set_bits( 32 );
                st.set_big_endian( false );
                st.set_pc( 0x8048000 );

                uint8 *buf = "\xcd\x21";
                st.disasm(buf, 2);
                stdout.printf("%s\n", st.buf_asm);
        }
}
---------------------------------------------------

For 1.3 the plans are to have some core apps replaced by the new libr ones and get more
stuff done in libr with some more refined apis.

Current libraries are:

asm: assemble, disassemble, pseudodisassemble in multiple architectures
bin: parse elf32/64 and pe32 (pe64, mach0, class will be imported soon)
cmd: handles short and long radare commands
config: handles configuration hashtable
cons: multi-os access to console (ansi, html and plain text)
core: implements a radare instance
flags: handles offset marks
hash: checksumming, hashing and entropy calculations
io: wraps IO with backend plugins with support for maps, sections, undos and caches
lib: multi-os library for loading plugins
line: minimalistic multi-os replacement for readline
macro: the scripting macro engine of radare
print: multiple print modes (hexdumps, dates, formatted structures, etc..)
range: handle ranged list of elements
search: search engine with binary masks, strings (ascii, widechar), patterns, expanded aes, ...
socket: easy-to-use socket api
syscall: multi-os api for querying syscall information
util: cross-library utilities for strings, memory, numbers, mathematical expressions...
vm: standalone implementation of the radare vm for emulating any architecture

vm, search, cons and io are not yet stabilized at api level. If you want to follow the
development, please pull the source from mercurial

  hg clone http://radare.org/hg/radare

Plans for 2.0 are:
  - use r_lib as plugin loading api (plugins for all!)
  - parrot bindings (no more scripting limitations)
  - vala bindings (fearless graphical programming)
  - switchable debugging api (use ptrace, mach, bochs, internal vm, etc.. seamlessly)
  - distribute libr as binary modules for your projects.
  - r_core_thread for multithreaded capabilities
  - lot of test cases
  - stabilize api
  - free beer :)

Thanks to Nibble for the development and nopcode people for supporting my crazy talks
about radare scripting. mandingo for the betatesting and esteve for the pattern finding
algorithm fix :)

Enjoy

radare 1.1 - codename "pre2k9"

Wed, 31 Dec 2008 15:04:34 -0600

I'm glad to announce the release of radare 1.1, codenamed "pre2k9".

Here's the summary of the changelog from 1.0 to 1.1.

I hope this release fixes all the building problems noticed by the
rpm and portage maintainers. I have also added a large list of
new feautres, commands and bug fixing.

If you dont have anything to celebrate tonight (yeah today is the
longest year), enjoy the 12:60 second \o/

Together with the 1.1 source code release I am also publishing the
mp3 of the chiptune session performed during the release of radare 1.0.

Thanks to neuroflip for the music! Thanks to brainstorm, Nibble, Anderson,
and JFV for their contribs, ideas and opinions!

I wish you a great entrance for this new year :)

Source:
  http://radare.org/get/radare-1.1.tar.gz

Chiptune session of the 1.0 release:

  http://nopcode.org/get/neuroflip-radare1.0-sidparty-20081108.mp3

Changelog-1.1:

* New bindiff engine (bindiff-ng)
  - Handles elf, pe and multiple arches and diffs disassm or hexdumps
  - Supports context view like grep -C
  - Color support
* Support to import/export filedescriptor information for processes
  - Integrated with process dumping and restoring debugger commands
* Analyze ranges can now give inverted results (boolean negation of ranges)
  - Ascii art output (ar%)
  - Trace analysis can be tagged to support multiple traces to be splitted
    internally and use different colors for graphing and so on
  - Full support for range analysis manipulation ('ar' command)
  - Can import information from debugger traces
  - Traces can be displayed with dwarf (C code) or serialized assembly
  - trace.bps enables/disables the stop on breakpoints when tracing code
* Code analysis graphs can now be exported as .dot files for graphviz with 'ac.' command
  - / and * keys in grava window are used to rotate graph in Z axis.
  - u and U keys are undo/redo seeks in grava window
* cmd.visualbind binds the 'b' key in visual mode to run a user defined command
* Fix !stepu command (until user code and until address)
* TouchTrace debugger tracing method now supports ranges
  - Make dbg.regs2 show extended registers (segment registers on x86) same as !reg2
  - Port touchtrace to mips and arm architectures
* Import ERESI's gdbwrap module
  - Enables gdb-remote support to the radare debugger
  - Added helper for connecting to vmware gdb remote
  - gdbwrap://ip:port
* Added initial test suite for the ruby bindings
* Initial port to CH3S NAS ARM5 Linux
  - Fix support for get/set registers with aliased names (pc,lr,..)
* Sync x86-64 fpu code with the i386 one on linux debugger
  - Fix !mp bug (xwr->rwx)
* Added support for virtual and physical addresses
  - Replaces outdated file.baddr with io.vaddr, io.paddr
  - Globally and per section
  - Makes ARM-WCE bins be loaded nicely
* Much more work on libr
  - Added initial libr_syscall
  - Lot of work on r_io
  - Add some test cases
  - libr_bin now supports section resizing
  - Refactoring for r_bin
  - Sync with vaddr/paddr
  - Interface for upcomming r_search
  - Full r_cons and r_hash
* Support for function variables (arg, fastarg, local)
  - Handles tracking and automatic detection with code analysis
  - By default 'cmd.vprompt2'
  - Handle variable type definitions (Cv) to setup int, float, char*, ..
  - Handle variable access list (get/set) for each variable
* Added user-query command '<?' to ask values to the user
* Initial implementation of the virtual machine
  - Supports 8,16,32,64 size read/writes
  - Pseudo opcode por portable architecture code emulation
  - Supports conditional evaluable expressions
  - Can emulate text files pieces or real code for x86 and mips atm
  - Import information from/to the debugger
  - Support for cached IO to avoid child process memory alteration
  - Supports register aliases to define register dependencies like in x86
  - Allows to define new opcodes based on a list of evaluation strings
  - Real code is converted to evaluable strings using the PAS engine
* @@ iterator now supports file based (per line) foreachs with @@.filename
* Added dbg.contsc2 to stop every 2 syscall events or not
  - This can be used to choose to stop before or after executing a syscall
* Enhacements for the native console grep command syntax
  - Lot of console buffer handling optimizations
  - Now can highlight words with '*' keyword
  - Also available with scr.grephigh
  - Can grep lines and columns ~[0] ~#0
  - Fixes some segfaults
  - Support for negated greps ~!
  - Can count number of lines with ~?
* Access to opcode metadata with variables $$F, $$l, ...
* Added repeatable format with math expressions for 'pm'
  - pm {ecx/8}xx @ esi
* Added quotable commands (prefix with '"') to skip shell pipes
* New write operation (wo) command to endian swap 2,4,8 bytes a memory block
* Fix olly assembler and disassembler bugs
  - asm.case can show assembly in lowercase or uppercase
    (olly disasm is now lowercase by default)
  - Use 0x on olly disasm to avoid confusions
* Added support for binary mask for write operations
  - Defined as cyclic keys (wm ff00ff && wx 909090)
* Support for conditional macro expressions. For example:
  - Added '()' command to 'break' macros
  - Fix recursive limits to avoid stack underflows
  - Oneliners are now possible!
  > (for-fun$)`C*~CF[3]#$@$
  > (step-post-anal,?z`ao@oeip~type = cond, \
    ??(),?eip==`ao@oeip~jump =[2], \
    ??CC likely@oeip,??(),CC unlikely@oeip,)
* Symbols are now prefixed with <flagspace>.<flagname> instead of '_'
* Colorize jump target addresses when placing cursor on top of a branch op
* Bunch of minur fixups
  - New documentation for the book (added cheatsheet)
  - Fix function length detection
  - Fixups on othe x86 code analysis module related to stack and vars
  - Fix installation problems found in 1.0
  - Make !bp autocomplete flags with readline
  - Fix some more warnings
  - Fix build on osx (intel and ppc)
  - scr.seek is working again
  - asm.pseudo is working again too
  - Fix disassembler offsets on x86
  - Allows to enable/disable undo operations ('u' command)
  - Added asm.profile=gas to support gas reassembling of code
  - Fix build for xrefs
  - Added examples of macros and scripts
  - Refactoring on much random places
  - 'f' suffix for 'p' command is now universal (to setup full function size)
  - Fix ^D exit issue for lua, python and ruby plugins

* Much more ...

Another invalid opcode representation

Thu, 20 Nov 2008 08:46:24 -0600

After the release of radare 1.0 somebody noted a bug in the disassembler, so we made some investigation and saw that udis86 is representing the

  83 e4 f0 as  "and esp,0xf0"

and other disassemblers (olly, gnu objdump, ..) are representing it as:

  "and esp, 0xfffffff0"

The problem is not directly related to udis86, because it is a missrepresentation of what intel really does at low level with this instruction.

In the specs says that the 83 opcode should affect only the lowest byte of the register pointed by the second byte. If this operation is performed against EAX we can properly represent the instruction as "AND AL, 0xF0", but neither EBP or  ESP has partial access representations.

The funny thing is that without having access to a part of a register following the intel syntax the cpu is able to do it, so I understand that this is a bug in the representation for all the disassemblers.

Both ones are correct to me because they will act in the same way (maybe the olly,objdump) is more correct, but it does not matches the reality of the instruction.

radare 1.0 released

Sun, 09 Nov 2008 16:49:24 -0600

Download source:

  http://radare.nopcode.org/get/radare-1.0.tar.gz

Download the book:

  http://radare.nopcode.org/get/radare.pdf

Slides about >1.0 TODO

  http://radare.nopcode.org/get/rage1.pdf

Changelog from 1.0 beta to 1.0

* Make valac optional to build the GUI
* Added asm.profile for disassembler flavours
* Lot of bugs fixed (segfaults, typos, errors, ..)
* Release of the book
* Added serial:// port IO plugin
* Added macros support '(' command, '.(' to call them
* Initial ruby API (thanks brainstorm)
* Fixes for sparc, mips and arm disassemblers
* Refactorize of the disassembler engine
* Added assembly string parser
* Giving initial asm.pseudo for x86 and mips
* Stabilize for the osx debugger
* Plugins are now loaded properly in OSX (.dynlib)
* Lot of refactoring on rabin (thanks Nibble!)
* Make Cs detect multiple strings (nibble)
* Added asm.stackptr showing stack information in functions
* Dewarnification
* Fixes on build/install system (thanks Timo)
* Make bindiff patching work with radiff -r | radare -w ..
* Fix file.insert
* Rabin: dietpe fixes for get_strings
* Fix bug in bdcolor for offsets
* Deprecate asm.follow use scr.seek instead
* Build ruby with ruby (integrate with waf+make)
* Apply patches for dietmach0 and osx debugger (thanks revenge)
* Make ruby plugin have a global Radare() instance in $r variable
* Fixed build for mingw32
* xc renamed to 'rax'
* Eval variables are now accessible with '$((file.baddr))'
* Lot of refactoring and code cleanup for Rabin (thanks nibble)
* Implementation of io_maps allowing multiple file open in the same virtual space
* Added 'IRA' perl scripts PoC decompiler based on objdump
* Allows multiarch decompilation by pipelines (just PoC)
* Rabin drops '-a' flag (already included in -I)
* Fix getregs for powerpc-linux
* Make 'VG' (Visual GUI) happy again with gtk-prefs
* Added asm.section
* Fix store/restore asm. flags for graph
* Fix code analysis bug for arm
* Port the debugger to powerpc32-linux
* Added brainfuck debugger plugin with tracing facilities and virtual mapped IO
* Added pseudo-opcode disassembler of brainfuck
* Lot of fixups for the metadata handling commands
* Make rabin extract strings properly in multiple file formats
* Make the code analysis detect push-based data references
* Added $$j $$f $$r (jump fail and ref) virtual variables for scripting code analysis
* Fix debugger bug for OpenBSD and NetBSD
* Added html-based version of the book
* Syscalls properly commented in disassembly
* Make -s flag be -e dbg.bep for debugger
* Add data-struct using Cm following the pm command syntax
* Native MACH-0 parser for rabin named as dietmach0
* Fixes for displaying data structures with 'pm' command
* Fixes for scr.filter - now supporting '?' chars as masks
* pm command now can define field names in structures
* Fixes for .ad* command (data analysis metadata import)
* Fix section privs for dietPE (nibble)
* Added sS command to seek to a section
* Added support for size based pointer references [8:0x8043] 1,2,4,8,16,32,64
* Pointer data references are now using cfg.bigendian
* Added nice keybindings for the visual mode (graph view)
* New default color theme for grava engine
* Added minimize button to each graph node
* Fix BSD issues for dietelf
* Show rva and offset symbols for diet-PE (Nibble)
* Fix add_call for the graph
* Drop tabs in arm gnu disaassembler
* Nice ctrl-shift+wheel for disassembler
* Added cmd.vprompt2 and cmd.vprompt3
* Major refactoring for dietPE (nibble)
* Add waf --with-sysproxy
* Make section_list mark your position like in !maps?
* Folding code is usable again
* Added 'S' command to handle section information in core
* Allows multiple base address for virtual sections
* Import maps and sections information here
* Added arch.aop=objdump
* Rename 'R' to 'g' (graph management command)
* Added radare_cmdf for printf like radare command call
* Added seek command using code analysis information
* Seek to call, seek to xref or seek to data reference
* Added 'ua' (undo all) and 'ur' (undo reset)
* Added '?x' command to get value only in hexa
* Fix socket_reading bug for serial:// and socket://
* Added inmunity debugger python remote plugin
* Added 'fi' command to interpolate flag positions and get ranges (carving)
* Make -p -d work together (debugger+project information)
* Added '$$' for here and '$$$' for opcode size at current seek
* Added opcode based seeking operations
* Unify project information
* Added documentation for the ero's python-bochs integration for radare
* Added lot of new commands for the the python-bochs debugger
* Support fo rbinary masks from file-based searchs
* Fix x86-64 debugger bug
* Remote python-radare api renamed to 'radapy'
* Add 'dtdumper' to kradare (Used for syscall kernel introspection)
* Addded scr.pal. eval variables to change color attributes
* Fix 64 bit support for remote IO plugin
* Fix segmentation in malloc:// io plugin
* Make rasm -E create 4-char based enumerations
* Added data reference support for the x86 code analysis
* Make OSX syscall tables handle 0x80 and 0x81 interrupts (not yet integrated)
* Added better x86-stack related code analysis
* Add search.align to only get results based on a certain value