Flag: Tornado! Hurricane!

 Forums >>  Brainstorms - General  >>  Searching for Entropy Tool

Topic created on: May 11, 2007 18:37 CDT by l0re .

I'm currently searching for a tool that does an entropy analyse. I want it to use it for finding a RSA key in a binary file. I have seen a tool that could do this on a workshop but unfortunately I don't know the name of tool and I can't find it with help of google. Does any one know the name of the tool or a tool that could do this?

  anonymouse     May 12, 2007 01:46.19 CDT
i may be too much wrong here
if i had got the meaning of entropy right then i think i saw Peid by j-q-s doing some such stuff
calculate the entropy of pe file
may be you could check that out and see if thats what you are looking for

peid.has.it

  l0re     May 12, 2007 06:15.51 CDT
I have looked at Peid but it didn't work on my file. I need it to work any binary file, not just PC files.

  ero     May 12, 2007 08:36.37 CDT
If you are into Python you might find this post helpful

  gera     May 12, 2007 09:24.49 CDT
I'm not sure if it's going to be good for what you want, but compressing is a good way of measuring the entropy of a stream of bytes... gzip then will be a nice tool.

On the other side, if, for example, the RSA key is stored as a number (ascii digits), I guess the entropy level of that is going to be low. The encoding of the key does make a difference.

  frankboldewin     May 12, 2007 16:07.17 CDT
a good one is also:

http://hellspawn.nm.ru/works/die_0.64.zip

  l0re     May 12, 2007 17:22.08 CDT
> ero: If you are into Python you might find this post helpful

Very interesting post. But I would need Mathematica to get it visually also?

  l0re     May 12, 2007 17:24.52 CDT
> frankboldewin: a good one is also:
>
> hellspawn.nm.ru/works/die_0.64.zip

Unfortunately this program says the same thing as Peid. That my file is not a valid PE file.

  ero     May 12, 2007 18:01.08 CDT
> l0re: > ero: If you are into Python you might find this post helpful
>
> Very interesting post. But I would need Mathematica to get it visually also?

Nope, you could use something like SciPy or Matplotlib instead

  frankboldewin     May 12, 2007 18:14.39 CDT
> Unfortunately this program says the same thing as Peid. That my file is not a valid PE file.

yep, but if you click on the "entro" tab and then click the "entropy" button, you should get information, if it is packed or not.

lot's of malwares i've analyzed in the last months use proprietary polymorphic and packer engines.

also read this websense blog posting:

http://www.websense.com/securitylabs/blog/blog.php?BlogID=123

  l0re     May 12, 2007 18:31.56 CDT
> yep, but if you click on the \"entro\" tab and then click the \"entropy\" button, you should get information, if it is packed or not.

I don't get it to work. It says "No file loaded"

  frankboldewin     May 12, 2007 19:44.01 CDT
can you upload the file to rapidshare or something similar?

  anonymouse     May 13, 2007 05:20.36 CDT
when you are on die_0.64 you could also check out entro.zip

http://www.practicalotp.com/trng.htm

  cyphunk   May 13, 2007 13:42.18 CDT
wrote a wrapper to Ero's code (thanks ero).  lets user set entropy threshold, prints block sample when passed, provides progress bar for processing of large files.  uses matplotlib.  would like to add total consecutive block size, mime/magic type check, compression check... but i'm hungry.

  l0re     May 14, 2007 16:40.56 CDT
Thanks cyphunk for the wrapper and thanks to ero for the original code! This will let me do what I want.

  MohammadHosein     May 15, 2007 13:04.51 CDT
this may sound very irrelevant , but couldnt resist to ask and see if any of crypto experts out there shed a little light on this for me or not

lets say we have huge amount of network traffic dumped somewhere and regardless of packet headers , protocols , etc would it be possible to use similar techniques to find [probably?] encrypted payloads even locate them inside dumps?

  jester   May 25, 2007 06:27.44 CDT
Have you looked at FTimes?   http://ftimes.sourceforge.net/FTimes/index.shtml
FTimes with XMagic has the ability to do entropy calculations on data and then you can use gnuplot to graph those results.  It can do 1-bye and 2-byte entropy calculations.
http://ftimes.sourceforge.net/FTimes/XMagic.shtml

  phantal     June 29, 2007 13:00.13 CDT
  I originally posted this on nzight, but I figured others that may not watch that blog may want to use it as well:

(snip)

  Sorry, there were problems inherent with the logic of that algorithm.

-Brian

  phantal     June 29, 2007 15:43.58 CDT
...

  frankboldewin     June 30, 2007 05:14.02 CDT
if you search wanna auto find ssl keys/certs, this tool might be of use:

http://www.trapkit.de/research/sslkeyfinder/index.html

or just give tobias process dumper/memory parser a try:

http://www.trapkit.de/research/forensic/mmp/index.html

  phantal     July 5, 2007 10:54.49 CDT
  I finally got some time to try implementing that algorithm I described & found a flaw in my logic: c1^c1 * c2^c2 * ... * ck^ck is a number much larger than 32-bits.  Fortunately I came up with a fast way to do SE:


for (i = 0; i < 256; i++)
  ek[i] = i/256 * Log(2,i); // pre-calculate the entropy for each possible probability

for (i = 0; i < windowsize; i++)
  ck[buffer[i]]++; // count how many of each byte there is in the window

for (i = 0; i < 256; i++)
  entropy[0] -= ek[ck[buffer[i]]]; // calculate the entropy for the first window position

for (i = 1; i < buffer.length - wsize; i++)
{               // slide the window position and update the new entropy as it slides.
  index1 = i-1;
  index2 = i+windowsize-1;

  entropy[i] = entropy[i-1] + ek[ck[buffer[index1]]] + ek[ck[buffer[index2]]];
  index1 = ck[buffer[index1]]--;
  index2 = ck[buffer[index2]]++;
  entropy[i] -= ek[index1] + ek[index2];
}

  carmaa     February 12, 2008 06:59.01 CST
>
> for (i = 0; i < 256; i++)
>   ek[i] = i/256 * Log(2,i); // pre-calculate the entropy for each possible probability
>


..just curious, how do you define log2 of 0? That's nan afaik...

  ero     February 12, 2008 07:37.29 CST
Given than the (i/256) it's supposed to mean the probability of "i" over the 256 possible cases, it's usually assumed that i/256 * Log(2,i) is zero in the case that i is zero. So one does not need to bother with Log(0). There's an "if(i==0) continue;" missing in that loop.

Note: Registration is required to post to the forums.

There are 31,117 total registered users.


Recently Created Topics
Information on the t...
Feb/08
Information on the m...
Feb/07
Order Finax, Fincar ...
Feb/07
Information on the m...
Feb/07
Order Proscar (Finas...
Feb/07
Order Proscar, Finax...
Feb/07
Order Finasteride, F...
Feb/07
How to view IDA Pro'...
Nov/02
reverse MC9S12DG128
Oct/07
Looking for an advan...
Mar/21


Recent Forum Posts
Looking for an advan...
tthtlc
Looking for an advan...
tthtlc
Looking for an advan...
clightning
Identify RVA data in...
sohlow
let 'IDAPython' impo...
sohlow
How to find specific...
hackgreti
Problem with ollydbg
sh3dow
How can I write olly...
sh3dow
New LoadMAP plugin v...
mefisto...
Intel pin in loaded ...
djnemo


Recent Blog Entries
nieo
Mar/22
Android Application Reversing

halsten
Mar/14
Breaking IonCUBE VM

oleavr
Oct/24
Anatomy of a code tracer

hasherezade
Sep/24
IAT Patcher - new tool for ...

oleavr
Aug/27
CryptoShark: code tracer ba...

More ...


Recent Blog Comments
nieo on:
Mar/22
IAT Patcher - new tool for ...

djnemo on:
Nov/17
Kernel debugger vs user mod...

acel on:
Nov/14
Kernel debugger vs user mod...

pedram on:
Dec/21
frida.github.io: scriptable...

capadleman on:
Jun/19
Using NtCreateThreadEx for ...

More ...


Imagery
SoySauce Blueprint
Jun 6, 2008

[+] expand

View Gallery (11) / Submit