Topic created on: January 16, 2008 05:46 CST by softworm .

I hope to view some obfuscated codes with IDA's graph,they didn't reside in any function,so i tried to write a plugin with IDA 5.2 SDK(with some modification to SDK's sample) to create a custom view.

My problem is, I have to limit the number of basic block,it works fine below about 1500, if more nodes added, IDA crashed,sometime with a message "interr sort_layer_nodes".

If I didn't add any edge or return true directy for grcode_calculating_layout (with a ugly graph beause i have no custom layout),IDA won't crash. All add_edge return true, __try/__except caught nothing.

The obfuscated codes have huge basic blocks(due to some faked jcc, I search basic blocks recursively from the entry),more than 60,000. I set IDA's option Maximum number of nodes to 200,000 but it coundn't help.

What should i do? Maybe i should try to deobfuscate the codes before create a graph(afterall it's too huge causing IDA freeze),but i doubt it can be compressed less than 1500 basic blocks.

Thx.