Obfuscation Detection v1.7 releases: IDA plugin to pinpoint obfuscated code

by do son · Published August 29, 2021 · Updated March 20, 2022

Obfuscation Detection

Automatically detect obfuscated code and other state machines

Scripts to automatically detect obfuscated code and state machines in binaries.

Implementation is based on IDA 7.4+ (Python3). Check out the following blog posts for more information on the Binary Ninja implementation:

Note:

Due to the recursive nature of plotting a dominator tree of every found function within the binary, the implementation and runtime overhead is expensive. As such, the flattening heuristic is omitted when the binary loaded has more than 50 functions. Functions will be skipped if the ctree structure is too large (more than 50 nodes) to prevent crashes.

MAX_FUNCTIONS = 50

MAX_NODES = 50

# --- snipped ---

if sum([1 for _ in idautils.Functions()]) > MAX_FUNCTIONS:

    detect.partial_heur()

else:

    detect.all_heur()

# --- snipped ---

if sum([1 for _ in FlowChart(get_func(ea))]) > MAX_NODES:

    pass

For more details on partial_heur() and all_heur():

all_heur() calls all heuristic functions on the binary, then prints output of the heuristics of all functions within the binary.

partial_heur() calls cyclomatic complexity, basic block size, and instruction overlapping heuristic functions on the binary, then prints output of the heuristics of the top 10% functions within the binary.

Instruction overlapping heuristic algorithm makes use of mcsema disassembly code to follow jmp and call instructions for better coverage.

Since the script uses the IDA API, any functions that are missed by IDA will likely not be detected.

Usage

Copy the obfDetect directory and obfDetect.py into the IDA Plugins directory.

When IDA has successfully finished loading a binary, the script will print out its banner into the IDC/Python console. If not, the script can be re-loaded using alt-E and selecting it within the plugin dropdown.

The script can be run via the File toolbar as shown below. Alternatively, Ctrl-Shift-H.