Analyzer: Open source threat intelligence framework

Analyzer

Analyzer is an open-source threat intelligence framework that automates extracting artifacts and IOCs from file/dump into a readable format.

The main tool called (QManager) that interacted with the rest of them through Pipes, APIs, Events, and RAW Files. The interaction happened in phases using a queue due to the variation and availability of some tools. Then, results were handled by a parser that piped structured and unstructured information into centralized databases. Finally, it informed me of the end of the process by sending a notification message. This worked just fine until recently when I wanted to implement some machine learning and a few other features to the process. After a lot of researching, I came to the conclusion that the best way is rewriting most of those old tools, and implement different opensource packages into modules. Then, have them compiled into one framework for easy management by researchers.

Features

• Runs locally and easy to maintain
• Analyze full folder or individual file
• Generates HTML and JSON as output
• Write your ideas under each output
• General file information MD5, charset, mime, ssdeep
• Different string/patterns analysis methods
• NL English words detection
• OCR words detections (!)
• IPS countries description
• IPS reserved hints
• World IPS countries image
• World IPS countries flags
• Ports description
• DNS servers description (Top servers)
• Websites similarity detection (Top 10000)
• Artifacts force directed image
• Xrefs force directed image
• Xrefs count table
• MITRE att&ck tools detection (could be FP)
• MITRE att&ck patterns detection (could be FP)
• Similarity image
• Yara module and yara rules included
• JSON editable records
• URL/EMAIL/TEL/Tags common patterns extraction
• Credit cards patterns extraction
• Encryption patterns (base64, md5, sha1..) detection
• DGA (domain generation algorithm) patterns detection
• BOM detection
• Credential extractor
• Linux
  • ELF information
  • API functions descriptions
  • System commands descriptions
  • Secitons descriptions
  • Lib descriptions
  • Encrypted section detection
  • Symbs extraction
  • MITRE artifacts detection
  • Xref detection
• macOS
  • DMG extraction
  • Shell code detection
  • PLIST information
  • MITRE artifacts detection
  • macOS information
• Windows
  • PE information
  • Encrypted section detection
  • Secitons descriptions
  • DLL descriptions
  • Symbs extraction
  • Signature extraction
  • API functions descriptions
  • PE ASLR, DEP, SEH and CFG detection
  • MITRE artifacts detection
  • API Behavior detections (DLL injection, Process Hollowing, Process Doppelganging etc..)
  • Xref detection
• Android
  • APK information
  • DEX information
  • Manifest descriptions
  • Intent descriptions
  • Resources extraction
  • Symbs extraction
  • Classes extraction
  • Big functinon identification
  • Xref detection
  • API Behavior detections
• Iphone
  • IPA information
• BlackBerry (COD)
  • COD information
  • Functions extraction
  • Strings extraction
• PCAP
  • Frame filter
  • HTTP filter
  • DNS filter
  • ARP filter
  • WAF detection
  • DGA detction
• PDF
  • Objects enumeration
  • Keys (javascript, js, OpenAction) enumeration
  • Streams parsing
  • String analysis
• Office[x]
  • Meta info extraction
  • Hyper and target links extraction
  • Bin printable parser
• RTF
  • Number of objects
  • Object extraction
• EMAIL
  • Header information
  • Attachment extraction and parsing
• Archives
  • Extract mimes and guess by extensions
  • Finding patterns in all unpacked files
• HTML
  • Extract scripts, iframes, links and forms
  • Decode/analyze links
  • Script entropy

Install

git clone https://github.com/qeeqbox/analyzer.git && cd analyzer && chmod +x run.sh && ./run.sh auto_configure

Run it with docker

docker-compose -f docker-compose-dev.yml up –build
Then open http://127.0.0.1:8000/login/

Copyright (C) 2020 qeeqbox

Source: https://github.com/bd249ce4/