ODIN: automating penetration testing tasks

by do son · Published August 31, 2018 · Updated August 31, 2018

ODIN

Observe, Detect, and Investigate Networks

A Python tool for automating intelligence gathering, testing and reporting. ODIN is still in active development, so check the dev branch for the bleeding edge. Feedback is welcome!

Note: ODIN is designed to be run on Linux. About 90% of it will absolutely work on Windows or MacOS with Python 3 and a copy of urlcrazy, but `extract`, used for pulling metadata from non-PDF files, is exclusive to Linux. You’ll be fine using an OS without access to `extract`, but you’ll see some warnings and get less information.

ODIN is made possible through the help, input, and work provided by others. Therefore, this project is the entire open source and available to all to use/modify. All this developer did was assemble the tools, convert some of them to Python 3, and stitch them together into an all-in-one toolkit.

What Can ODIN Do?

ODIN is still very much in development, but it aims to automate many of the common recon tasks carried out by penetration testers and red teamers.

Phase 1 – Asset Discovery

Collect basic organization information from sources like the Full Contact marketing database.
Check DNS Dumpster, Netcraft, and TLS certificates to discover subdomains for the provided domains.
Resolve domain and subdomains to IP addresses via socket connections and DNS records.
Collect information for all IP addresses, such as ownership and organization data, from RDAP, whois, and other data sources.
Lookup domains and search for IP addresses on Shodan to collect additional data, such as operating systems, service banners, and open ports.
Check for the possibility of domain takeovers or domain fronting with the domains and subdomains.

Phase 2 – Employee Discovery

Harvest email addresses and employee names for the target organization.
Link employees to social media profiles via search engines and the Twitter API.
Cross check discovered email addresses with Have I Been Pwned.

Phase 3 – Cloud and Web Services

Hunt for Office files and PDFs under the target domain, download them and extract metadata.
Search for AWS S3 buckets and Digital Ocean Spaces using keywords related to the organization.
Take screenshots of discovered web services for a quick, early review of services.

Phase 4 – Reporting

Save all data to an SQLite3 database to allow the data to be easily queried.
Generate an HTML report using default SQL queries to make it simple to peruse the data in a web browser.
Create a Neo4j graph database that ties all of the discovered entities (IP addresses, domains, subdomains, ports, and certificates) together with relationships (e.g. RESOLVES_TO, HAS_PORT).

Changelog

August 11, 2018, Post-Vegas Edition, 1.9.2

Improved WhoXY record parsing and stopped assuming whois and reverse whois results had identical JSON.
ODIN now checks WhoXY whois and reverse whois API crdit balances and warns users if their balance is below the user’s WhoXY limit.
Added a new option for –whoxy-limit to allow users to set a limit for how many domains found via reverse whois look-ups will be automatically added to the master list of domains that are later resolved and used for queries against services like Shodan and Censys.
Email harvesting is now performed for all domains, both those provided and those discovered via reverse whois (see above point).
Switched to Full Contact’s latest API, v3, for better “enriched” data.
Improved grapher.py’s ability to create organization nodes in instances where an organization’s name may not have been returned from Full Contact on whois queries.
Tweaked the HTML report to improve some of the languages.
Cleaned-up some of the dev branch code to prepare for merging with the master.
Added a sample Neo4j graph for the documentation.