harpoon: open source and threat intelligence

by do son · Published December 12, 2018 · Updated December 12, 2018

Harpoon is a tool to automate threat intelligence and open source intelligence tasks. It is written in Python 3 and organized in plugins so the idea is to have one plugin per platform or task.

Some notes on the principles behind this tool :

Python 3 only, python 2 is dead, I am not even trying to support it
Many OSINT tools try to gather as much information as possible from an indicator (domain or email) without really any interest in where it comes from. Harpoon does not follow this philosophy. It mostly allows you to realize a single task per command (with a couple of more general commmands using several tools). I think it is really important during an investigation to understand where an information comes from and how reliable it is.
In many cases, I also wanted to explore APIs to see what was possible with them, and I ended up rewriting some libraries (like SpyOnWeb) just because I wanted to understand exactly what it did and how. So I reinvented the wheel many times and I am totally fine with it.
Harpoon is organized into subcommands that are easy to implement, these commands rely on internal or external libraries. These commands also use a single configuration file that you need to complete manually when an API key is needed.
This tool is not perfect, it only does what I needed to automate, it is likely buggy and has a long list of things I would like to implement one day (but it may never happen). Feel free to open issues or propose Pull Requests.

Install

pip install git+ssh://git@github.com/Te-k/harpoon  --process-dependency-links

Optionally if you want to use the screenshot plugin, you need phantomjs and npm installed:



npm install -g phantomjs

To configure harpoon, run harpoon config and fill in the needed API keys. Then run harpoon config -u to download needed files. Check what plugins are configured with harpoon config -c.

Usage

help                Give help on an Harpoon command

safebrowsing        Check if the given domain is in Google safe Browsing list

hibp                Request Have I Been Pwned API (https://haveibeenpwned.com/)

cache               Requests webpage cache from different sources

misp                Get information from a MISP server through the API

spyonweb            Search in SpyOnWeb through the API

censys              Request information from Censys database (https://censys.io/)

shodan              Requests Shodan API

vt                  Request Virus Total API

config              Configure Harpoon

fullcontact         Requests Full Contact API (https://www.fullcontact.com/)

googl               Requests Google url shortener API

ip                  Gather information on an IP address

twitter             Requests Twitter API

asn                 Gather information on an ASN

robtex              Search in Robtex API (https://www.robtex.com/api/)

hunter              Request hunter.io information through the API

otx                 Requests information from AlienVault OTX

crtsh               Search in https://crt.sh/ (Certificate Transparency database)

github              Request Github information through the API

bitly               Request bit.ly information through the API

screenshot          Takes a screenshot of a webpage

greynoise           Request Grey Noise API

telegram            Request information from Telegram through the API

threatgrid          Request Threat Grid API

pgp                 Search for information in PGP key servers

totalhash           Request Total Hash API

dns                 Map DNS information for a domain or an IP

hybrid              Requests Hybrid Analysis platform

malshare            Requests MalShare database

Tutorial

Source: https://github.com/Te-k/

harpoon: open source and threat intelligence

Search

Brilliantly

Content & Links