birdwatcher: Data analysis and OSINT framework for Twitter
Birdwatcher is a data analysis and OSINT framework for Twitter. It supports creating multiple workspaces where arbitrary Twitter users can be added and their Tweets harvested through the Twitter API for offline storage and analysis. Birdwatcher comes with several modules which can be invoked to further enrich collected data or work with it, e.g. Retrieving user’s Klout score, generating social graphs between users and weighted word clouds based on their Tweets.
- Ruby 1.9.3 or above
You need to set up a user and a database in PostgreSQL for Birdwatcher. Execute the following commands in a terminal:
brew update && brew install graphviz
sudo apt-get update && sudo apt-get install graphviz
brew update && brew install imagemagick
sudo apt-get update && sudo apt-get install libmagickwand-dev imagemagick
$ gem install birdwatcher
git clone https://github.com/michenriksen/birdwatcher.git
Birdwatcher needs to know a bit about what database to connect to as well as API keys to use for API communication. For Twitter, you will need to register an application in order to get a consumer key and consumer secret. Head over to apps.twitter.com and set up your application. You don’t need to have a valid callback URL as we won’t be doing any OAuth authentication.
To make Birdwatcher even more useful, it is recommended to also obtain an API key for the Klout API which is required for some modules to work. If you don’t intend to use Klout modules, you can, of course, skip this step.
On the first run, Birdwatcher will automatically start a configuration wizard where it will ask for the configuration it needs. If you have PostgreSQL connection details, Twitter consumer key & secret, and optional Klout API key, you can start Birdwatcher for the first time:
Enter the details into the configuration wizard:
As can be seen from the above screenshot, Birdwatcher supports multiple Twitter and Klout keys. If you configure Birdwatcher with several keys it will randomly shuffle between them when communicating with APIs to potentially avoid any rate limiting issues on extensive use.
It will save its configuration to
~/.birdwatcherrc. Be careful not to push this file up to any public code repositories!
Birdwatcher pages long command output with the operating system’s default pager command (usually
less) however the command output can be colored and will show up strangely if the pager is not configured to render terminal colors. It is advised to add the following to your
~/.bash_profile or similar file:
This will set up
less as your default pager command as well as configure
less to support terminal colors. Execute the command
source ~/.bash_profile to apply the configuration immediately instead of next time you open a terminal.
Birdwatcher is built as a console and if you have any experience with other frameworks such as Metasploit or Recon-ng, you should feel right at home as Birdwatcher has many of the same concepts and commands.
Copyright (c) 2016 Michael Henriksen