birdwatcher: Data analysis and OSINT framework for Twitter

by do son · Published · Updated

birdwatcher

Birdwatcher is a data analysis and OSINT framework for Twitter. It supports creating multiple workspaces where arbitrary Twitter users can be added and their Tweets harvested through the Twitter API for offline storage and analysis. Birdwatcher comes with several modules which can be invoked to further enrich collected data or work with it, e.g. Retrieving user’s Klout score, generating social graphs between users and weighted word clouds based on their Tweets.

birdwatcher

Installation

Requirement

  • Ruby 1.9.3 or above
  • RubyGems
  • PostgreSQL
    You need to set up a user and a database in PostgreSQL for Birdwatcher. Execute the following commands in a terminal:
    sudo su postgres # Not necessary on Mac OS X

    createuser -s birdwatcher --pwprompt

    createdb -O birdwatcher birdwatcher

     

     

  • Graphviz
    brew update && brew install graphviz
    or
    sudo apt-get update && sudo apt-get install graphviz
  • ImageMagick
    brew update && brew install imagemagick
    or
    sudo apt-get update && sudo apt-get install libmagickwand-dev imagemagick
  • Birdwatcher
    $ gem install birdwatcher

Download

git clone https://github.com/michenriksen/birdwatcher.git

Configuration

Birdwatcher needs to know a bit about what database to connect to as well as API keys to use for API communication. For Twitter, you will need to register an application in order to get a consumer key and consumer secret. Head over to apps.twitter.com and set up your application. You don’t need to have a valid callback URL as we won’t be doing any OAuth authentication.

To make Birdwatcher even more useful, it is recommended to also obtain an API key for the Klout API which is required for some modules to work. If you don’t intend to use Klout modules, you can, of course, skip this step.

On the first run, Birdwatcher will automatically start a configuration wizard where it will ask for the configuration it needs. If you have PostgreSQL connection details, Twitter consumer key & secret, and optional Klout API key, you can start Birdwatcher for the first time:

$ birdwatcher

Enter the details into the configuration wizard:

Birdwatcher configuration wizard

As can be seen from the above screenshot, Birdwatcher supports multiple Twitter and Klout keys. If you configure Birdwatcher with several keys it will randomly shuffle between them when communicating with APIs to potentially avoid any rate limiting issues on extensive use.

It will save its configuration to ~/.birdwatcherrc. Be careful not to push this file up to any public code repositories!

System pager

Birdwatcher pages long command output with the operating system’s default pager command (usually less) however the command output can be colored and will show up strangely if the pager is not configured to render terminal colors. It is advised to add the following to your ~/.bash_profile or similar file:

# Get color support for 'less'

export LESS="--raw-control-chars"

export PAGER="less"

 

This will set up less as your default pager command as well as configure less to support terminal colors. Execute the command source ~/.bash_profile to apply the configuration immediately instead of next time you open a terminal.

Usage

Birdwatcher is built as a console and if you have any experience with other frameworks such as Metasploit or Recon-ng, you should feel right at home as Birdwatcher has many of the same concepts and commands.

Copyright (c) 2016 Michael Henriksen

Source: https://github.com/michenriksen/

Tags: birdwatcherOSINTOSINT Frameworktwitter