osintbuddy: Node graphs, OSINT data mining, and plugins

by do son · October 5, 2023

OSINTBuddy

Welcome to the OSINTBuddy project where you can connect, combine, and get insights from unstructured and public data as results that can be explored step-by-step. An easy-to-use plugin system allows any Python developer to quickly integrate new data sources.

An almost incomprehensible amount of data is created every day. And each year, figures are growing at an ever-increasing rate. These data sources can be divided up into six different categories of information flow:

Public government data
Media (newspapers, magazines, radio)
Internet (blogs, discussion groups, citizen media, etc)
Professional and academic publications (budgets, hearings, telephone directories, websites, etc)
Commercial data (commercial imagery, financial assessments, databases, etc)
Grey literature (technical reports, preprints, patents, business documents, etc)

Some of the main hindrances to practical OSINT is the volume of information it has to deal with (information explosion) and the issue of low quality data. With the majority of this data being unstructured, there’s many challenges to analyzing it and producing actionable intelligence since most data analytics databases are designed for structured data.

We’ve decided to do something about it. The rapid developments in technologies such as AI and big data analytics have opened new horizons for OSINT which weren’t previously available. We want our information quick and to the point, that’s why we’re building an open-source OSINT tool that’s free. Free to use, free to modify, free to do with as you wish, and built with plain old web technologies anyone can learn. But this isn’t really the project. This is a free new community. A community for OSINT enthusiast around the world and we need your help to design it, to program it, and to build it. We want to hear your suggestions, your ideas, and we’re going to build it right in front of your eyes. The notion of a “needle in a haystack” is taken to the extreme on the internet. Let’s build a magnet.

Key Features

Simplified data fetching from multiple sources.
Visual representation of fetched data for easy understanding.
A powerful development platform that is open for contributions.
New plugin-based system for transforming data, check out the osintbuddy package. Documentation is coming soon.