Scumblr: Automated OSINT
Scumblr
Scumblr is a web application that allows performing periodic syncs of data sources (such as Github repositories and URLs) and performing analysis (such as static analysis, dynamic checks, and metadata collection) on the identified results. Scumblr helps you streamline proactive security through an intelligent automation framework to help you identify, track, and resolve security issues faster.
Scumblr ships with a number of tasks to help you streamline security automation including:
Sync Tasks
- Github – Sync results from github Repositories
- Route53 DNS – Sync FQDNs from Route53 DNS
- Manual Result Upload – Specify a new line delimited list of results you’d like to sync into Scumblr
Security Tasks
- Github Search – Search Github for secrets, anti-patterns, and vulnerabilities in your repositories
- Curl – Execute curl commands to identify vulnerabilities or issues against Scumblr results
- Bandit – Perform static code analysis against Python projects
- Brakeman – Perform static code analysis against Ruby on Rails projects.
Search Tasks (legacy)
- iTunes Store
- Certificate Transparency
- eBay
- Google Play
- RSS Feeds (useful for full disclosure searches)
- YouTube
Scumblr also provides a number of novel features that streamline security automation including:
- Tracking, ticketing, regression monitoring, and auto-remediation of security vulnerabilities
- Metadata storage in results to allow for advanced result filtering
- Customizable views and sorting of results and tasks to get you to the important details faster
- Saveable result filters that can be shared with colleagues
- Event model for auditing changes to results so you can keep an eye on what is happening
- Email subscriptions for specific results or tasks you care about (such as monitoring when a security task finds a new vulnerability)
- Advanced asynchronous task scheduling to allow for task chaining and task batching
Scumblr uses the Workflowable gem to allow setting up flexible workflows for different types of results.
What can Scumblr look for and analyze?
Just about anything! Scumblr searches utilize plugins called Tasks. Each Task knows how to perform a search or sync via a certain site or API (Github, Route53, Google, Pastebin, Twitter, etc.). Tasks can be configured from within Scumblr based on the options available by the Task. What are some things you might want to look for or analyze? How about:
- Your organization’s public or private github repositories
- When new FQDNS are created in your organization’s DNS
- Detection of anti-patterns in the source code
- Dynamic checks against running web servers for security issues
- Static code analysis across a large number of repositories using Brakeman or Bandit
- Get an alert on full disclosure security reports on vulnerabilities in your team’s version of Apache
These are just a few examples of things that you may want to keep an eye on!
Scumblr found stuff, now what?
Scumblr provides a handy vulnerability object you can use to monitor a particular result security issues. You can also create Status fields to associate with results, allowing you to track the state of a result or it’s remediation over time.
You can create simple or complex workflows to be used along with your results. This can be as simple as marking results as “Reviewed” once they’ve been looked at, or much more complex involving multiple steps with automated actions occurring during the process.
Installation && Tutorial
Copyright 2016 Netflix, Inc.
Source: https://github.com/Netflix/