TorBoT v4.0 releases: Deep and Dark Web OSINT Tool

TorBoT – OSINT tool for Deep and Dark Web.

The basic procedure executed by the web crawling algorithm takes a list of seed URLs as its input and repeatedly executes the following steps:

1. Remove a URL from the URL list.
2. Check existence of the page.
3. Download the corresponding page.
4. Check the Relevancy of the page.
5. Extract any links contained in it.
6. Check the cache if the links are already in it.
7. Add the unique links back to the URL list.
8. After all URLs are processed, return the most relevant page.

Features

1. Onion Crawler (.onion)
2. Returns Page title and address with a short description about the site
3. Save links to database
4. Get data from site
5. Save crawl info to JSON file
6. Crawl custom domains
7. Check if the link is live
8. Built-in Updater
9. Build visual tree of link relationship that can be quickly viewed or saved to an image file
10.  Visualization Module Revamp
11. Implement BFS Search for webcrawler
12. Use Golang service for concurrent webcrawling
13. Improve stability (Handle errors gracefully, expand test coverage and etc.)
14. Improve performance (Done with gotor)

Changelog v4.0

• Bump urllib3 from 1.26.9 to 1.26.17 by @dependabot in #296
• Fix typo in README.md by @eltociear in #297
• fix typos in README.md by @rahul0x00 in #301
• Rename PULL_REQUEST_TEMPLATE.md to .github/PULL_REQUEST_TEMPLATE.md by @vedant-z in #306
• Removing unused files by @KingAkeem in #308
• Moved Markdown files to doc directory by @shaswat-satyam in #305
• Replacing gotor with httpx and other major changes by @KingAkeem in #307

Use

Before you run the torBot make sure the following things are done properly:

• Run tor service sudo service tor start
• Make sure that your torrc is configured to SOCKS_PORT localhost:9050

On Linux platforms, you can make an executable for TorBot by using the install.sh script. You will need to give the script the correct permissions using chmod +x install.sh Now you should have an executable file named torBot, run ./torBot to execute the program.

usage: Gather and analayze data from Tor sites.



optional arguments:

  -h, --help            show this help message and exit

  --version             Show current version of TorBot.

  --update              Update TorBot to the latest stable version

  -q, --quiet

  -u URL, --url URL     Specifiy a website link to crawl

  -s, --save            Save results in a file

  -m, --mail            Get e-mail addresses from the crawled sites

  -p, --phone           Get phone numbers from the crawled sites

  --depth DEPTH         Specifiy max depth of crawler (default 1)

  --gather              Gather data for analysis

  -v, --visualize       Visualizes tree of data gathered.

  -d, --download        Downloads tree of data gathered.

  -e EXTENSION, --extension EXTENSION

                        Specifiy additional website extensions to the list(.com , .org, .etc)

  -c, --classify        Classify the webpage using NLP module

  -cAll, --classifyAll  Classify all the obtained webpages using NLP module

  -i, --info            Info displays basic info of the scanned site` 

• NOTE: All flags under -u URL, –url URL must also be passed a -u flag.

Read more about torrc here: Torrc

Install

Copyright (C) 2018 P5N4PPZ

Source: https://github.com/DedSecInside/