WhatWeb v0.5: Next generation web scanner
WhatWeb can be stealthy and fast, or thorough but slow. WhatWeb supports an aggression level to control the tradeoff between speed and reliability. When you visit a website in your browser, the transaction includes many hints of what web technologies are powering that website. Sometimes a single webpage visit contains enough information to identify a website but when it does not, WhatWeb can interrogate the website further. The default level of aggression, called ‘stealthy’, is the fastest and requires only one HTTP request of a website. This is suitable for scanning public websites. More aggressive modes were developed for use in penetration tests.
Most WhatWeb plugins are thorough and recognize a range of cues from subtle to obvious. For example, most WordPress websites can be identified by the meta HTML tag, e.g. ‘<meta name=”generator” content=”WordPress 2.6.5″>’, but a minority of WordPress websites remove this identifying tag but this does not thwart WhatWeb. The WordPress WhatWeb plugin has over 15 tests, which include checking the favicon, default installation files, login pages, and checking for “/wp-content/” within relative links.
- Over 1800 plugins
- Control the trade-off between speed/stealth and reliability
- Performance Tuning. Control how many websites to scan concurrently.
- Multiple log formats: Brief (greppable), Verbose (human readable), XML, JSON, MagicTree, RubyObject, MongoDB, ElasticSearch, SQL.
- Proxy support including TOR
- Custom HTTP headers
- Basic HTTP authentication
- Control over webpage redirection
- IP address ranges
- Fuzzy matching
- Result certainty awareness
- Custom plugins defined on the command line
- IDN (International Domain Name) support
Version 0.5.0 – June 9, 2019
* Refactored and modularised the codebase
* Added 9 unit tests
* Better error reporting in commandline options
* Updated README and converted to Markdown format
* Updated manpage
* Added loading cookies from a FILE with –cookie-jar=FILE in the document.cookie format
* Plugins instance variable handling improvement. @Code0x58
* Plugin name defined within plugin as variable. @Code0x58
* Changed from functions to blocks for passive and aggressive plugins. @Code0x58
* Changed plugin locking @Code0x58
* Bug fix – clean incorrent UTF-8 byte sequences when loading from file @dirtyfilthy
* Updated all plugins with an authors array to show multiple authors
* Added plugin-development/convert-plugin-authors.rb script
* Changed the behaviour of –grep to only output results that match the grep plugin
* Changed the behaviour of –grep so regexp or text can be matched
* Removed feature –follow-redirect same-domain because it relies on the out-dated TLD library for valid TLDs and SLDs
* Removed requirement for ‘nmap’ and now use the IPAddr gem for IP address ranges. CIDR, x.x.x.x-x.x.x.x and x.x.x.x-x supported
* Added IDN (International Domain Name) support
* Merged 31 new plugins by Bhavin Senjaliya and 22 plugin updates from the unofficial WhatWeb gem at https://rubygems.org/gems/whatweb/
* Added WhatWeb Logo and icons.
* Updated the usage help. Reduced the size of the short usage help.
* Plugin list now shows plugin names and websites instead of a truncated plugin description.
* Bug fix – Ruby 2.4 and 2.5 OpenSSL issue
* Bug fix – Fixed ElasticSearch version 6+ issue
* Bug fix – SQL output issue
* Bug fix – In HTTP Auth parsing @rmaksimov
* Fix bug causing WhatWeb to exit with no output. Thanks @mguillau42 for tracking down this race condition.
* Reached over 1800 plugins
sudo apt-get install ruby ruby-dev libopenssl-ruby
sudo gem install bson
sudo gem install bson_ext
sudo gem install mongo
sudo gem install rchardet
git clone https://github.com/urbanadventurer/WhatWeb.git