crawpy: content discovery tool

Crawpy

Yet another content discovery tool is written in python.

What makes this tool different than others:

• It is written to work asynchronously which allows reaching to maximum limits. So it is very fast.
• Calibration mode applies filters on its own
• Has a bunch of flags that help you fuzz in detail
• Recursive scan mode for given status codes and with depth
• Report generations, you can later go and check your results
• Multiple url scans

Install

git clone https://github.com/morph3/crawpy
pip3 install -r requirements.txt
or
python3 -m pip install -r requirements.txt

Use

Example

python3 crawpy.py -u https://facebook.com/FUZZ -w ./common.txt -k -ac -e .php,.html
python3 crawpy.py -u https://google.com/FUZZ -w ./common.txt -k -fw 9,83 -r 301,302 -rd 2
python3 crawpy.py -u https://morph3sec.com/FUZZ -w ./common.txt -e .php,.html -t 20 -ac -k
python3 crawpy.py -u https://google.com/FUZZ -w ./common.txt -ac -gr
python3 crawpy.py -u https://google.com/FUZZ -w ./common.txt -ac -gr -o /tmp/test.txt
sudo python3 crawpy.py -l urls.txt -lt 20 -gr -w ./common.txt -t 20 -o custom_reports -k -ac -s
python3 crawpy.py -u https://google.com/FUZZ -w ./common.txt -ac -gr -rd 1 -rc 302,301 -rp admin,backup,support -k

Source: https://github.com/morph3/