Kali Intelligence Suite v0.3-alpha releases: the fast, autonomous, central, and comprehensive collection of intelligence

by do son · Published October 30, 2021 · Updated October 25, 2022

Kali Intelligence Suite

Kali Intelligence Suite (KIS) shall aid in the fast, autonomous, central, and comprehensive collection of intelligence by automatically:

executing Kali Linux tools (e.g., dnsrecon, gobuster, hydra, nmap, etc.)
querying publicly available APIs (e.g., Censys.io, Haveibeenpwned.com, Hunter.io, Securitytrails.com, DNSdumpster.com, Shodan.io, etc.)
storing the collected data in a central rational database (see next section)
providing an interface to query and analyze the gathered intelligence

After the execution of each Kali Linux tool or querying APIs, KIS analyses the collected information and extracts as well as reports interesting information like newly identified user credentials, hosts/domains, TCP/UDP services, HTTP directories, etc. The extracted information is then internally stored in different PostgreSql database tables, which enables the continuous, structured enhancement and re-use of the collected intelligence by subsequently executed Kali Linux tools.

Additional features are:

pre-defined dependencies between Kali Linux tools ensure that relevant information like SNMP default community strings or default credentials is known to KIS before trying to access the respective services
remembering the execution status of each Kali Linux tool and API query ensures that already executed OS commands are not automatically executed again
data imports of scan results of external scanners like Masscan, Nessus, or Nmap
supporting the intelligence collection based on virtual hosts (vhost)
using a modular approach that allows the fast integration of new Kali Linux tools
parallel Kali Linux command execution by using a specifiable number of threads
enables users to kill Kali commands via the KIS user interface in case they take too long
access public APIs to enhance data with OSINT

KIS’ Data and Collection Model

The following figure illustrates KIS’ data and collection model. Thereby, each node represents a table in the rational database, and each solid line between the nodes documents the corresponding relationship. The dashed directed graphs document based on which already collected intelligence (source node) KIS is able to collect further information (destination node). The labels of the directed graphs document the techniques used by KIS to perform the collection.

Scoping the Engagement

Scoping is an essential feature of KIS, which specifies on which IP networks, IP addresses, hostnames, etc.,
KIS is allowed to collect data (e.g., via OSINT or active scans). Before diving into scoping, it is important to understand the following collection types, which are supported by KIS:

Passive: Passive collections do not directly interact with the targets but obtain the information from third-party sources like whois. Per default, KIS automatically executes these collections and, thereby, no scoping is required.
Active: Active collections directly interact with the targets by for example actively scanning them. Thus, in contrast to passive collections, this type of collection requires permission from the target’s owner and, therefore, KIS does not automatically perform active collections unless the targets are explicitly marked as in scope.
Active*: Active* collections are actually passive collections. Nevertheless, as accessing some third-party sources is somehow limited (e.g., querying certain sources like Shodan.io cost credits), they are treated like active collectors, and, as a result, targets must be marked as in scope in order to perform active* collections on them.

Changelog v0.3 alpha

Update database model to version v0.3.0
- Add new table and relationships to store information about virtual hosts
- Add database version control and automatic patching
- Update database triggers
Remove collectors dnsdumpster, vncviewer and httpeyewitness
Update command line arguments and output parser for collector sslyze
Update output parser for collectors dnsgobuster, vhostgobuster, dnsspf, hostio and theharvester
Refactor and improve KIS reporting
Update documentation
Update Python3 module sqlalchemy to the latest version
Add unittests and fix bugs