Token-Hunter: Collect OSINT for GitLab groups and members

Token-Hunter

Collect OSINT for GitLab groups and members and search the group and group members’ snippets, issues, and issue discussions for sensitive data that may be included in these assets. The information gathered is intended to compliment and inform the use of additional tools such as TruffleHog or GitRob, which search git commit history using a similar technique of regular expression matching.

How the tool works

Start by providing a group ID for a specific group on GitLab. You can find the group ID underneath the group name in the GitLab UI. Token-Hunter will use the GitLab group ID to find all associated projects for that group and, optionally, the groups members personal projects. Configure the tool to look for sensitive data in assets related to the projects it finds. Token-Hunter uses the same set of regular expressions as TruffleHog with a few additions for GitLab specific tokens. Token-Hunter depends on these easily configurable regular expressions for accuracy and effectiveness. Currently, the tool supports GitLab snippets, issues, and issue discussions with plans for future expansion to other assets. The tool is intended to be very configurable to allow for efficient discovery of sensitive data in the assets you’re specifically interested in.

Install

Before running the tool, you will need to generate a GitLab Personal Access Token (PAT) and export it as an environment variable. This can be done as shown below (please select api in the scopes section):

export GITLAB_API_TOKEN=xxxxx

Download

git clone https://gitlab.com/gitlab-com/gl-security/gl-redteam/token-hunter.git
pip3 install -r ./requirements.txt

Use

Example

./token-hunter.py -g 123456

The simplest use case is to return all the project URLs associated with a group by providing the group ID with the -g switch. You can find the group ID underneath the group name in the GitLab UI. No token searches are performed with this configuration.

./token-hunter.py -g 123456 -m

Finds all projects for group 123456 as well as all of the personal projects for the group members. No token searches are performed with this configuration.

./token-hunter.py -g 123456 -ms

Finds all projects for group 123456 as well as all of the personal projects for the group members. The -s switch tells Token-Hunter to search GitLab snippets associated with each found project for sensitive data.

./token-hunter.py -g 123456 -msir

Finds all projects for group 123456 as well as all of the personal projects for the group members. The -s switch tells Token-Hunter to search GitLab snippets associated with each found project for sensitive data. The -i switch tells Token-Hunter to also search issues and discussions for each of the found projects for sensitive data. The -r switch tells Token-Hunter to also search merge requests and merge request discussions for each of the found projects. CAUTION: This configuration has the potential to pull a lot of data!

./token-hunter.py -g 123456 -msit -u https://mygitlab-instance.com -p http://127.0.01:8080 -c /Users/hacker/owasp_zap_ca_cert.cer -l ./appended-output.txt

Performs the same asset searches as the previous example against a self-hosted installation of GitLab running at https://mygitlab-instance.com. Requests and responses that the tool generates are proxied through http://127.0.01:8080 using the certificate defined at the fully qualified path /Users/hacker/owasp_zap_ca_cert.cer to decrypt the TLS traffic. Timestamps and origin IP are excluded from the output with the -t switch. The output is APPENDED to the ./appended-output.txt file with the -l switch.

Copyright (c) 2019 GitLab Red Team

Source: https://github.com/codeEmitter/