gitlab watchman v2.0 releases: audit GitLab for sensitive data and credentials exposed internally

GitLab Watchman

GitLab Watchman is an application that uses the GitLab API to audit GitLab for sensitive data and credentials exposed internally.

Features

It searches GitLab for internally shared projects and looks at:

• Code
• Commits
• Wiki pages
• Issues
• Merge requests
• Milestones

For the following data:

• GCP keys and service account files
• AWS keys
• Azure keys and service account files
• Google API keys
• Slack API tokens & webhooks
• Private keys (SSH, PGP, any other misc private key)
• Exposed tokens (Bearer tokens, access tokens, client_secret, etc.)
• S3 config files
• Tokens for services such as Heroku, PayPal, and more
• Passwords in plaintext
• and more

Time-based searching

You can run GitLab Watchman to look for results going back as far as:

• 24 hours
• 7 days
• 30 days
• All-time

This means after one deep scan, you can schedule GitLab Watchman to run regularly and only return results from your chosen timeframe.

Rules

GitLab Watchman uses custom YAML rules to detect matches in GitLab.

They follow this format:

---

filename:

enabled: #[true|false]

meta:

  name:

  author:

  date:

  description: #what the search should find#

  severity: #rating out of 100#

scope: #what to search, any combination of the below#

- blobs

- commits

- milestones

- wiki_blobs

- issues

- merge_requests

test_cases:

  match_cases:

  - #test case that should match the regex#

  fail_cases:

  - #test case that should not match the regex#

strings:

- #search query to use in GitLab#

pattern: #Regex pattern to filter out false positives#

There are Python tests to ensure rules are formatted properly and that the Regex patterns work in the tests dir

More information about rules, and how you can add your own, is in the file docs/rules.md.

Logging

GitLab Watchman gives the following logging options:

• Log file
• Stdout
• TCP stream

Results are output in JSON format, perfect for ingesting into a SIEM or other log analysis platform.

For file and TCP stream logging, configuration options need to be passed via .conf file or environment variable. See the file docs/logging.md for instructions on how to set it up.

If no logging option is given, GitLab Watchman defaults to Stdout logging.

Changelog v2.0

Added:

• New scopes for finding exposed data in:
  • notes
  • snippets
• Docker image now available from the Docker hub, or by building from source. (Credit @adioss for the inspiration)
• Complete rewrite of the codebase to make searching faster and more efficient.
  • More modern packaging and distribution.
• Logs now include more data
• Additional signatures added to find more leaked data
• Updated logo to play nicely with dark mode displays

Removed:

• Logging to file and TCP stream – logs to stdout like a true 12 factor app. Reroute stdout as you see fit. –output
• .conf file for configuration options. Pass the environment variables GITLAB_WATCHMAN_TOKEN and GITLAB_WATCHMAN_URL

Breaking changes:

• The –output flag is no longer required, and therefore not supported

Install & Use

Copyright (C) 2021 PaperMtn