dns monster v0.9.3 releases: Passive DNS Capture/Monitoring Framework

DNS Monster

Passive DNS collection and monitoring built with Golang, Clickhouse, and Grafana: dnsmonster implements a packet sniffer for DNS traffic. It can accept traffic from a pcap file, a live interface, or a dnstap socket, and can be used to index and store thousands of DNS queries per second (it has shown to be capable of indexing 200k+ DNS queries per second on a commodity computer). It aims to be scalable, simple, and easy to use, and help security teams to understand the details about an enterprise’s DNS traffic. dnsmonster does not look to follow DNS conversations, rather it aims to index DNS packets as soon as they come in. It also does not aim to breach the privacy of the end-users, with the ability to mask source IP from 1 to 32 bits, making the data potentially untraceable. Blogpost

IMPORTANT NOTE: The code before version 1.x is considered beta quality and is subject to breaking changes. Please check the release notes for each tag to see the list of breaking scenarios between each release, and how to mitigate potential data loss.

Main features

• Can use Linux’s afpacket and zero-copy packet capture.
• Supports BPF
• Can fuzz source IP to enhance privacy
• Can have a pre-processing sampling ratio
• Can have a list of “skip” fqdns to avoid writing some domains/suffix/prefix to storage, thus improving DB performance
• Can have a list of “allow” domains to only log hits of certain domains in Clickhouse/Stdout/File
• Modular output with different logic per output stream. Currently stdout/file/clickhouse
• Hot-reload of skipping and allowing domain files
• Automatic data retention policy using ClickHouse’s TTL attribute
• Built-in dashboard using Grafana
• Can be shipped as a single, statically-linked binary
• Ability to be configured using Env variables, command-line options, or configuration file
• Ability to sample output metrics using ClickHouse’s SAMPLE capability
• High compression ratio thanks to ClickHouse’s built-in LZ4 storage
• Supports DNS Over TCP, Fragmented DNS (udp/tcp), and IPv6
• Supports dnstrap over Unix socket or TCP

Changelog v0.9.3

Notable Changes:

• Brand new documentation dnsmonster.dev
• Capture now accepts dash (---pcapFile=-) as a more elegant solution to #9
• Dev container populated for each commit (docker pull ghcr.io/mosajjal/dnsmonster:dev)
• Fixed a bug in dnstap pushing data to ClickHouse when the IP address is omitted
• SIGPIPE is now handled gracefully in Linux
• Major performance improvements to skipDomains and allowDomains. dnsmonster no longer uses a basic suffix and prefix string match. Instead, it uses a Ternary Search Tree to provide a much faster solution. I’ve tested Cisco Umbrella 1M against my test pcap as suffix matching. and there was almost no performance degredation! A small sample of the csv:
  $ head top-1m-umbrella.csv
  google.com.,suffix
  www.google.com.,suffix
  microsoft.com.,suffix
  netflix.com.,suffix
  data.microsoft.com.,suffix
  ftl.netflix.com.,suffix
  prod.ftl.netflix.com.,suffix
  events.data.microsoft.com.,suffix
  api-global.netflix.com.,suffix
  safebrowsing.googleapis.com.,suffix

Install & Use

Copyright (C) 2021 mosajjal