dns monster v0.9.3 releases: Passive DNS Capture/Monitoring Framework
DNS Monster
Passive DNS collection and monitoring built with Golang, Clickhouse, and Grafana: dnsmonster implements a packet sniffer for DNS traffic. It can accept traffic from a pcap file, a live interface, or a dnstap socket, and can be used to index and store thousands of DNS queries per second (it has shown to be capable of indexing 200k+ DNS queries per second on a commodity computer). It aims to be scalable, simple, and easy to use, and help security teams to understand the details about an enterprise’s DNS traffic. dnsmonster does not look to follow DNS conversations, rather it aims to index DNS packets as soon as they come in. It also does not aim to breach the privacy of the end-users, with the ability to mask source IP from 1 to 32 bits, making the data potentially untraceable. Blogpost
IMPORTANT NOTE: The code before version 1.x is considered beta quality and is subject to breaking changes. Please check the release notes for each tag to see the list of breaking scenarios between each release, and how to mitigate potential data loss.
Main features
- Can use Linux’s afpacket and zero-copy packet capture.
- Supports BPF
- Can fuzz source IP to enhance privacy
- Can have a pre-processing sampling ratio
- Can have a list of “skip” fqdns to avoid writing some domains/suffix/prefix to storage, thus improving DB performance
- Can have a list of “allow” domains to only log hits of certain domains in Clickhouse/Stdout/File
- Modular output with different logic per output stream. Currently stdout/file/clickhouse
- Hot-reload of skipping and allowing domain files
- Automatic data retention policy using ClickHouse’s TTL attribute
- Built-in dashboard using Grafana
- Can be shipped as a single, statically-linked binary
- Ability to be configured using Env variables, command-line options, or configuration file
- Ability to sample output metrics using ClickHouse’s SAMPLE capability
- High compression ratio thanks to ClickHouse’s built-in LZ4 storage
- Supports DNS Over TCP, Fragmented DNS (udp/tcp), and IPv6
- Supports dnstrap over Unix socket or TCP
Changelog v0.9.3
Notable Changes:
- Brand new documentation dnsmonster.dev
- Capture now accepts dash (
---pcapFile=-
) as a more elegant solution to #9 - Dev container populated for each commit (
docker pull ghcr.io/mosajjal/dnsmonster:dev
) - Fixed a bug in
dnstap
pushing data to ClickHouse when the IP address is omitted SIGPIPE
is now handled gracefully in Linux- Major performance improvements to
skipDomains
andallowDomains
.dnsmonster
no longer uses a basic suffix and prefix string match. Instead, it uses a Ternary Search Tree to provide a much faster solution. I’ve tested Cisco Umbrella 1M against my testpcap
as suffix matching. and there was almost no performance degredation! A small sample of the csv:
$ head top-1m-umbrella.csv
google.com.,suffix
www.google.com.,suffix
microsoft.com.,suffix
netflix.com.,suffix
data.microsoft.com.,suffix
ftl.netflix.com.,suffix
prod.ftl.netflix.com.,suffix
events.data.microsoft.com.,suffix
api-global.netflix.com.,suffix
safebrowsing.googleapis.com.,suffix
Install & Use
Copyright (C) 2021 mosajjal