intelmq v3.2.1 releases: collecting and processing security feeds

by do son · Published May 23, 2019 · Updated August 29, 2023

IntelMQ is a solution for IT security teams (CERTs, CSIRTs, abuse departments,…) for collecting and processing security feeds (such as log files) using a message queuing protocol. It’s a community-driven initiative called IHAP (Incident Handling Automation Project) which was conceptually designed by European CERTs/CSIRTs during several InfoSec events. Its main goal is to give to incident responders an easy way to collect & process threat intelligence thus improving the incident handling processes of CERTs.

IntelMQ’s design was influenced by AbuseHelper however it was re-written from scratch and aims at:

Reducing the complexity of system administration
Reducing the complexity of writing new bots for new data feeds
Reducing the probability of events lost in all process with persistence functionality (even system crash)
Use and improve the existing Data Harmonization Ontology
Use JSON format for all messages
Integration of the existing tools (AbuseHelper, CIF)
Provide easy way to store data into Log Collectors like ElasticSearch, Splunk, databases (such as PostgreSQL)
Provide easy way to create your own black-lists
Provide easy communication with other systems via HTTP RESTFUL API

It follows the following basic meta-guidelines:

Don’t break simplicity – KISS
Keep it open source – forever
Strive for perfection while keeping a deadline
Reduce complexity/avoid feature bloat
Embrace unit testing
Code readability: test with inexperienced programmers
Communicate clearly

Changelog v3.2.1

All Bots

Fixes an issue which prevented bots from stopping gracefully after reloading.
As logrotate reloads all bots regularly, this bug affects most IntelMQ installations.

Reverse DNS Expert

Until IntelMQ version 3.2.0, the bot incorrectly cached and re-used results for /24 networks instead of single IP addresses.
If the bot retrieved the PTR for 192.0.43.7, it was cached for 192.0.43.0/24 and used for all IP addresses in this range, for example for 192.0.43.8.
IntelMQ version 3.2.1 fixes this issue.

The bugfix will correctly increase the cache sizes and decrease the performance, as less (incorrect) data is re-used.