ee-outliers: Open-source framework to detect outliers in Elasticsearch events
ee-outliers is a framework to detect outliers in events stored in an Elasticsearch cluster. The framework was developed for the purpose of detecting anomalies in security events, however, it could just as well be used for the detection of outliers in other types of data.
The framework makes use of statistical models that are easily defined by the user in a configuration file. In case the models detect an outlier, the relevant Elasticsearch events are enriched with additional outlier fields. These fields can then be dashboarded and visualized using the tools of your choice (Kibana or Grafana for example).
The possibilities of the type of anomalies you can spot using ee-outliers is virtually limitless. A few examples of types of outliers we have detected ourselves using ee-outliers during threat hunting activities include:
- Detect beaconing (DNS, TLS, HTTP, etc.)
- Detect geographical improbable activity
- Detect obfuscated & suspicious command execution
- Detect fileless malware execution
- Detect malicious authentication events
- Detect processes with suspicious outbound connectivity
- Detect malicious persistence mechanisms (scheduled tasks, auto-runs, etc.)
- Create your own custom outlier detection use cases specifically for your own needs
- Send automatic e-mail notifications in case one of your outlier use cases hit
- Automatic tagging of asset fields to quickly spot the most interesting assets to investigate
- Fine-grained control over which historical events are checked for outliers
- …and much more!