HELK v0.1.7-alpha03042019 Releases: The Hunting ELK

by do son · Published March 4, 2019 · Updated March 4, 2019

HELK [Alpha]

HELK is an ELK (Elasticsearch, Logstash & Kibana) stack with advanced hunting analytic capabilities provided by the implementation of Spark & Graphframes technologies. The Hunting ELK or simply the HELK is one of the first public builds that enables data science features to an ELK stack for free. In addition, it comes with a Jupyter Notebook integration for prototyping in Big Data/Machine learning use cases. This stack provides a full-text search engine mixed with great visualizations, graph relational queries and advanced analytics.

Goals

Provide a free hunting platform to the community and share the basics of Threat Hunting.
Make sense of a large number of event logs and add more context to suspicious events during hunting.
Expedite the time it takes to deploy an ELK stack.
Improve the testing of hunting use cases in an easier and more affordable way.
Enable Data Science via Apache Spark, GraphFrames & Jupyter Notebooks.

HELK Features

Kafka: A distributed publish-subscribe messaging system that is designed to be fast, scalable, fault-tolerant, and durable.
Elasticsearch: A highly scalable open-source full-text search and analytics engine.
Logstash: A data collection engine with real-time pipelining capabilities.
Kibana: An open source analytics and visualization platform designed to work with Elasticsearch.
ES-Hadoop: An open-source, stand-alone, self-contained, small library that allows Hadoop jobs (whether using Map/Reduce or libraries built upon it such as Hive, Pig or Cascading or new upcoming libraries like Apache Spark ) to interact with Elasticsearch.
Spark: A fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs.
GraphFrames: A package for Apache Spark which provides DataFrame-based Graphs.
Jupyter Notebook: An open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text.

Changelog v0.1.7-alpha03042019

fix https://github.com/Cyb3rWard0g/HELK/issues/215
– Logstash plugins offline install (default)
– Logstash mutate statements update
– ES Memory Calculation fix
– Compose files typo