Cloudmarker

Cloudmarker is a cloud monitoring tool and framework. It can be used as a ready-made tool that audits your Azure or GCP cloud environments as well as a framework that allows you to develop your own cloud monitoring software to audit your clouds.

As a monitoring tool, it performs the following actions:

• Retrieves data about each configured cloud using the cloud APIs.
• Saves or indexes the retrieved data into each configured storage system or indexing engine.
• Analyzes the data for potential issues and generates events that represent the detected issues.
• Saves the events to configured storage or indexing engines as well as sends the events as alerts to alerting destinations.

Each of the above four aspects of the tool can be configured via a configuration file.

For example, the tool can be configured to pull data from Azure and index its data in Elasticsearch while it also pulls data from GCP and indexes the GCP data in MongoDB. Similarly, it is possible to configure the tool to check for unencrypted disks in Azure, generate events for it, and send them as alerts by email while it checks for insecure firewall rules in both Azure and GCP, generate events for them, and save those events in MongoDB.

This degree of flexibility to configure audits for different clouds in different ways comes from the fact that Cloudmarker is designed as a combination of lightweight framework and a bunch of plugins that do the heavy-lifting for retrieving cloud data, storing the data, analyzing the data, generating events, and sending alerts. These four types of plugins are formally known as cloud plugins, store plugins, event plugins, and alert plugins, respectively.

As a result of this plugin-based architecture, Cloudmarker can also be used as a framework to develop your own plugins that extend its capabilities by adding support for new types of clouds or data sources, storage or indexing engines, event generation, and alerting destinations.

Why Cloudmarker?

One might wonder why we need a new project like this when similar projects exist. When we began working on this project in 2017, we were aware of similar tools that supported AWS and GCP but none that supported Azure at that time. As a result, we wrote our own tool to support Azure. We later added support for GCP as well. What began as a tiny proof of concept gradually turned into a fair amount of code, so we thought, we might as well share this project online, so that others could use it and see if they find value in it.

So far, some of the highlights of this project are:

• It is simple. It is easy to understand how to use the four types of plugins (clouds, stores, events, and alerts) to perform an audit.
• It is excellent at creating an inventory of the cloud environment.
• The data inventory it creates is easy to query.
• It is good at detecting insecure firewall rules and unencrypted disks. New detection mechanisms are coming up.

Features

Since Cloudmarker is not just a tool but also a framework, a lot of its functionality can be extended by writing plugins. However, Cloudmarker also comes bundled with a default set of plugins that can be used as-is without writing a single line of code. Here is a brief overview of the features that come bundled with Cloudmarker:

• Perform scheduled or ad hoc audits of the cloud environment.
• Retrieve data from Azure and GCP.
• Store or index retrieved data in Elasticsearch, MongoDB, Splunk, and the file system.
• Look for insecure firewall rules and generate firewall rule events.
• Look for unencrypted disks (Azure only) and generate events.
• Send alerts for events via email and Slack as well as save alerts in one of the supported storage or indexing engines (see the third point above).
• Normalize firewall rules from Azure and GCP which are in different formats to a common object model ("com") so that a single query or event rule can search for or detect issues in firewall rules from both clouds.

Wishlist

• Add more event plugins to detect different types of insecure configuration.
• Normalize other types of data into a common object model ("com") just like we do right now for firewall rules.

Install && Use

Copyright (c) 2018 Cloud Marker Authors and Contributors