StackStorm v3.8 releases: auto-remediation, security responses, troubleshooting, deployments

StackStorm is a platform for integration and automation across services and tools. It ties together your existing infrastructure and application environment so you can more easily automate that environment — with a particular focus on taking actions in response to events.

StackStorm helps automate common operational patterns. Some examples are:

• Facilitated Troubleshooting – triggering on system failures captured by Nagios, Sensu, New Relic and other monitoring, running a series of diagnostic checks on physical nodes, OpenStack or Amazon instances, and application components, and posting results to a shared communication context, like HipChat or JIRA.
• Automated remediation – identifying and verifying hardware failure on OpenStack compute node, properly evacuating instances and emailing VM about potential downtime, but if anything goes wrong – freezing the workflow and calling PagerDuty to wake up a human.
• Continuous deployment – build and test with Jenkins, provision a new AWS cluster, turn on some traffic with the load balancer, and roll-forth or roll-back based on NewRelic app performance data.

StackStorm helps you compose these and other operational patterns as rules and workflows or actions; and these rules and workflows – the content within the StackStorm platform – are stored as code which means they support the same approach to collaboration that you use today for code development and can be shared with the broader open source community via StackStorm.com/community for example.

How it works

StackStorm plugs into the environment via the extensible set of adapters: sensors and actions.

• Sensors are python plugins for inbound integration that watch for events from external systems and fire a StackStorm trigger when an event happens.
• Triggers are StackStorm representations of external events. There are generic triggers (e.g. timers, webhooks) and integration triggers (e.g. Sensu alert, JIRA issue updated). A new trigger type can be defined by writing a sensor plugin.
• Actions are StackStorm outbound integrations. There are generic actions (ssh, REST call), integrations (OpenStack, Docker, Puppet), or custom actions. Actions are either python plugins or any scripts, consumed into StackStorm by adding a few lines of metadata. Actions can be invoked directly by user via CLI or API, or used and called as part of automation – rules and workflows.
• Rules map triggers to actions (or to workflows), applying matching criteria and mapping trigger payload to action inputs.
• Workflows stitch actions together into “uber-actions”, defining the order, transition conditions, and passing the data. Most automation is more than one-step and thus need more than one action. Workflows, just like “atomic” actions, are available in action library, can be invoked manually or triggered by rules.
• Packs are the units of content deployment. They simplify the management and sharing of StackStorm pluggable content by grouping integrations (triggers and actions) and automation (rules and workflows). A growing number of packs is available on StackStorm community. User can create their own packs, share them on Github, or submit to StackStorm community repo.
• Audit trail of action executions, manual or automated, is recorded and stored with full details of triggering context and execution results. It is also captured in audit logs for integrating with external logging and analytical tools: LogStash, Splunk, statsd, syslog.

StackStorm is a service with modular architecture. It comprises loosely coupled service components that communicate over the message bus and scales horizontally to deliver automation at scale. StackStorm has a full REST API, CLI client for admins and users to operate it locally or remotely, and Python client bindings for the developer’s convenience. Web UI is coming soon.

Changelog v3.8

Fixed

• Fix redis SSL problems with sentinel #5660
• Fix a bug in the pack config loader so that objects covered by an patternProperties schema
  or arrays using additionalItems schema(s) can use encrypted datastore keys and have their
  default values applied correctly. #5321

  Contributed by @cognifloyd

• Fixed st2client/st2client/base.py file to check for http_proxy and https_proxy environment variables for both lower and upper cases.

  Contributed by @S-T-A-R-L-O-R-D

• Fixed a bug where calling ‘get_by_name’ on client for getting key details was not returning any results despite key being stored. #5677

  Contributed by @bharath-orchestral

• Fixed st2client/st2client/base.py file to use https_proxy(not http_proxy) to check HTTPS_PROXY environment variables.

  Contributed by @wfgydbu

• Fixed schema utils to more reliably handle schemas that define nested arrays (object-array-object-array-string) as discovered in some
  of the ansible installer RBAC tests (see #5684). This includes a test that reproduced the error so we don’t hit this again. #5685
• Fixed eventlet monkey patching so more of the unit tests work under pytest. #5689
• Fix and reenable prance-based openapi spec validation, but make our custom x-api-model validation optional as the spec is out-of-date. #5709
  Contributed by @cognifloyd
• Fixed generation of st2.conf.sample to show correct syntax for [sensorcontainer].partition_provider (space separated key:value pairs). #5710
  Contributed by @cognifloyd
• Fix access to key-value pairs in workflow and action execution where RBAC rules did not get applied #5764

  Contributed by @m4dcoder

• Add backward compatibility to secret masking introduced in #5319 to prevent security-relative issues.
  Migration to the new schema is required to take advantage of the full output schema validation. #5783

  Contributed by @m4dcoder

Added

• Added graceful shutdown for workflow engine. #5463
  Contributed by @khushboobhatia01
• Add ST2_USE_DEBUGGER env var as alternative to the --use-debugger cli flag. #5675
  Contributed by @cognifloyd
• Added purging of old tokens. #5679
  Contributed by Amanda McGuinness (@amanda11 intive)
• Begin introducing pants <https://www.pantsbuild.org/docs>_ to improve DX (Developer Experience)
  working on StackStorm, improve our security posture, and improve CI reliability thanks in part
  to pants’ use of PEX lockfiles. This is not a user-facing addition. #5713 #5724 #5726 #5725 #5732 #5733 #5737 #5738 #5758 #5751 #5774 #5776 #5777 #5782
  Contributed by @cognifloyd

Changed

• BREAKING CHANGE for anyone that uses output_schema, which is disabled by default.
  If you have [system].validate_output_schema = True in st2.conf AND you have added
  output_schema to any of your packs, then you must update your action metadata.

  output_schema must be a full jsonschema now. If a schema is not well-formed, we ignore it.
  Now, output can be types other than object such as list, bool, int, etc.
  This also means that all of an action’s output can be masked as a secret.

  To get the same behavior, you’ll need to update your output schema.
  For example, this schema:

    output_schema:
      property1:
        type: bool
      property2:
        type: str

should be updated like this:

    output_schema:
      type: object
      properties:
        property1:
          type: bool
        property2:
          type: str
      additionalProperties: false

#5319

Contributed by @cognifloyd

• Changed the X-XSS-Protection HTTP header from 1; mode=block to 0 in the conf/nginx/st2.conf to align with the OWASP security standards. #5298

  Contributed by @LiamRiddell

• Use PEP 440 direct reference requirements instead of legacy PIP VCS requirements. Now, our *.requirements.txt files use
  package-name@ git+https://url@version ; markers instead of git+https://url@version#egg=package-name ; markers. #5673
  Contributed by @cognifloyd
• Move from udatetime to ciso8601 for date functionality ahead of supporting python3.9 #5692
  Contributed by Amanda McGuinness (@amanda11 intive)
• Refactor tests to use python imports to identify test fixtures. #5699 #5702 #5703 #5704 #5705 #5706
  Contributed by @cognifloyd
• Refactor st2-generate-schemas so that logic is in an importable module. #5708
  Contributed by @cognifloyd

Removed

• Removed st2exporter service. It is unmaintained and does not get installed. It was
  originally meant to help with analytics by exporting executions as json files that
  could be imported into something like elasticsearch. Our code is now instrumented
  to make a wider variety of stats available to metrics drivers. #5676
  Contributed by @cognifloyd

Install && Use

Copyright 2014-2018 StackStorm, Inc.