puppet-st2

Table of Contents

1. Description
2. Setup - The basics of getting started with st2
   • What st2 affects
   • Setup requirements
   • Beginning with st2
3. Usage - Configuration options and additional functionality
   • Reference Documentation
   • Configuration
     • Profiles
     • Installing and Configuring Packs
     • Configuring Authentication
     • Configuring ChatOps
   • Tasks
     • Task List
     • Task Authentication
     • Using Tasks With API Key
     • Using Tasks With Auth Tokens
     • Using Tasks With Username and Password
4. Limitations - OS compatibility, etc.
   • Supported Platforms
   • Supported Puppet versions
   • Upgrading StackStorm
5. Development - Guide for contributing to the module
   • Maintainers
   • Help

Description

Module to manage StackStorm with Puppet.

Setup

What st2 Affects

The st2 module configures the existing into a complete and dedicated StackStorm node with the following components:

• StackStorm
• MongoDB
• Postgres
• RabbitMQ
• Nginx
• NodeJS

Setup Requirements

This module, similar to normal StackStorm installs, expects to be run on a blank system without any existing configurations. The only hard requirements are on the Operating System and machine specs. See Limitations and the official StackStorm system requirements.

Module Dependencies

This module installs and configures all of the components required for StackStorm. In order to not repeat others work, we've utilized many existing modules from the forge. We manage the module dependenies using a Puppetfile for each OS we support. These Puppetfile can be used both with r10k and librarian-puppet.

• RHEL/CentOS 7 - Puppet 5 - build/centos7-puppet5/Puppetfile
• RHEL/CentOS 7 - Puppet 6 - build/centos7-puppet6/Puppetfile
• Ubuntu 16.04 - Puppet 5 - build/ubuntu16-puppet5/Puppetfile
• Ubuntu 16.04 - Puppet 6 - build/ubuntu16-puppet6/Puppetfile
• Ubuntu 18.04 - Puppet 5 - build/ubuntu18-puppet5/Puppetfile
• Ubuntu 18.04 - Puppet 6 - build/ubuntu18-puppet6/Puppetfile

Beginning with st2

For a full installation on a single node, a profile already exists to get you setup and going with minimal effort. Simply:

puppet module install stackstorm-st2
puppet apply -e "include st2::profile::fullinstall"

Usage

Reference Documentation

This module uses Puppet Strings as the documentation standard. An live version is available online at puppetmodule.info/m/stackstorm-st2. A markdown version is available directly in this repo in REFERENCE.md.

Configuration

This module aims to provide sane default configurations, but also stay out of your way in the event you need something more custom. To accomplish this, this module uses the Roles/Profiles pattern. Included in this module are several modules that come with sane defaults that you can use directly or use to compose your own site-specific profile for StackStorm installation.

Configuration can be done directly via code composition, or set via Hiera data bindings. A few notable parameters to take note of:

• st2::version - Version of ST2 to install. This will be set as the ensure value on the st2 packages. The default is present resulting in the most up to date packages being installed initially. If you would like to hard code to an older version you can specify that here (ex: 2.6.0). Note Setting this to latest is NOT recommended. It will cause the StackStorm packages to be automatically updated without the proper upgrade steps being taken (proper steps detailed here: https://docs.stackstorm.com/install/upgrades.html)

All other classes are documented with Puppetdoc. Please refer to specific classes for use and configuration.

Profiles

• st2::profile::client - Profile to install all client libraries for st2
• st2::profile::fullinstall - Full installation of StackStorm and dependencies
• st2::profile::mistral - Install of OpenStack Mistral
• st2::profile::mongodb - st2 configured MongoDB installation
• st2::profile::nodejs - st2 configured NodeJS installation
• st2::profile::python - Python installed and configured for st2
• st2::profile::rabbitmq - st2 configured RabbitMQ installation
• st2::proflle::server - st2 server components
• st2::profile::web - st2 web components
• st2::profile::chatops - st2 chatops components

Installing and Configuring Packs

StackStorm packs can be installed and configured directly from Puppet. This can be done via the st2::pack and st2::pack::config defined types.

Installation/Configuration via modules:

  # install pack from the exchange
  st2::pack { 'linux': }
  
  # install pack from a git URL
  st2::pack { 'private':
    repo_url => 'https://private.domain.tld/git/stackstorm-private.git',
  }
  
  # install pack and apply configuration
  st2::pack { 'slack':
    config   => {
      'post_message_action' => {
        'webhook_url' => 'XXX',
      },
    },
  }

Installation/Configuration via Hiera:

st2::packs:
  linux:
    ensure: present
  private:
    ensure: present
    repo_url: https://private.domain.tld/git/stackstorm-private.git
  slack:
    ensure: present
    config:
      post_message_action:
        webhook_url: XXX

Configuring Authentication

StackStorm uses a pluggable authentication system where auth is delegated to an external service called a "backend". The st2auth service can be configured to use various backends (only one active). For more information on StackStorm authentication see the authentication documentation page.

The following backends are currently available:

• flat_file - Authenticates against an htpasswd file (default) link
• keystone - Authenticates against an OpenStack Keystone service link
• ldap - Authenticates against an LDAP server such as OpenLDAP or Active Directory link
• mongodb - Authenticates against a collection named users in MongoDB link
• pam - Authenticates against the PAM Linux service link

By default the flat_file backend is used. To change this you can configure it when instantiating the ::st2 class in a manifest file:

class { 'st2':
  auth_backend => 'ldap',
}

Or in Hiera:

st2::auth_backend: ldap

Each backend has their own custom configuration settings. The settings can be found by looking at the backend class in the manifests/st2/auth/ directory. These parameters map 1-for-1 to the configuration options defined in each backends GitHub page (links above). Backend configurations are passed in as a hash using the auth_backend_config option. This option can be changed when instantiating the ::st2 class in a manifest file:

class { 'st2':
  auth_backend        => 'ldap',
  auth_backend_config => {
    ldap_uri      => 'ldaps://ldap.domain.tld',
    bind_dn       => 'cn=ldap_stackstorm,ou=service accounts,dc=domain,dc=tld',
    bind_pw       => 'some_password',
    ref_hop_limit => 100,
    user          => {
      base_dn       => 'ou=domain_users,dc=domain,dc=tld',
      search_filter => '(&(objectClass=user)(sAMAccountName={username})(memberOf=cn=stackstorm_users,ou=groups,dc=domain,dc=tld))',
      scope         => 'subtree'
    },
  },
}

Or in Hiera:

st2::auth_backend: ldap
st2::auth_backend_config:
  ldap_uri: "ldaps://ldap.domain.tld"
  bind_dn: "cn=ldap_stackstorm,ou=service accounts,dc=domain,dc=tld"
  bind_pw: "some_password"
  ref_hop_limit: 100
  user:
    base_dn: "ou=domain_users,dc=domain,dc=tld"
    search_filter: "(&(objectClass=user)(sAMAccountName={username})(memberOf=cn=stackstorm_users,ou=groups,dc=domain,dc=tld))"
    scope: "subtree"

Configuring ChatOps

Configuration via Hiera:

  # character to trigger the bot that the message is a command
  # example: !help
  st2::chatops_hubot_alias: "'!'"
  
  # name of the bot in chat, sometimes requires special characters like @
  st2::chatops_hubot_name: '"@RosieRobot"'
  
  # API key generated by: st2 apikey create
  st2::chatops_api_key: '"xxxxyyyyy123abc"'
 
  # Public URL used by ChatOps to offer links to execution details via the WebUI.
  st2::chatops_web_url: '"stackstorm.domain.tld"'
  
  # install and configure hubot adapter (rocketchat, nodejs module installed by nodejs)
  st2::chatops_adapter:
    hubot-adapter:
      package: 'hubot-rocketchat'
      source: 'git+ssh://[email protected]:npm/hubot-rocketchat#master'

  # adapter configuration (hash)
  st2::chatops_adapter_conf:
    HUBOT_ADAPTER: rocketchat
    ROCKETCHAT_URL: "https://chat.company.com:443"
    ROCKETCHAT_ROOM: 'stackstorm'
    LISTEN_ON_ALL_PUBLIC: true
    ROCKETCHAT_USER: st2
    ROCKETCHAT_PASSWORD: secret123
    ROCKETCHAT_AUTH: password
    RESPOND_TO_DM: true

Tasks

This module provides several tasks for interacting with StackStorm. These tasks are modeled after the st2 CLI command, names of the tasks and parameters reflect this. Under the hood, the tasks invoke the st2 CLI command so they must be executed on a node where StackStorm is installed.

Task List

• st2::key_decrypt - Decrypts an encrypted key/value pair
• st2::key_get - Retrieves the value for a key from the datastore
• st2::key_load - Loads a list of key/value pairs into the datastore
• st2::pack_install - Installs a list of packs
• st2::pack_list - Get a list of installed packs
• st2::pack_register: Registers a list of packs based from paths on the filesystem
• st2::pack_remove - Removes a list of packs
• st2::rule_disable: Disables a rule
• st2::rule_list: Lists all rules, or just the rules in a given pack
• st2::run: Runs a StackStorm action

Task Authentication

Tasks that interact with the st2 CLI command require authentication with the StackStorm instance. There are three options for authentication:

• API Key
• Auth token
• Username/password

Using Tasks With API Key

API keys are the recommended way for systems to authenticate with StackStorm. To do this via a task, you would first create an API key in StackStorm:

$ st2 apikey create -m '{"used_by": "bolt"}'

Copy the API key parameter in the output, and then use it when invoking one of the tasks in this module via the api_key parameter:

Usage via command line:

bolt task run st2::key_get key="testkey" api_key='xyz123'

Usage in a plan:

$res = run_task('st2::key_get', $stackstorm_target,
                key        => 'testkey',
                api_key    => $api_key)

Using Tasks With Auth Tokens

Auth tokens can be used by bolt to communicate with StackStorm. First, the user needs to create an auth token, then pass it in via the auth_token parameter

$ st2 auth myuser

Copy the auth token in the output, and then use it when invoking one of the tasks in this module:

Usage via command line:

bolt task run st2::key_get key="testkey" auth_token='xyz123'

Usage in a plan:

$res = run_task('st2::key_get', $stackstorm_target,
                key        => 'testkey',
                auth_token => $auth_token)

Using Tasks With Username and Password

Finally bolt can accept username/passwords to communicate with StackStorm.

Usage via command line:

bolt task run st2::key_get key="testkey" username="myuser" password="xyz123"

Usage in a plan:

$res = run_task('st2::key_get', $stackstorm_target,
                key      => 'testkey',
                username => $username,
                password => $password)

Limitations

Supported platforms

• Ubuntu 16.04
• Ubuntu 18.04
• RHEL/CentOS 7

Supported Puppet versions

• Puppet 5
• Puppet 6

⚠️ End-of-Support Notice - Mistral

Support for Mistral has been dropped as of StackStorm 3.3.0.

As of version 1.8 this module no longer supports Mistral (and subsequently PostgreSQL) Neither Mistral nor Postgresql will be installed or managed by this module.

⚠️ End-of-Support Notice - CentOS 6

Support for CentOS 6 has been dropped as of StackStorm 3.3.0.

As of version 1.8 this module no longer supports CentOS 6, so changes will not be tested against this platform.

⚠️ Deprecation Notice - Puppet 4

Puppet 4 reached End of Life on 2018-12-31. As of version 1.4 use of Puppet 4 with this module is officially deprecated.

• As of version 1.5.0 this module no longer tests against Puppet 4 in its build matrix.
• The next major release of the module will drop support for Puppet 4 by adjusting the minimum supported Puppet version in metadata.json.

⚠️ Deprecation Notice - Puppet 3

This module no longer supports Puppet 3 as of version 1.1

Upgrading StackStorm

By default this module does NOT handle upgrades of StackStorm. It is the responsiblity of the end user to upgrade StackStorm according to the upgrade documenation.

In a future release a Puppet task may be included to perform these update on demand using bolt.

Development

Contributions to this module are more than welcome! If you have a problem with the module or would like to see a new feature, please raise an issue. If you are amazing, find a bug or implement a new feature and want to add it to the module, please submit a Pull Request.

Maintainers

• Nick Maludy
  • GitHub - @nmaludy
• StackStorm [email protected]
• James Fryman
• Patrick Hoolboom

Help

If you're in stuck, our community always ready to help, feel free to:

• Ask questions in our public Slack channel in channel #puppet
• Report bug, provide feature request or just give us a ✮ star

Your contribution is more than welcome!