Stack Orchestration Tools

A stack orchestration tool is usually a custom set of scripts written by a team to run their stack management tools. They may be written as Makefiles, shell scripts, batchfiles, or using more advanced scripting languages such as Ruby or Python. In some cases, teams develop complex tools to serve this purpose.

Why these are used

Some typical scenarios:

  • To orchestrate multiple stacks. For example, when systems are split across multiple stacks, scripts may manage the order in which they are provisioned and updated. They may also manage dependencies between the stacks.
  • To orchestrate different layers of infrastructure tools. One tool may be used to provision a stack, and then other tools may be run to configure servers within the stack, deploy applications, load data, etc.
  • To handle configuration of stack instances. In more complex configuration scenarios, parameter values may be built up from different levels of defaults depending on environment, location, products, etc. The orchestration tooling may manage the retrieval and resolution of these and provide a resolved set of parameters to each stack instance.
  • To manage versioning of stack project code, perhaps building, uploading, downloading, and unpacking artefacts.
  • Run tests, smoke tests, and other validation checks.

Pitfalls

These orchestration tools can become complicated and messy. In many cases, maintaining the scripts and code that run the infrastructure tools is more work than maintaining the code that actually defines the infrastructure.

This may be a smell that indicates design issues, in which case it may be a good idea to reconsider the overall infrastructure design and look for ways to simplify the dependencies and configuration.

A common issue is that orchestration code often lacks automated tests. If this code is at all complex - if there is very much of this code, if it’s spread across multiple files, if it’s difficult to understand, if debugging and fixing errors takes very much time - then it should have tests to drive cleaner design and reliability of changes.

Example tools

Terragrunt. There are a number of tools for CloudFormation.

A separate discussion is more procedural scripting tools like Pulomi, vs. “classic” declarative tools.