v0.9.0

Add `settings` and `env` sections. Use latest `terraform-provider-utils` @aknysh (#12)

what

• Add settings and env sections
• Use latest terraform-provider-utils

why

• settings sections are deep-merged and used for consumption by external services (e.g. for Spacelift and Terraform Cloud Terraform modules)
• env sections are deep-merged and used to specify ENV vars for consumption by external services (e.g. for Spacelift and Terraform Cloud)
• Workaround for a deep-merge bug in mergo.Merge(). When deep-merging slice of maps in a for loop,
  mergo modifies the source of the previous loop iteration if it's a complex map and mergo gets a pointer to it,
  not only the destination of the current loop iteration
• Much faster remote-state for Terraform components (at native Go compiled-binary speed)

references

• cloudposse/terraform-provider-utils#16
• cloudposse/terraform-provider-utils#18

test

Given this config:

terraform:
  vars: {}
  settings:
    spacelift:
      workspace_enabled: false
      autodeploy: false
  env:
    ENV_TEST_1: test1
    ENV_TEST_2: test2
    ENV_TEST_3: test3

    aurora-postgres:
      vars:
        instance_type: db.r4.large
        cluster_size: 1
      env:
        ENV_TEST_4: test4
        ENV_TEST_5: test5
        ENV_TEST_6: test6
        ENV_TEST_7: test7

    aurora-postgres-2:
      component: aurora-postgres
      vars:
        instance_type: db.r4.xlarge
      settings:
        spacelift:
          workspace_enabled: true
          autodeploy: true
          branch: "dev"
          triggers: []
      env:
        ENV_TEST_1: test1_override2
        ENV_TEST_2: test2_override2
        ENV_TEST_8: test8

    eks:
      vars:
        spotinst_instance_profile: eg-gbl-dev-spotinst-worker
        spotinst_oceans:
          main:
            desired_group_size: 1
            max_group_size: 3
            min_group_size: 1
            kubernetes_version: null
            ami_release_version: null
            attributes: null
            disk_size: 100
            instance_types: null
            ami_type: "AL2_x86_64"
            tags: null
      settings:
        spacelift:
          workspace_enabled: true
          autodeploy: true
          branch: "test"
          triggers: []
      env:
        ENV_TEST_1: test1_override
        ENV_TEST_2: test2_override
        ENV_TEST_4: test4

it produces the following outputs:

uw2_uat_aurora_postgres_2_settings = {
  "spacelift" = {
    "autodeploy" = true
    "branch" = "dev"
     "triggers" =  []
    "workspace_enabled" = true
  }
}

uw2_dev_aurora_postgres_2_env = {
  "ENV_TEST_1" = "test1_override2"
  "ENV_TEST_2" = "test2_override2"
  "ENV_TEST_3" = "test3"
  "ENV_TEST_4" = "test4"
  "ENV_TEST_5" = "test5"
  "ENV_TEST_6" = "test6"
  "ENV_TEST_7" = "test7"
  "ENV_TEST_8" = "test8"
}

uw2_uat_aurora_postgres_settings = {
  "spacelift" = {
    "autodeploy" = false
    "workspace_enabled" = false
  }
}

uw2_dev_aurora_postgres_env = {
  "ENV_TEST_1" = "test1"
  "ENV_TEST_2" = "test2"
  "ENV_TEST_3" = "test3"
  "ENV_TEST_4" = "test4"
  "ENV_TEST_5" = "test5"
  "ENV_TEST_6" = "test6"
  "ENV_TEST_7" = "test7"
}

uw2_uat_eks_settings = {
  "spacelift" = {
    "autodeploy" = false
    "branch" = "test"
    "triggers" = []
    "workspace_enabled" = true
  }
}

uw2_dev_eks_env = {
  "ENV_TEST_1" = "test1_override"
  "ENV_TEST_2" = "test2_override"
  "ENV_TEST_3" = "test3"
  "ENV_TEST_4" = "test4"
}

v0.8.1

🤖 Automatic Updates

Update README.md and docs @cloudpossebot (#11)

what

This is an auto-generated PR that updates the README.md and docs

why

To have most recent changes of README.md and doc from origin templates

v0.8.0

Use `component` attribute. Bump `terraform-provider-utils` version @aknysh (#9)

what

• Use component attribute in the workspaces of Terraform components that inherit from a base component
• Bump terraform-provider-utils version

why

• The component attribute is used in remote backends to decide whether or not to add the component name to the Terraform workspace name

references

• cloudposse/terraform-provider-utils#16

v0.7.0

Use `terraform-provider-utils` Terraform provider for the module and all submodules @aknysh (#8)

what

• Use terraform-provider-utils Terraform provider for the module and all submodules
• Add examples/stacks example
• Update the module and all submodules to the latest context.tf and terraform-null-label

why

• Speed up the stack processing (especially for SaaS as Spacelift where all components from all stacks are processed at once)
• Return vars and backend configurations for all Terraform and helmfile components for all the provided stacks at the same time
• Support unlimited imports and unlimited levels of imports in YAML configs

references

• cloudposse/terraform-provider-utils#14

test

terraform config

NOTE: In the example below, using version 0.2.0 of the terraform-provider-utils provider, terraform apply finishes in about a second processing all the Terraform and helmfile components from the 4 stacks (including processing/deep-merging of all imports for all stacks, and deep-merging of vars and backend from different config sections)

    module "stacks" {
      source = "cloudposse/stack-config/yaml"
      # version     = "x.x.x"
    
      stack_config_local_path = "./stacks"

      stacks = [
        "uw2-dev",
        "uw2-prod",
        "uw2-staging",
        "uw2-uat"
      ]
    }

outputs

config = [
  {
    "components" = {
      "helmfile" = {
        "alb-controller" = {
          "vars" = {
            "account_number" = "1234567890"
            "chart_values" = {
              "enableCertManager" = true
            }
            "environment" = "uw2"
            "installed" = true
            "namespace" = "eg"
            "region" = "us-west-2"
            "ssm_region" = "us-west-2"
            "stage" = "dev"
          }
        }
        "cert-manager" = {
          "vars" = {
            "account_number" = "1234567890"
            "environment" = "uw2"
            "installed" = true
            "namespace" = "eg"
            "region" = "us-west-2"
            "ssm_region" = "us-west-2"
            "stage" = "dev"
          }
        }
      }
      "terraform" = {
        "aurora-postgres" = {
          "backend" = {
            "acl" = "bucket-owner-full-control"
            "bucket" = "eg-uw2-root-tfstate"
            "dynamodb_table" = "eg-uw2-root-tfstate-lock"
            "encrypt" = true
            "key" = "terraform.tfstate"
            "region" = "us-west-2"
            "role_arn" = "arn:aws:iam::XXXXXXXXXXXX:role/eg-gbl-root-terraform"
            "workspace_key_prefix" = "aurora-postgres"
          }
          "backend_type" = "s3"
          "vars" = {
            "cluster_size" = 1
            "environment" = "uw2"
            "instance_type" = "db.r4.large"
            "namespace" = "eg"
            "region" = "us-west-2"
            "stage" = "dev"
          }
        }
        "aurora-postgres-2" = {
          "backend" = {
            "acl" = "bucket-owner-full-control"
            "bucket" = "eg-uw2-root-tfstate"
            "dynamodb_table" = "eg-uw2-root-tfstate-lock"
            "encrypt" = true
            "key" = "terraform.tfstate"
            "region" = "us-west-2"
            "role_arn" = "arn:aws:iam::XXXXXXXXXXXX:role/eg-gbl-root-terraform"
            "workspace_key_prefix" = "aurora-postgres"
          }
          "backend_type" = "s3"
          "vars" = {
            "cluster_size" = 1
            "environment" = "uw2"
            "instance_type" = "db.r4.xlarge"
            "namespace" = "eg"
            "region" = "us-west-2"
            "stage" = "dev"
          }
        }
        "eks" = {
          "backend" = {
            "acl" = "bucket-owner-full-control"
            "bucket" = "eg-uw2-root-tfstate"
            "dynamodb_table" = "eg-uw2-root-tfstate-lock"
            "encrypt" = true
            "key" = "terraform.tfstate"
            "region" = "us-west-2"
            "role_arn" = "arn:aws:iam::XXXXXXXXXXXX:role/eg-gbl-root-terraform"
            "workspace_key_prefix" = "eks"
          }
          "backend_type" = "s3"
          "vars" = {
            "environment" = "uw2"
            "namespace" = "eg"
            "region" = "us-west-2"
            "region_availability_zones" = [
              "us-west-2b",
              "us-west-2c",
              "us-west-2d",
            ]
            "spotinst_instance_profile" = "eg-gbl-dev-spotinst-worker"
            "spotinst_oceans" = {
              "main" = {
                "ami_release_version" = null
                "ami_type" = "AL2_x86_64"
                "attributes" = null
                "desired_group_size" = 1
                "disk_size" = 100
                "instance_types" = null
                "kubernetes_version" = null
                "max_group_size" = 3
                "min_group_size" = 1
                "tags" = null
              }
            }
            "stage" = "dev"
          }
        }
       "tfstate-backend" = {
          "backend" = {
            "acl" = "bucket-owner-full-control"
            "bucket" = "eg-uw2-root-tfstate"
            "dynamodb_table" = "eg-uw2-root-tfstate-lock"
            "encrypt" = true
            "key" = "terraform.tfstate"
            "region" = "us-west-2"
            "role_arn" = null
            "workspace_key_prefix" = "tfstate-backend"
          }
          "backend_type" = "s3"
          "vars" = {
            "environment" = "uw2"
            "namespace" = "eg"
            "region" = "us-west-2"
            "stage" = "dev"
          }
        }
        "vpc" = {
          "backend" = {
            "acl" = "bucket-owner-full-control"
            "bucket" = "eg-uw2-root-tfstate"
            "dynamodb_table" = "eg-uw2-root-tfstate-lock"
            "encrypt" = true
            "key" = "terraform.tfstate"
            "region" = "us-west-2"
            "role_arn" = "arn:aws:iam::XXXXXXXXXXXX:role/eg-gbl-root-terraform"
            "workspace_key_prefix" = "vpc"
          }
          "backend_type" = "s3"
          "vars" = {
            "availability_zones" = [
              "us-west-2b",
              "us-west-2c",
              "us-west-2d",
            ]
            "cidr_block" = "10.114.0.0/18"
            "environment" = "uw2"
            "namespace" = "eg"
            "region" = "us-west-2"
            "stage" = "dev"
            "subnet_type_tag_key" = "eg.com/subnet/type"
            "vpc_flow_logs_bucket_environment_name" = "uw2"
            "vpc_flow_logs_bucket_stage_name" = "audit"
            "vpc_flow_logs_enabled" = true
            "vpc_flow_logs_traffic_type" = "ALL"
          }
        }
      }
    }
  }
]

v0.6.0

Default include_component_in_workspace_name to false @Nuru (#6)

what

• Change include_component_in_workspace_name default to false

why

• All current clients need include_component_in_workspace_name set to false for s3 backend
• By the time anyone needs this set to true, there will be a different mechanism (stack config itself) for setting it to false

v0.5.0

Update stack variable @aknysh (#5)

what

• If var.stack is not provided, construct the stack name from environment and stage

why

• Simplifies code that calls the module
• The module already accepts environment and stage in context.tf

v0.4.0

Update `remote-state` module @aknysh (#4)

what

• Update remote-state module

why

• No need to read environment and stage from the YAML configs, they are provided as variables to the module
• Make the module faster by not deep-merging the vars sections from YAML configs

v0.3.0

Update `remote-state` module @aknysh (#3)

what

• Update remote-state module

why

• Use the region defined in the backend config (not in general stack config)
• For remote-state module, no need to provide the component since we specify workspace_key_prefix via var.component

v0.2.0

Update `terraform-yaml-config` module. Update S3 and remote backend workspace name calculation @aknysh (#2)

what

• Update terraform-yaml-config module
• Add deepmerge-generate submodule
• Remove README from submodules not supposed to be used separately from the main module
• Update S3 and remote backend workspace name calculation
• Add var.include_component_in_workspace_name

why

• Make deep merging much faster (we don't need map depth of 100, and can easily regenerate it by using deepmerge-generate submodule)

• Use deepmerge-generate submodule to regenerate the max depth for the deepmerge submodule

• If submodules have README.md, Terraform considers them public modules and will show them in the registry

• The namespace name for s3 and remote backends can be provided in the following ways (from the highest to the lowest order of precedence):

  • explicitly using var.workspace
  • from the YAM stack config backend section
  • using format("%s-%s", local.environment, local.stage)

• var.include_component_in_workspace_name controls if we need to include the component name in the workspace name (for backwards compatibility)

references

• cloudposse/terraform-yaml-config#9

0.1.0 Initial implementation

what

• Initial implementation

why

• Terraform module that accepts local or remote YAML stack configurations and returns deep-merged variables, backend config, and remote state outputs for Terraform and helmfile components.

The module is composed of three sub-modules:

• vars - accepts stack configuration and returns deep-merged variables for a Terraform or helmfile component.
• backend - accepts stack configuration and returns backend config for a Terraform component.
• remote-state - accepts stack configuration and returns remote state outputs for a Terraform component. The module supports s3 and remote (Terraform Cloud) backends.