How do you override provider versions in child modules?

[ShankyJS]

Context: gruntwork-io/terragrunt#2530

This is related to using mono-repo like environments where providers might not be the same for all your configurations, e.g

└── aws
    ├── dev
    │   ├── account.hcl
    │   └── infra
    │       ├── eks
    │       │   └── terragrunt.hcl
    │       ├── eks_addons
    │       │   └── terragrunt.hcl
    │       └── vpc
    │           └── terragrunt.hcl
    ├── prod
    │   ├── account.hcl
    │   ├── infra
    │   │   ├── eks
    │   │   │   └── terragrunt.hcl
    │   │   └── eks_addons
    │   │       └── terragrunt.hcl
    │   └── networking
    │       └── vpc
    │           └── terragrunt.hcl
    | └── terragrunt.hcl

So let's say that terragrunt.hcl (root level) includes the following providers:

generate "versions" {
  path      = "versions.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "5.46.0"
    }
    random = {
      source = "hashicorp/random"
      version = "3.5.1"
    }
  }
}
EOF
}

And let's say that the eks_addons needs

generate "versions_overwrite" {
  path      = "versions.tf"
  if_exists = "overwrite"
  contents  = <<EOF
    terraform {
      required_providers {
        aws = {
          source  = "hashicorp/aws"
          version = ">= 5.0"
        }
        helm = {
          source  = "hashicorp/helm"
          version = ">= 2.9"
        }
        kubernetes = {
          source  = "hashicorp/kubernetes"
          version = ">= 2.20"
        }
        time = {
          source  = "hashicorp/time"
          version = ">= 0.9"
        }
      }
    }
EOF
}

Because I have the overwrite in both places; Terragrunt actually is fighting over which overwrites at the end which sometimes results in errors like:

╷
│ Error: Failed to query available provider packages
│
│ Could not retrieve the list of available versions for provider

Or

│ Error: Failed to query available provider packages
│
│ Could not retrieve the list of available versions for provider
│ hashicorp/random: locked provider registry.terraform.io/hashicorp/random
│ 3.6.2 does not match configured version constraint 3.5.1; must use
│ terraform init -upgrade to allow selection of new versions

Just because a single module requires more providers it shouldn't mean that I need to add the provider to the root terragrunt, then that means all my modules require now to use the same providers, even if they don't need it and the worst case is that we can't properly test provider upgrades if we can't successfully override versions on child modules.

How do y'all handle this?

---

Tracked in ticket #111329

[odgrim]

Hello Jhan! Terraform actually has something to handle this built in. You're looking for override files.

In this case, your versions.tf for eks_addons just needs to be named "versions_override.tf". From there terraform/tofu will do a block level merge of all of that blocks parameters. The specifics for how the terraform block gets merged is at the bottom of the docs here.

Does that solve your problem?

[ShankyJS]

Hey @odgrim I will give this a try and let you know!

It would be great if there could be some documentation on precedence with Terragrunt when overriding files like this, too!

Thanks.

[ShankyJS]

Hey @odgrim I can confirm this works as expected!

Thanks