New terraform registry source not working

[lionslair]

This was discussed on the discord channel but I will copy and paste here for history sake.

I have tried updating to use this new repository as the source but it currently is not working. See below.

can someone please confirm what the source path is for me now.

  source  = "philips-labs/github-runner/aws" 

to

  source  = "github-aws-runners/terraform-aws-github-runner/aws"

Is there more steps I need to do?

yes that should be all, and update the way you got your lambda's if not building yourselves. See https://registry.terraform.io/modules/github-aws-runners/github-runner/aws/latest

I tried

module "github-runner" {
  source  = "github-aws-runners/terraform-aws-github-runner/aws"
  version = "6.1.2"

gives me

╰─$ terraform init                                                                                                                                                               1 ↵
Initializing the backend...
Initializing modules...
Downloading registry.terraform.io/philips-labs/github-runner/aws 6.1.2 for github-runner...
╷
│ Error: Failed to download module
│ 
│   on main.tf line 5:
│    5: module "github-runner" {
│ 
│ Could not download module "github-runner" (main.tf:5) source code from
│ "git::https://github.com/philips-labs/terraform-aws-github-runner?ref=6a4a0e028d5a1705100e37afe353c05c81505090": error downloading
│ 'https://github.com/philips-labs/terraform-aws-github-runner?ref=6a4a0e028d5a1705100e37afe353c05c81505090': /usr/bin/git exited with 128: fatal: reference is not a tree:
│ 6a4a0e028d5a1705100e37afe353c05c81505090
│ .

I think something is broken in the way we version / tag. Not sure what, but getting also errors when using the registry. This will work:

source = "git::https://github.com/github-aws-runners/terraform-aws-github-runner.git?ref=v6.1.2"

Still appears to be pulling from the old repo.

[npalm]

I have just tested the terraform registry, ran a mirgration to mimic before and after the move

Terraform code, based on examples/default

locals {
  environment = var.environment != null ? var.environment : "default"
  aws_region  = var.aws_region
}

resource "random_id" "random" {
  byte_length = 20
}

module "base" {
  source = "../base"

  prefix     = local.environment
  aws_region = local.aws_region
}

module "runners" {
  #source  = "philips-labs/github-runner/aws"
  source  = "github-aws-runners/github-runner/aws"
  version = "6.1.0"

  create_service_linked_role_spot = true
  aws_region                      = local.aws_region
  vpc_id                          = module.base.vpc.vpc_id
  subnet_ids                      = module.base.vpc.private_subnets

  prefix = local.environment
  tags = {
    Project = "ProjectX"
  }

  github_app = {
    key_base64     = var.github_app.key_base64
    id             = var.github_app.id
    webhook_secret = random_id.random.hex
  }

  enable_organization_runners = true
  runner_extra_labels         = ["default", "example"]

  # enable access to the runners via SSM
  enable_ssm_on_runners = true

  instance_types = ["m7a.large", "m5.large"]

  # override delay of events in seconds
  delay_webhook_event   = 5
  runners_maximum_count = 2

  # override scaling down
  scale_down_schedule_expression = "cron(* * * * ? *)"

  enable_user_data_debug_logging_runner = true

  # prefix GitHub runners with the environment name
  runner_name_prefix = "${local.environment}_"

  # by default eventbridge is used, see multi-runner example. Here we disable the eventbridge
  eventbridge = {
    enable = false
  }


  enable_ami_housekeeper = true
  ami_housekeeper_cleanup_config = {
    ssmParameterNames = ["*/ami-id"]
    minimumDaysOld    = 10
    amiFilters = [
      {
        Name   = "name"
        Values = ["*al2023*"]
      }
    ]
  }

  instance_termination_watcher = {
    enable = true
  }

}

module "webhook_github_app" {
  source  = "github-aws-runners/github-runner/aws//modules/webhook-github-app"
  version = "6.1.0"
  depends_on = [module.runners]

  github_app = {
    key_base64     = var.github_app.key_base64
    id             = var.github_app.id
    webhook_secret = random_id.random.hex
  }
  webhook_endpoint = module.runners.webhook.endpoint
}

terraform init

# output
Initializing the backend...
Initializing modules...
Downloading registry.terraform.io/philips-labs/github-runner/aws 6.1.0 for runners...
- runners in .terraform/modules/runners
- runners.ami_housekeeper in .terraform/modules/runners/modules/ami-housekeeper
- runners.instance_termination_watcher in .terraform/modules/runners/modules/termination-watcher
- runners.instance_termination_watcher.termination_handler in .terraform/modules/runners/modules/termination-watcher/termination
- runners.instance_termination_watcher.termination_handler.termination_handler in .terraform/modules/runners/modules/lambda
- runners.instance_termination_watcher.termination_notification in .terraform/modules/runners/modules/termination-watcher/notification
- runners.instance_termination_watcher.termination_notification.termination_warning_watcher in .terraform/modules/runners/modules/lambda
- runners.runner_binaries in .terraform/modules/runners/modules/runner-binaries-syncer
- runners.runners in .terraform/modules/runners/modules/runners
- runners.runners.job_retry in .terraform/modules/runners/modules/runners/job-retry
- runners.runners.job_retry.job_retry in .terraform/modules/runners/modules/lambda
- runners.runners.pool in .terraform/modules/runners/modules/runners/pool
- runners.ssm in .terraform/modules/runners/modules/ssm
- runners.webhook in .terraform/modules/runners/modules/webhook
- runners.webhook.direct in .terraform/modules/runners/modules/webhook/direct
- runners.webhook.eventbridge in .terraform/modules/runners/modules/webhook/eventbridge
Downloading registry.terraform.io/philips-labs/github-runner/aws 6.1.0 for webhook_github_app...
- webhook_github_app in .terraform/modules/webhook_github_app/modules/webhook-github-app

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Reusing previous version of hashicorp/local from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
- Using previously-installed hashicorp/aws v5.82.1
- Using previously-installed hashicorp/local v2.5.2
- Using previously-installed hashicorp/random v3.6.3
- Using previously-installed hashicorp/null v3.2.3

Next ran

cd .terraform/modules/runners/lambdas
yarn install --frozen-lockfile && yarn run dist
cd -
terraform apply

Result deployed module

Next updated sources to

...
  source  = "github-aws-runners/github-runner/aws"
  version = "6.1.0"
...
  source  = "github-aws-runners/github-runner/aws//modules/webhook-github-app"
  version = "6.1.0"
...

Ran

terraform init

# output
Downloading registry.terraform.io/github-aws-runners/github-runner/aws 6.1.0 for runners...
- runners in .terraform/modules/runners
- runners.ami_housekeeper in .terraform/modules/runners/modules/ami-housekeeper
- runners.instance_termination_watcher in .terraform/modules/runners/modules/termination-watcher
- runners.instance_termination_watcher.termination_handler in .terraform/modules/runners/modules/termination-watcher/termination
- runners.instance_termination_watcher.termination_handler.termination_handler in .terraform/modules/runners/modules/lambda
- runners.instance_termination_watcher.termination_notification in .terraform/modules/runners/modules/termination-watcher/notification
- runners.instance_termination_watcher.termination_notification.termination_warning_watcher in .terraform/modules/runners/modules/lambda
- runners.runner_binaries in .terraform/modules/runners/modules/runner-binaries-syncer
- runners.runners in .terraform/modules/runners/modules/runners
- runners.runners.job_retry in .terraform/modules/runners/modules/runners/job-retry
- runners.runners.job_retry.job_retry in .terraform/modules/runners/modules/lambda
- runners.runners.pool in .terraform/modules/runners/modules/runners/pool
- runners.ssm in .terraform/modules/runners/modules/ssm
- runners.webhook in .terraform/modules/runners/modules/webhook
- runners.webhook.direct in .terraform/modules/runners/modules/webhook/direct
- runners.webhook.eventbridge in .terraform/modules/runners/modules/webhook/eventbridge
Downloading registry.terraform.io/github-aws-runners/github-runner/aws 6.1.0 for webhook_github_app...
- webhook_github_app in .terraform/modules/webhook_github_app/modules/webhook-github-app

Initializing provider plugins...
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/aws from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
- Reusing previous version of hashicorp/local from the dependency lock file
- Using previously-installed hashicorp/local v2.5.2
- Using previously-installed hashicorp/random v3.6.3
- Using previously-installed hashicorp/aws v5.82.1
- Using previously-installed hashicorp/null v3.2.3

Terraform has been successfully initialized!

Applying (after rebuild lambdas)

terraorm apply

Result module deployed. In this case a change is applied because I not kept a lambda copy, but did a rebuild.

Conclusion is the error is magically gone.