Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add eks cluster gha #112

Merged
merged 52 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from 47 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
9aef1b3
feat: add eks cluster gha
leiicamundi Sep 11, 2024
afd8508
don't install if already installed
leiicamundi Sep 11, 2024
5cee55c
add some params
leiicamundi Sep 11, 2024
794cfe8
copy missing backend
leiicamundi Sep 11, 2024
df35114
add missing parameters
leiicamundi Sep 11, 2024
1bcf4e3
fix param
leiicamundi Sep 11, 2024
65ae75a
fix param
leiicamundi Sep 11, 2024
a99a8fb
add tmate debug
leiicamundi Sep 12, 2024
8e35b9e
tmp disable tests
leiicamundi Sep 12, 2024
9e35eba
detach
leiicamundi Sep 12, 2024
56838a7
fix
leiicamundi Sep 12, 2024
b31cb08
fix delete cluster
leiicamundi Sep 12, 2024
11b0485
add utility acton
leiicamundi Sep 12, 2024
5de0684
fix missing action
leiicamundi Sep 12, 2024
921c4e7
fix action
leiicamundi Sep 12, 2024
8746379
add aurora in the tests
leiicamundi Sep 12, 2024
e0954f5
cidrs and vpc is now optional for aurora
leiicamundi Sep 12, 2024
eb874ad
fix inded
leiicamundi Sep 12, 2024
67eebe7
add default cidrs
leiicamundi Sep 12, 2024
7b6c58a
fix tags
leiicamundi Sep 12, 2024
607c01d
remove tags
leiicamundi Sep 12, 2024
fc8ba7d
default vpc
leiicamundi Sep 12, 2024
f8228bf
default vpc
leiicamundi Sep 12, 2024
0f7dba7
fix
leiicamundi Sep 12, 2024
67fd689
subnet is now optionnal
leiicamundi Sep 12, 2024
0244eb2
integrate aurora gha in eks
leiicamundi Sep 13, 2024
9d329b9
fix index
leiicamundi Sep 13, 2024
e9e6805
ensure output is single line
leiicamundi Sep 13, 2024
00133d8
fix tee
leiicamundi Sep 13, 2024
9e39163
fix values
leiicamundi Sep 13, 2024
3318c31
fix types
leiicamundi Sep 13, 2024
3290687
fix inputs
leiicamundi Sep 13, 2024
2182d64
fix availability zones
leiicamundi Sep 13, 2024
6e9f87c
update action
leiicamundi Sep 13, 2024
f80203e
fix iam roles
leiicamundi Sep 13, 2024
177760a
enforce list of strings
leiicamundi Sep 13, 2024
214d0f3
pin versions and cleanup
leiicamundi Sep 13, 2024
bfff6a3
fix params
leiicamundi Sep 16, 2024
5f40c90
realign doc of gha and params
leiicamundi Sep 16, 2024
633c098
reformat yaml using yamlftm
leiicamundi Sep 16, 2024
ece9e08
use tf-state-key
leiicamundi Sep 16, 2024
cf649ff
auto generate action readmes
leiicamundi Sep 16, 2024
db88d39
use additional variables for non default on eks
leiicamundi Sep 16, 2024
8db575c
remove outdate parameter
leiicamundi Sep 16, 2024
568a642
update pre-commits
leiicamundi Sep 16, 2024
c73b2d9
fix user mapping
leiicamundi Sep 16, 2024
d84fb16
Merge branch 'main' into feature/gha-eks
leiicamundi Sep 16, 2024
8415dae
fix var file order and action generation
leiicamundi Sep 17, 2024
5d961bc
fix inputs tfvars
leiicamundi Sep 17, 2024
07478b3
fix plan file
leiicamundi Sep 17, 2024
737704d
fix deployment
leiicamundi Sep 17, 2024
102008e
use new generation of docs
leiicamundi Sep 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
151 changes: 151 additions & 0 deletions .github/actions/aurora-manage-cluster/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# Deploy RDS Aurora Cluster

## Description

This GitHub Action automates the deployment of an RDS Aurora cluster using Terraform.
This action will also install Terraform and awscli. It will output the Aurora cluster endpoint.


## Inputs

| name | description | required | default |
| --- | --- | --- | --- |
| `cluster-name` | <p>Name of the RDS Aurora cluster to deploy</p> | `true` | `""` |
| `username` | <p>Username for the PostgreSQL admin user</p> | `true` | `""` |
| `password` | <p>Password for the PostgreSQL admin user</p> | `true` | `""` |
| `vpc-id` | <p>VPC ID to create the cluster in</p> | `true` | `""` |
| `subnet-ids` | <p>List of subnet IDs to create the cluster in</p> | `true` | `""` |
| `cidr-blocks` | <p>CIDR blocks to allow access from and to</p> | `true` | `""` |
| `availability-zones` | <p>Array of availability zones to use for the Aurora cluster</p> | `true` | `""` |
| `additional-terraform-vars` | <p>JSON object containing additional Terraform variables</p> | `false` | `{}` |
| `s3-backend-bucket` | <p>Name of the S3 bucket to store Terraform state</p> | `true` | `""` |
| `s3-bucket-region` | <p>Region of the bucket containing the resources states</p> | `false` | `""` |
| `tf-modules-revision` | <p>Git revision of the tf modules to use</p> | `false` | `main` |
| `tf-modules-path` | <p>Path where the tf Aurora modules will be cloned</p> | `false` | `./.action-tf-modules/aurora/` |
| `tf-cli-config-credentials-hostname` | <p>The hostname of a HCP Terraform/Terraform Enterprise instance to place within the credentials block of the Terraform CLI configuration file. Defaults to <code>app.terraform.io</code>.</p> | `false` | `app.terraform.io` |
| `tf-cli-config-credentials-token` | <p>The API token for a HCP Terraform/Terraform Enterprise instance to place within the credentials block of the Terraform CLI configuration file.</p> | `false` | `""` |
| `tf-terraform-version` | <p>The version of Terraform CLI to install. Defaults to <code>latest</code>.</p> | `false` | `latest` |
| `tf-terraform-wrapper` | <p>Whether or not to install a wrapper to wrap subsequent calls of the <code>terraform</code> binary and expose its STDOUT, STDERR, and exit code as outputs named <code>stdout</code>, <code>stderr</code>, and <code>exitcode</code> respectively. Defaults to <code>true</code>.</p> | `false` | `true` |
| `awscli-version` | <p>Version of the aws cli to use</p> | `false` | `2.15.52` |


## Outputs

| name | description |
| --- | --- |
| `aurora-endpoint` | <p>The endpoint of the deployed Aurora cluster</p> |
| `terraform-state-url` | <p>URL of the Terraform state file in the S3 bucket</p> |
| `all-terraform-outputs` | <p>All outputs from Terraform</p> |


## Runs

This action is a `composite` action.

## Usage

```yaml
- uses: ***PROJECT***@***VERSION***
leiicamundi marked this conversation as resolved.
Show resolved Hide resolved
with:
cluster-name:
# Name of the RDS Aurora cluster to deploy
#
# Required: true
# Default: ""

username:
# Username for the PostgreSQL admin user
#
# Required: true
# Default: ""

password:
# Password for the PostgreSQL admin user
#
# Required: true
# Default: ""

vpc-id:
# VPC ID to create the cluster in
#
# Required: true
# Default: ""

subnet-ids:
# List of subnet IDs to create the cluster in
#
# Required: true
# Default: ""

cidr-blocks:
# CIDR blocks to allow access from and to
#
# Required: true
# Default: ""

availability-zones:
# Array of availability zones to use for the Aurora cluster
#
# Required: true
# Default: ""

additional-terraform-vars:
# JSON object containing additional Terraform variables
#
# Required: false
# Default: {}

s3-backend-bucket:
# Name of the S3 bucket to store Terraform state
#
# Required: true
# Default: ""

s3-bucket-region:
# Region of the bucket containing the resources states
#
# Required: false
# Default: ""

tf-modules-revision:
# Git revision of the tf modules to use
#
# Required: false
# Default: main

tf-modules-path:
# Path where the tf Aurora modules will be cloned
#
# Required: false
# Default: ./.action-tf-modules/aurora/

tf-cli-config-credentials-hostname:
# The hostname of a HCP Terraform/Terraform Enterprise instance to place within the credentials block of the Terraform CLI configuration file. Defaults to `app.terraform.io`.
#
# Required: false
# Default: app.terraform.io

tf-cli-config-credentials-token:
# The API token for a HCP Terraform/Terraform Enterprise instance to place within the credentials block of the Terraform CLI configuration file.
#
# Required: false
# Default: ""

tf-terraform-version:
# The version of Terraform CLI to install. Defaults to `latest`.
#
# Required: false
# Default: latest

tf-terraform-wrapper:
# Whether or not to install a wrapper to wrap subsequent calls of the `terraform` binary and expose its STDOUT, STDERR, and exit code as outputs named `stdout`, `stderr`, and `exitcode` respectively. Defaults to `true`.
#
# Required: false
# Default: true

awscli-version:
# Version of the aws cli to use
#
# Required: false
# Default: 2.15.52
```
168 changes: 168 additions & 0 deletions .github/actions/aurora-manage-cluster/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
---
name: Deploy RDS Aurora Cluster

description: |
This GitHub Action automates the deployment of an RDS Aurora cluster using Terraform.
This action will also install Terraform and awscli. It will output the Aurora cluster endpoint.

inputs:
cluster-name:
description: Name of the RDS Aurora cluster to deploy
required: true

username:
description: Username for the PostgreSQL admin user
required: true

password:
description: Password for the PostgreSQL admin user
required: true

vpc-id:
description: VPC ID to create the cluster in
required: true

subnet-ids:
description: List of subnet IDs to create the cluster in
required: true

cidr-blocks:
description: CIDR blocks to allow access from and to
required: true

availability-zones:
description: Array of availability zones to use for the Aurora cluster
required: true

additional-terraform-vars:
description: JSON object containing additional Terraform variables
required: false
default: '{}'

s3-backend-bucket:
description: Name of the S3 bucket to store Terraform state
required: true

s3-bucket-region:
description: Region of the bucket containing the resources states
required: false

tf-modules-revision:
description: Git revision of the tf modules to use
default: main

tf-modules-path:
description: Path where the tf Aurora modules will be cloned
default: ./.action-tf-modules/aurora/

# inherited from https://github.com/hashicorp/setup-terraform/blob/main/action.yml
tf-cli-config-credentials-hostname:
description: The hostname of a HCP Terraform/Terraform Enterprise instance to place within the credentials block of the Terraform CLI configuration
file. Defaults to `app.terraform.io`.
default: app.terraform.io

tf-cli-config-credentials-token:
description: The API token for a HCP Terraform/Terraform Enterprise instance to place within the credentials block of the Terraform CLI configuration
file.
required: false

tf-terraform-version:
description: The version of Terraform CLI to install. Defaults to `latest`.
default: latest

tf-terraform-wrapper:
description: Whether or not to install a wrapper to wrap subsequent calls of the `terraform` binary and expose its STDOUT, STDERR, and exit code
as outputs named `stdout`, `stderr`, and `exitcode` respectively. Defaults to `true`.
default: 'true'

awscli-version:
description: Version of the aws cli to use
# renovate: datasource=github-releases depName=aws/aws-cli
default: 2.15.52

outputs:
aurora-endpoint:
description: The endpoint of the deployed Aurora cluster
value: ${{ steps.apply.outputs.aurora_endpoint }}

terraform-state-url:
description: URL of the Terraform state file in the S3 bucket
value: ${{ steps.utility.outputs.terraform-state-url }}

# Add all terraform outputs dynamically
all-terraform-outputs:
description: All outputs from Terraform
value: ${{ steps.fetch_outputs.outputs.all_terraform_outputs }}

runs:
using: composite
steps:
- name: Use Utility Actions
id: utility
# see https://github.com/orgs/community/discussions/41927 it's not possible to optimize this yet
# steps.uses cannot access the github context.
# uses: ${{ github.action_repository }}/utility-action@${{ github.action_ref }}
# TODO: hard pin this one once it's merged
#uses: "camunda/camunda-tf-eks-module/utility-action@main"
uses: ./.github/actions/utility-action
with:
awscli-version: ${{ inputs.awscli-version }}
terraform-version: ${{ inputs.terraform-version }}

s3-backend-bucket: ${{ inputs.s3-backend-bucket }}
s3-bucket-region: ${{ inputs.s3-bucket-region }}

tf-state-key: terraform/${{ inputs.cluster-name }}/gha/aurora/terraform.tfstate

tf-cli-config-credentials-hostname: ${{ inputs.tf-cli-config-credentials-hostname }}
tf-cli-config-credentials-token: ${{ inputs.tf-cli-config-credentials-token }}
tf-terraform-wrapper: ${{ inputs.tf-terraform-wrapper }}

- name: Checkout Repository Aurora modules
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4
with:
repository: camunda/camunda-tf-eks-module
ref: ${{ inputs.tf-modules-revision }}
path: ${{ inputs.tf-modules-path }}
fetch-depth: 0

- name: Terraform Init
shell: bash
id: init
working-directory: ${{ inputs.tf-modules-path }}/modules/aurora/
run: |
cp ../fixtures/backend.tf ./
terraform version
terraform init -backend-config="bucket=${{ steps.utility.outputs.TFSTATE_BUCKET }}" -backend-config="key=${{ steps.utility.outputs.TFSTATE_KEY }}" -backend-config="region=${{ steps.utility.outputs.TFSTATE_REGION }}"
terraform validate -no-color

- name: Terraform Plan
shell: bash
id: plan
working-directory: ${{ inputs.tf-modules-path }}/modules/aurora/
run: |
terraform plan -no-color -out aurora.plan -var "cluster_name=${{ inputs.cluster-name }}" \
-var "username=${{ inputs.username }}" \
-var "password=${{ inputs.password }}" \
-var 'availability_zones=${{ inputs.availability-zones }}' \
-var "vpc_id=${{ inputs.vpc-id }}" \
-var 'subnet_ids=${{ inputs.subnet-ids }}' \
-var 'cidr_blocks=${{ inputs.cidr-blocks }}' \
-var-file=<(echo '${{ inputs.additional-terraform-vars }}')

- name: Terraform Apply
shell: bash
id: apply
working-directory: ${{ inputs.tf-modules-path }}/modules/aurora/
run: |
terraform apply -no-color aurora.plan
export aurora_endpoint="$(terraform output -raw aurora_endpoint)"
echo "aurora_endpoint=$aurora_endpoint" >> "$GITHUB_OUTPUT"

- name: Fetch Terraform Outputs
shell: bash
id: fetch_outputs
working-directory: ${{ inputs.tf-modules-path }}/modules/aurora/
run: |
all_outputs=$(terraform output -json | jq -c .)
echo "all_terraform_outputs=$all_outputs" | tee -a "$GITHUB_OUTPUT"
57 changes: 57 additions & 0 deletions .github/actions/eks-cleanup-resources/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Delete EKS resources

## Description

This GitHub Action automates the deletion of EKS resources using a shell script.


## Inputs

| name | description | required | default |
| --- | --- | --- | --- |
| `tf-bucket` | <p>Bucket containing the resources states</p> | `true` | `""` |
| `tf-bucket-region` | <p>Region of the bucket containing the resources states, if not set, will fallback on AWS_REGION</p> | `false` | `""` |
| `max-age-hours` | <p>Maximum age of resources in hours</p> | `false` | `20` |
| `target` | <p>Specify an ID to destroy specific resources or "all" to destroy all resources</p> | `false` | `all` |
| `temp-dir` | <p>Temporary directory prefix used for storing resource data during processing</p> | `false` | `./tmp/eks-cleanup/` |


## Runs

This action is a `composite` action.

## Usage

```yaml
- uses: ***PROJECT***@***VERSION***
with:
tf-bucket:
# Bucket containing the resources states
#
# Required: true
# Default: ""

tf-bucket-region:
# Region of the bucket containing the resources states, if not set, will fallback on AWS_REGION
#
# Required: false
# Default: ""

max-age-hours:
# Maximum age of resources in hours
#
# Required: false
# Default: 20

target:
# Specify an ID to destroy specific resources or "all" to destroy all resources
#
# Required: false
# Default: all

temp-dir:
# Temporary directory prefix used for storing resource data during processing
#
# Required: false
# Default: ./tmp/eks-cleanup/
```
Loading
Loading