Merge branch 'release/3.10.0'

CHANGELOG.md

@@ -4,6 +4,11 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/)
 and this project adheres to [Semantic Versioning](http://semver.org/).
 
+## 3.10.0 - 2019-07-29
+- Chnaged
+  - THe user data script for the EC2 runner agent instance is not logging anymore on trace level. To enable bash xtrace set `enable_runner_user_data_trace_log` to `true`. #49
+  - Generate links for Readme during release, #63
+
 ## 3.9.0 - 2019-07-26
 - Changed
   - Update default runner version to 12.1.0 (#106)
@@ -154,7 +159,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 - Update default AMI's to The latest Amazon Linux AMI 2017.09.1 - released on 2018-01-17.
 - Minor updates in the example
 
-[Unreleased]: https://github.com/npalm/terraform-aws-gitlab-runner/compare/3.9.0...HEAD
+[Unreleased]: https://github.com/npalm/terraform-aws-gitlab-runner/compare/3.10.0...HEAD
+[3.10.0]: https://github.com/npalm/terraform-aws-gitlab-runner/compare/3.9.0...3.10.0
 [3.9.0]: https://github.com/npalm/terraform-aws-gitlab-runner/compare/3.8.0...3.9.0
 [3.8.0]: https://github.com/npalm/terraform-aws-gitlab-runner/compare/3.7.0...3.8.0
 [3.7.0]: https://github.com/npalm/terraform-aws-gitlab-runner/compare/3.6.0...3.7.0

README.md

@@ -4,7 +4,7 @@
 
 > *WIP*: Work in progress, conversion to Terraform 0.12 \#73. Feel free to checkout branch [Terraform 0.12](https://github.com/npalm/terraform-aws-gitlab-runner/tree/feature/terraform-0.12).
 
-> *NEW*: Multiple instnaces of the runner can be created that share the same cache. See [example](./examples/runner-public) *MIGRATIONS*: Since 3.7 the runner cache is handled by sub module. To avoid re-creation of the bucket while upgrading a state migration is need. Please see the migration script `./migrations/migration-state-3.7.x.sh`
+> *NEW*: Multiple instnaces of the runner can be created that share the same cache. See [example](https://github.com/npalm/terraform-aws-gitlab-runner/tree/3.10.0/examples/runner-public) *MIGRATIONS*: Since 3.7 the runner cache is handled by sub module. To avoid re-creation of the bucket while upgrading a state migration is need. Please see the migration script `./migrations/migration-state-3.7.x.sh`
 
 This [Terraform](https://www.terraform.io/) modules creates a [GitLab CI runner](https://docs.gitlab.com/runner/). A blog post describes the original version of the the runner. See the post at [040code](https://040code.github.io/2017/12/09/runners-on-the-spot/). The original setup of the module is based on the blog post: [Auto scale GitLab CI runners and save 90% on EC2 costs](https://about.gitlab.com/2017/11/23/autoscale-ci-runners/).
 
@@ -112,7 +112,7 @@ Finally, the runner still supports the manual runner creation. No changes are re
 
 By default the module creates a a cache for the runner in S3. Old objects are automatically remove via a configurable life cycle policy on the bucket.
 
-Creation of the bucket can be disabled and managed outside this module. A good use case is for sharing the cache cross multiple runners. For this purpose the cache is implemented as sub module. For more details see the [cache module](./cache). An example implementation of this use case can be find in the [runner-public](./examples/runner-public) example.
+Creation of the bucket can be disabled and managed outside this module. A good use case is for sharing the cache cross multiple runners. For this purpose the cache is implemented as sub module. For more details see the [cache module](./cache). An example implementation of this use case can be find in the [runner-public](https://github.com/npalm/terraform-aws-gitlab-runner/tree/3.10.0/examples/runner-public) example.
 
 ## Usage
 
@@ -130,7 +130,7 @@ The base image used to host the GitLab Runner agent is the latest available Amaz
 
 ### Usage module
 
-Below a basic examples of usages of the module. The dependencies such as a VPC, and SSH keys have a look at the [default example](./examples/runner-default).
+Below a basic examples of usages of the module. The dependencies such as a VPC, and SSH keys have a look at the [default example](https://github.com/npalm/terraform-aws-gitlab-runner/tree/3.10.0/examples/runner-default).
 
 ``` hcl
 
@@ -220,6 +220,7 @@ terraform destroy
 | enable\_cloudwatch\_logging | Boolean used to enable or disable the CloudWatch logging. | string | `"true"` | no |
 | enable\_gitlab\_runner\_ssh\_access | Enables SSH Access to the gitlab runner instance. | string | `"false"` | no |
 | enable\_manage\_gitlab\_token | Boolean to enable the management of the GitLab token in SSM. If `true` the token will be stored in SSM, which means the SSM property is a terraform managed resource. If `false` the Gitlab token will be stored in the SSM by the user-data script during creation of the the instance. However the SSM parameter is not managed by terraform and will remain in SSM after a `terraform destroy`. | string | `"true"` | no |
+| enable\_runner\_user\_data\_trace\_log | Enable bash xtrace for the user data script that creates the EC2 instance for the runner agent. Be aware this could log sensitive data such as you GitLab runner token. | string | `"false"` | no |
 | environment | A name that identifies the environment, used as prefix and for tagging. | string | n/a | yes |
 | gitlab\_runner\_registration\_config | Configuration used to register the runner. See the README for an example, or reference the examples in the examples directory of this repo. | map | `<map>` | no |
 | gitlab\_runner\_ssh\_cidr\_blocks | List of CIDR blocks to allow SSH Access to the gitlab runner instance. | list | `<list>` | no |

_docs/README.md

@@ -2,10 +2,9 @@
 
 # Terraform module for GitLab auto scaling runners on AWS spot instances
 
-
 > *WIP*: Work in progress, conversion to Terraform 0.12 \#73. Feel free to checkout branch [Terraform 0.12](https://github.com/npalm/terraform-aws-gitlab-runner/tree/feature/terraform-0.12).
 
-> *NEW*: Multiple instnaces of the runner can be created that share the same cache. See [example](./examples/runner-public)
+> *NEW*: Multiple instnaces of the runner can be created that share the same cache. See [example](https://github.com/npalm/terraform-aws-gitlab-runner/tree/__GIT_REF__/examples/runner-public)
 > *MIGRATIONS*: Since 3.7 the runner cache is handled by sub module. To avoid re-creation of the bucket while upgrading a state migration is need. Please see the migration script `./migrations/migration-state-3.7.x.sh`
 
 This [Terraform](https://www.terraform.io/) modules creates a [GitLab CI runner](https://docs.gitlab.com/runner/). A blog post describes the original version of the the runner. See the post at [040code](https://040code.github.io/2017/12/09/runners-on-the-spot/). The original setup of the module is based on the blog post: [Auto scale GitLab CI runners and save 90% on EC2 costs](https://about.gitlab.com/2017/11/23/autoscale-ci-runners/).
@@ -114,7 +113,7 @@ Finally, the runner still supports the manual runner creation. No changes are re
 
 By default the module creates a a cache for the runner in S3. Old objects are automatically remove via a configurable life cycle policy on the bucket.
 
-Creation of the bucket can be disabled and managed outside this module. A good use case is for sharing the cache cross multiple runners. For this purpose the cache is implemented as sub module. For more details see the [cache module](./cache). An example implementation of this use case can be find in the [runner-public](./examples/runner-public) example.
+Creation of the bucket can be disabled and managed outside this module. A good use case is for sharing the cache cross multiple runners. For this purpose the cache is implemented as sub module. For more details see the [cache module](./cache). An example implementation of this use case can be find in the [runner-public](https://github.com/npalm/terraform-aws-gitlab-runner/tree/__GIT_REF__/examples/runner-public) example.
 
 ## Usage
 
@@ -132,7 +131,7 @@ The base image used to host the GitLab Runner agent is the latest available Amaz
 
 ### Usage module
 
-Below a basic examples of usages of the module. The dependencies such as a VPC, and SSH keys have a look at the [default example](./examples/runner-default).
+Below a basic examples of usages of the module. The dependencies such as a VPC, and SSH keys have a look at the [default example](https://github.com/npalm/terraform-aws-gitlab-runner/tree/__GIT_REF__/examples/runner-default).
 
 ``` hcl
 

_docs/TF_MODULE.md

@@ -23,6 +23,7 @@
 | enable\_cloudwatch\_logging | Boolean used to enable or disable the CloudWatch logging. | string | `"true"` | no |
 | enable\_gitlab\_runner\_ssh\_access | Enables SSH Access to the gitlab runner instance. | string | `"false"` | no |
 | enable\_manage\_gitlab\_token | Boolean to enable the management of the GitLab token in SSM. If `true` the token will be stored in SSM, which means the SSM property is a terraform managed resource. If `false` the Gitlab token will be stored in the SSM by the user-data script during creation of the the instance. However the SSM parameter is not managed by terraform and will remain in SSM after a `terraform destroy`. | string | `"true"` | no |
+| enable\_runner\_user\_data\_trace\_log | Enable bash xtrace for the user data script that creates the EC2 instance for the runner agent. Be aware this could log sensitive data such as you GitLab runner token. | string | `"false"` | no |
 | environment | A name that identifies the environment, used as prefix and for tagging. | string | n/a | yes |
 | gitlab\_runner\_registration\_config | Configuration used to register the runner. See the README for an example, or reference the examples in the examples directory of this repo. | map | `<map>` | no |
 | gitlab\_runner\_ssh\_cidr\_blocks | List of CIDR blocks to allow SSH Access to the gitlab runner instance. | list | `<list>` | no |

ci/bin/autodocs.sh

@@ -1,5 +1,7 @@
 #!/bin/bash
 
+GIT_REF=${GIT_REF:-develop}
+
 # script to auto-generate terraform documentation
 
 pandoc -v &> /dev/null || { echo >&2 "ERROR: Pandoc not installed" ; exit 1 ; }
@@ -27,6 +29,10 @@ do
 
         # merge the tf docs with the main readme
         pandoc --wrap=none -f gfm -t gfm $docs_dir/README.md -A $docs_dir/TF_MODULE.md > $i/README.md
+
+        # Create a absolute link for terraform registry
+        sed -i ".bak" -e "s|__GIT_REF__|${GIT_REF}|" $i/README.md
+        rm -rf $i/README.md.bak
 
         # do some cleanup
         # because sed on macOS is special..

main.tf

@@ -100,8 +100,9 @@ data "template_file" "user_data" {
   template = "${file("${path.module}/template/user-data.tpl")}"
 
   vars {
-    logging       = "${var.enable_cloudwatch_logging ? data.template_file.logging.rendered : ""}"
-    gitlab_runner = "${data.template_file.gitlab_runner.rendered}"
+    logging             = "${var.enable_cloudwatch_logging ? data.template_file.logging.rendered : ""}"
+    gitlab_runner       = "${data.template_file.gitlab_runner.rendered}"
+    user_data_trace_log = "${var.enable_runner_user_data_trace_log}"
   }
 }
 

tags.tf

@@ -1,7 +1,7 @@
 locals {
   tags = "${merge(map("Name", format("%s", var.environment)),
-              map("Environment", format("%s", var.environment)),
-              var.tags)}"
+    map("Environment", format("%s", var.environment)),
+  var.tags)}"
 
   tags_string = "${replace(replace(jsonencode(local.tags), "/[\\{\\}\"\\s]/", ""), ":", ",")}"
 }

template/user-data.tpl

@@ -1,6 +1,19 @@
-#!/bin/bash -ex
+#!/bin/bash -e
 exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
 
+if [[ `echo ${user_data_trace_log}` == 1 ]] 
+then
+  set -x
+fi
+
+
+# if [[ `echo }` == "docker" ]]
+# then
+#   exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
+# fi #!/bin/bash -ex
+
+
+
 
 # Add current hostname to hosts file
 tee /etc/hosts <<EOL

variables.tf

@@ -395,3 +395,8 @@ variable "cache_bucket" {
     bucket = ""
   }
 }
+
+variable "enable_runner_user_data_trace_log" {
+  description = "Enable bash xtrace for the user data script that creates the EC2 instance for the runner agent. Be aware this could log sensitive data such as you GitLab runner token."
+  default     = false
+}