Dbp 000 Hotfixes 3 (#82)

* rename scripts consistently
* change operation to "replace" (from "add") when restoring moodle probes, so that there is only one probe
* change var release_name to more appropriate deployment_name
* rename value update_migration.enabled to moodleUpdatePreparationJob.enabled, since that value already exists
* rename scripts consistently
* split up value for image into repository, image and tag where possible
* change values for postregs and mariadb password & admin password (each) to be the same value now called database_password and database_root_password. (still refered to as mariadb-password in secret key). Also added a note in the secret specifying why that is
* add helper for cronjob name
* change etherpad_api_key helper to default to random alpha numeric string instead of moodle
* add value dbpMoodle.backup.s3_certificate_path & dbpMoodle.backup.s3_certificate_key to be able to include a certificate for s3 endpoints with self signed certificates for backup/restore jobs
* add value dbpMoodle.backup.s3_certificate_path & dbpMoodle.backup.s3_certificate_key to be able to include a certificate for s3 endpoints with self signed certificates for backup/restore jobs
* bump version so 0.0.13
* update values and insert cert for restore

charts/dbp-moodle/Chart.yaml

@@ -6,7 +6,7 @@ description: |
   The Chart can be deployed without any modification but it is advised to set own secrets acccording to this readme.
 type: application
 home: https://dbildungsplattform.github.io/dbp-moodle/
-version: 0.0.12
+version: 0.0.13
 appVersion: "4.1.14"
 dependencies:
   - name: moodle

charts/dbp-moodle/README.md

@@ -1,6 +1,6 @@
 # dbp-moodle
 
-![Version: 0.0.12](https://img.shields.io/badge/Version-0.0.12-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 4.1.14](https://img.shields.io/badge/AppVersion-4.1.14-informational?style=flat-square)
+![Version: 0.0.13](https://img.shields.io/badge/Version-0.0.13-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 4.1.14](https://img.shields.io/badge/AppVersion-4.1.14-informational?style=flat-square)
 
 This is a Helm Chart bundling some of the bitnami resources to deploy Moodle for DBildungsplattform. Extending them with features such as
 MariaDB and PostgreSQL support, Horizontal Autoscaling capabilities, Redis Session Store, Etherpad-Lite.
@@ -72,7 +72,7 @@ The Chart can be deployed without any modification but it is advised to set own
 | backup-cronjob.extraVolumes[2].projected.sources[0].configMap.name | string | `"moodle-backup-duply"` |  |
 | backup-cronjob.extraVolumes[2].projected.sources[1].secret.name | string | `"moodle-backup-gpg-keys"` |  |
 | backup-cronjob.image.repository | string | `"ghcr.io/dbildungsplattform/moodle-tools"` |  |
-| backup-cronjob.image.tag | string | `"1.0.7"` |  |
+| backup-cronjob.image.tag | string | `"1.0.8"` |  |
 | backup-cronjob.jobs[0].args[0] | string | `"/scripts/backup-script"` |  |
 | backup-cronjob.jobs[0].command[0] | string | `"/bin/sh"` |  |
 | backup-cronjob.jobs[0].command[1] | string | `"-c"` |  |
@@ -96,10 +96,12 @@ The Chart can be deployed without any modification but it is advised to set own
 | backup-cronjob.serviceAccount.name | string | `"moodle-backup-job"` |  |
 | backup-cronjob.tolerations | list | `[]` |  |
 | dbpMoodle.allowInternalNetworkingOnly | bool | `false` | disallows all egress from release namespace for the moodle deployment |
-| dbpMoodle.backup | object | `{"cluster_name":"","enabled":false,"endpoint":"","gpg_key_names":"","gpgkeys":{"existingSecret":"","gpgkey.dbpinfra.pub.asc":"","gpgkey.dbpinfra.sec.asc":""},"max_full_backup_age":"1W","retention_time":"6M","rules":[{"apiGroups":["apps"],"resources":["deployments"],"verbs":["get","patch","list","watch"]},{"apiGroups":["batch"],"resources":["cronjobs","jobs"],"verbs":["get","patch"]}],"s3_bucket_name":"","secrets":{"existingSecret":"","s3_access_key":"","s3_access_secret":"","s3_endpoint_url":""}}` | Backup configuration. Set enabled=true to enable the backup-cronjob. Also set s3 location credentials |
+| dbpMoodle.backup | object | `{"cluster_name":"","enabled":false,"endpoint":"","gpg_key_names":"","gpgkeys":{"existingSecret":"","gpgkey.dbpinfra.pub.asc":"","gpgkey.dbpinfra.sec.asc":""},"max_full_backup_age":"1W","retention_time":"6M","rules":[{"apiGroups":["apps"],"resources":["deployments"],"verbs":["get","patch","list","watch"]},{"apiGroups":["batch"],"resources":["cronjobs","jobs"],"verbs":["get","patch"]}],"s3_bucket_name":"","s3_certificate_secret":{"enabled":false,"key":"certificate.crt","mountpath":"/certs","name":"s3-certificate"},"secrets":{"existingSecret":"","s3_access_key":"","s3_access_secret":"","s3_endpoint_url":""}}` | Backup configuration. Set enabled=true to enable the backup-cronjob. Also set s3 location credentials |
 | dbpMoodle.backup.gpgkeys.existingSecret | string | `""` | Existing  secret for gpg keys |
 | dbpMoodle.backup.max_full_backup_age | string | `"1W"` | Defines the maximum age of a full backup before a new full backup is created. The backups in between are incremental |
 | dbpMoodle.backup.retention_time | string | `"6M"` | Defines the maximum age of a backup before it is deleted |
+| dbpMoodle.backup.s3_certificate_secret | object | `{"enabled":false,"key":"certificate.crt","mountpath":"/certs","name":"s3-certificate"}` | Secret key of a certificate for duply to connect to s3 endpoint using SSL, useful to trust self-signed certificates -- certificate has to mounted "manually" under values backup-cronjob |
+| dbpMoodle.backup.s3_certificate_secret.key | string | `"certificate.crt"` | Path where the certificate is mounted |
 | dbpMoodle.backup.secrets | object | `{"existingSecret":"","s3_access_key":"","s3_access_secret":"","s3_endpoint_url":""}` | Either provide an existing secret, or set each secret value here. If both are set the existingSecret is used |
 | dbpMoodle.backup.secrets.existingSecret | string | `""` | Existing secret for s3 endpoint |
 | dbpMoodle.external_pvc.accessModes[0] | string | `"ReadWriteMany"` |  |
@@ -125,19 +127,18 @@ The Chart can be deployed without any modification but it is advised to set own
 | dbpMoodle.moodleUpdatePreparationHook.rules[1].verbs[2] | string | `"create"` |  |
 | dbpMoodle.moodleUpdatePreparationHook.rules[1].verbs[3] | string | `"patch"` |  |
 | dbpMoodle.moodleUpdatePreparationHook.rules[1].verbs[4] | string | `"watch"` |  |
-| dbpMoodle.moodleUpdatePreparationJob | object | `{"affinity":{},"kubectlImage":"bitnami/kubectl:1.30.4-debian-12-r3","resources":{},"tolerations":[]}` | A preperation job which disables the php-cronjob, scales down the deployment and creates a backup if dbpMoodle.backup.enabled=true |
-| dbpMoodle.moodleUpdatePreparationJob.kubectlImage | string | `"bitnami/kubectl:1.30.4-debian-12-r3"` | Which kubectl image to use |
+| dbpMoodle.moodleUpdatePreparationJob | object | `{"affinity":{},"enabled":false,"image":"moodle-tools","repository":"ghcr.io/dbildungsplattform","resources":{},"tag":"1.0.8","tolerations":[]}` | A preperation job which disables the php-cronjob, scales down the deployment and creates a backup if dbpMoodle.backup.enabled=true |
+| dbpMoodle.moodleUpdatePreparationJob.repository | string | `"ghcr.io/dbildungsplattform"` | Which kubectl image to use |
 | dbpMoodle.moodlecronjob | object | `{"rules":[{"apiGroups":[""],"resources":["pods","pods/exec"],"verbs":["get","list","create","watch"]}],"wait_timeout":"15m"}` | Configuration for the moodle-cronjob which runs moodles cron.php. This is required since moodle does not run as root |
 | dbpMoodle.name | string | `"infra"` |  |
 | dbpMoodle.phpConfig.additional | string | `""` | Any additional text to be included into the config.php |
 | dbpMoodle.phpConfig.debug | bool | `false` | Moodle debugging is not safe for production |
 | dbpMoodle.phpConfig.existingConfig | string | `""` | Provide an existing secret containing the config.php instead of generating it from chart -- Remember to adjust moodle.extraVolumes & moodle.extraVolumeMounts when setting this. -- Secret key is by default expected to be config.php |
 | dbpMoodle.phpConfig.extendedLogging | bool | `false` | Extended php logging |
 | dbpMoodle.redis | object | `{"host":"moodle-redis-master","password":"","port":6379}` | Configurations for the optional redis |
-| dbpMoodle.restore | object | `{"affinity":{},"enabled":false,"existingSecretDatabaseConfig":"moodle-database","existingSecretDatabasePassword":"moodle","existingSecretGPG":"","existingSecretKeyDatabasePassword":"","existingSecretKeyS3Access":"","existingSecretKeyS3Secret":"","existingSecretS3":"","image":"ghcr.io/dbildungsplattform/moodle-tools:1.0.7","resources":{"limits":{"cpu":"2000m","memory":"16Gi"},"requests":{"cpu":"1000m","memory":"8Gi"}},"rules":[{"apiGroups":["apps"],"resources":["deployments/scale","deployments"],"verbs":["get","list","scale","patch"]}],"tolerations":[]}` | This restores moodle to the latest snapshot. Requires an existing s3 backup. ONLY USE FOR ROLLBACK |
-| dbpMoodle.secrets | object | `{"etherpad_api_key":"","etherpad_postgresql_password":"","mariadb_password":"","mariadb_root_password":"","moodle_password":"","pgsql_admin_password":"","useChartSecret":true}` | Creates a secret with all relevant credentials for moodle -- Set useChartSecret: false to provide your own secret -- If you create your own secret, also set moodle.existingSecret (and moodle.externalDatabase.existingSecret if you bring your own DB) |
+| dbpMoodle.restore | object | `{"affinity":{},"enabled":false,"existingSecretDatabaseConfig":"moodle-database","existingSecretDatabasePassword":"moodle","existingSecretGPG":"","existingSecretKeyDatabasePassword":"","existingSecretKeyS3Access":"","existingSecretKeyS3Secret":"","existingSecretS3":"","image":"moodle-tools","repository":"ghcr.io/dbildungsplattform","resources":{"limits":{"cpu":"2000m","memory":"16Gi"},"requests":{"cpu":"1000m","memory":"8Gi"}},"rules":[{"apiGroups":["apps"],"resources":["deployments/scale","deployments"],"verbs":["get","list","scale","patch"]}],"tag":"1.0.8","tolerations":[]}` | This restores moodle to the latest snapshot. Requires an existing s3 backup. ONLY USE FOR ROLLBACK |
+| dbpMoodle.secrets | object | `{"database_password":"","database_root_password":"","etherpad_api_key":"","etherpad_postgresql_password":"","moodle_password":"","useChartSecret":true}` | Creates a secret with all relevant credentials for moodle -- Set useChartSecret: false to provide your own secret -- If you create your own secret, also set moodle.existingSecret (and moodle.externalDatabase.existingSecret if you bring your own DB) |
 | dbpMoodle.stage | string | `"infra"` |  |
-| dbpMoodle.update_migration | object | `{"enabled":false}` | The dbp update process to migrate moodle data when moodle versions are increased |
 | etherpad-postgresql.auth.database | string | `"etherpad"` |  |
 | etherpad-postgresql.auth.enablePostgresUser | bool | `false` |  |
 | etherpad-postgresql.auth.existingSecret | string | `"moodle"` |  |
@@ -281,7 +282,7 @@ The Chart can be deployed without any modification but it is advised to set own
 | moodlecronjob.affinity | object | `{}` |  |
 | moodlecronjob.clusterRole.create | bool | `false` |  |
 | moodlecronjob.image.repository | string | `"ghcr.io/dbildungsplattform/moodle-tools"` |  |
-| moodlecronjob.image.tag | string | `"1.0.7"` |  |
+| moodlecronjob.image.tag | string | `"1.0.8"` |  |
 | moodlecronjob.jobs[0].args[0] | string | `"/scripts/cronjob-script"` |  |
 | moodlecronjob.jobs[0].backoffLimit | int | `1` |  |
 | moodlecronjob.jobs[0].command[0] | string | `"/bin/bash"` |  |

charts/dbp-moodle/scripts/backup.sh

@@ -54,18 +54,18 @@ function clean_up() {
             # Careful, format the string with tabs only!
             restore_probe_patch=$(cat <<-EOF
 				[{
-					"op": "add",
+					"op": "replace",
 					"path": "/spec/template/spec/containers/0/readinessProbe",
 					"value": $(cat ${readiness_bckp})
 				},
 				{
-					"op": "add",
+					"op": "replace",
 					"path": "/spec/template/spec/containers/0/livenessProbe",
 					"value": $(cat ${liveness_bckp})
 				}]
 				EOF
             )
-            kubectl patch "deployment/${release_name}" -n "{{ .Release.Namespace }}" --type=json -p="$restore_probe_patch"
+            kubectl patch "deployment/${deployment_name}" -n "{{ .Release.Namespace }}" --type=json -p="$restore_probe_patch"
         else
             echo "Unable to turn on liveness and readiness probes. Either the readiness_bckp or the liveness_bckp does not exist or is empty."
         fi
@@ -91,10 +91,15 @@ trap "clean_up" EXIT
 # Create liveness probe file
 touch "${health_file}"
 
+{{ if and .Values.dbpMoodle.backup.s3_certificate_secret.enabled }}
+printf "Appendending custom certificate (%s/%s) to /etc/ssl/certs/ca-certificates.crt\n" "{{ .Values.dbpMoodle.backup.s3_certificate_secret.mountpath }}" "{{ .Values.dbpMoodle.backup.s3_certificate_secret.key }}"
+cat "{{ .Values.dbpMoodle.backup.s3_certificate_secret.mountpath }}/{{ .Values.dbpMoodle.backup.s3_certificate_secret.key }}" >> /etc/ssl/certs/ca-certificates.crt
+{{ end }}
+
 # Deployment has "-moodle" appended if the Release.Name does not contain "moodle" 
-release_name="{{ .Release.Name }}"
-if [[ $release_name != "moodle" && $release_name != *"moodle"* ]]; then
-    release_name="${release_name}-moodle"
+deployment_name="{{ .Release.Name }}"
+if [[ $deployment_name != "moodle" && $deployment_name != *"moodle"* ]]; then
+    deployment_name="${deployment_name}-moodle"
 fi
 
 # Create destination dir if not exists
@@ -109,11 +114,11 @@ if ! [ -a /mountData/moodledata/CliUpdate ]; then
     kubectl patch cronjobs "{{ .Release.Name }}-moodlecronjob-{{ include "moodlecronjob.job_name" . }}" -n "{{ .Release.Namespace }}" -p '{"spec" : {"suspend" : true }}'
 
     echo "=== Turn off liveness and readiness probe ==="
-    kubectl get "deployment/${release_name}" -n "{{ .Release.Namespace }}" -o jsonpath="{.spec.template.spec.containers[0].readinessProbe}" > ${readiness_bckp}
-    kubectl get "deployment/${release_name}" -n "{{ .Release.Namespace }}" -o jsonpath="{.spec.template.spec.containers[0].livenessProbe}" > ${liveness_bckp}
-    kubectl patch "deployment/${release_name}" -n "{{ .Release.Namespace }}" --type=json -p="$dummy_probe_patch"
+    kubectl get "deployment/${deployment_name}" -n "{{ .Release.Namespace }}" -o jsonpath="{.spec.template.spec.containers[0].readinessProbe}" > ${readiness_bckp}
+    kubectl get "deployment/${deployment_name}" -n "{{ .Release.Namespace }}" -o jsonpath="{.spec.template.spec.containers[0].livenessProbe}" > ${liveness_bckp}
+    kubectl patch "deployment/${deployment_name}" -n "{{ .Release.Namespace }}" --type=json -p="$dummy_probe_patch"
 
-    kubectl rollout status "deployment/${release_name}" -n "{{ .Release.Namespace }}"
+    kubectl rollout status "deployment/${deployment_name}" -n "{{ .Release.Namespace }}"
 
     # Wait for running jobs to finish to avoid errors
     echo "=== Waiting for jobs to finish ==="

charts/dbp-moodle/scripts/moodleRestoreScript.sh → charts/dbp-moodle/scripts/restore.sh

@@ -20,20 +20,25 @@ health_file="/tmp/healthy"
 # Create liveness probe file
 touch "${health_file}"
 
+{{ if and .Values.dbpMoodle.backup.s3_certificate_secret.mountpath .Values.dbpMoodle.backup.s3_certificate_secret.key }}
+printf "Appendending custom certificate (%s/%s) to /etc/ssl/certs/ca-certificates.crt\n" "{{ .Values.dbpMoodle.backup.s3_certificate_secret.mountpath }}" "{{ .Values.dbpMoodle.backup.s3_certificate_secret.key }}"
+cat "{{ .Values.dbpMoodle.backup.s3_certificate_secret.mountpath }}/{{ .Values.dbpMoodle.backup.s3_certificate_secret.key }}" >> /etc/ssl/certs/ca-certificates.crt
+{{ end }}
+
 # Deployment has "-moodle" appended if the Release.Name does not contain "moodle" 
-release_name="{{ .Release.Name }}"
-if [[ "$release_name" != "moodle" && "$release_name" != *"moodle"* ]]; then
-    release_name="${release_name}-moodle"
+deployment_name="{{ .Release.Name }}"
+if [[ "$deployment_name" != "moodle" && "$deployment_name" != *"moodle"* ]]; then
+    deployment_name="${deployment_name}-moodle"
 fi
 
 # Get current replicas and scale down deployment
-replicas=$(kubectl get "deployment/${release_name}" -n {{ .Release.Namespace }} -o=jsonpath='{.status.replicas}')
+replicas=$(kubectl get "deployment/${deployment_name}" -n {{ .Release.Namespace }} -o=jsonpath='{.status.replicas}')
 echo "=== Current replicas detected: $replicas ==="
 if [ -z "$replicas" ] || [ "$replicas" -eq 0 ]; then 
     replicas=1
 fi
 echo "=== Scale moodle deployment to 0 replicas for restore operation ==="
-kubectl scale "deployment/${release_name}" --replicas=0 -n {{ .Release.Namespace }}
+kubectl scale "deployment/${deployment_name}" --replicas=0 -n {{ .Release.Namespace }}
 echo "=== After restore operation is completed will scale back to: $replicas replicas ==="
 
 # Restore
@@ -96,7 +101,7 @@ PGPASSWORD="$DATABASE_PASSWORD" psql -h "$DATABASE_HOST" -p "$DATABASE_PORT" -U
 echo "=== Finished DB restore ==="
 
 echo "=== Scaling deployment replicas to $replicas ==="
-kubectl scale "deployment/${release_name}" --replicas=$replicas -n {{ .Release.Namespace }}
+kubectl scale "deployment/${deployment_name}" --replicas=$replicas -n {{ .Release.Namespace }}
 sleep 2
-scaledTo=$(kubectl get "deployment/${release_name}" -n {{ .Release.Namespace }} -o=jsonpath='{.status.replicas}')
+scaledTo=$(kubectl get "deployment/${deployment_name}" -n {{ .Release.Namespace }} -o=jsonpath='{.status.replicas}')
 echo "=== Deployment scaled to: $scaledTo ==="

charts/dbp-moodle/scripts/hook/moodle-update-preparation.sh → charts/dbp-moodle/scripts/update-preparation.sh

@@ -6,8 +6,14 @@ set -o pipefail
 
 health_file="/tmp/healthy"
 
+# Deployment has "-moodle" appended if the Release.Name does not contain "moodle" 
+deployment_name="{{ .Release.Name }}"
+if [[ "$deployment_name" != "moodle" && "$deployment_name" != *"moodle"* ]]; then
+    deployment_name="${deployment_name}-moodle"
+fi
+
 get_current_deployment_image() {
-    kubectl get "deploy/{{ .Release.Name }}" -n "{{ .Release.Namespace }}" -o jsonpath='{..image}' |\
+    kubectl get "deploy/${deployment_name}" -n "{{ .Release.Namespace }}" -o jsonpath='{..image}' |\
         tr -s '[:space:]' '\n' |\
         grep '{{- .Values.moodle.image.repository -}}'
 }
@@ -30,14 +36,16 @@ printf 'Image change detected\n'
 printf 'Disabling regular cronjob to prevent failing runs\n'
 kubectl patch cronjobs "{{ .Release.Name }}"-moodlecronjob-"{{ include "moodlecronjob.job_name" . }}" -n "{{ .Release.Namespace }}" -p '{"spec" : {"suspend" : true }}'
 
-printf 'Scaling deployment "{{ .Release.Name }}" to 0 replicas\n'
-kubectl patch "deploy/{{ .Release.Name }}" -n "{{ .Release.Namespace }}" -p '{"spec":{"replicas": 0}}'
+printf 'Scaling deployment "%s" to 0 replicas\n' "$deployment_name"
+kubectl patch "deploy/${deployment_name}" -n "{{ .Release.Namespace }}" -p '{"spec":{"replicas": 0}}'
 
 {{ if .Values.dbpMoodle.backup.enabled }}
 if [ "$BACKUP_ENABLED" = true ]; then
     printf 'Starting pre-update backup\n'
-    kubectl create job moodle-pre-update-backup-job -n "{{ .Release.Namespace }}" --from=cronjob.batch/moodle-backup-cronjob-backup
+    kubectl create job moodle-pre-update-backup-job -n "{{ .Release.Namespace }}" --from="cronjob.batch/{{ include "backup-cronjob.job_name" . }}"
     printf 'Waiting for backup to finish...\n'
     kubectl wait --for=condition=complete --timeout=10m job/moodle-pre-update-backup-job
 fi
 {{ end }}
+
+printf 'Preparations completed successfully, exting...'

charts/dbp-moodle/templates/_helpers.tpl

@@ -19,19 +19,22 @@
 {{- end -}}
 {{- end -}}
 
-{{- define "dbpMoodle.secrets.moodle_password" -}}
-{{- default (randAlphaNum 16) .Values.dbpMoodle.secrets.moodle_password }}
+{{- define "backup-cronjob.job_name" -}}
+{{- $releasename := .Release.Name -}} 
+{{- with (index .Values "backup-cronjob" "jobs") -}}
+{{- printf "%s-backup-cronjob-%s" $releasename (index . 0).name -}}
+{{- end -}}
 {{- end -}}
 
-{{- define "dbpMoodle.secrets.pgsql_admin_password" -}}
-{{- default (randAlphaNum 16) .Values.dbpMoodle.secrets.pgsql_admin_password }}
+{{- define "dbpMoodle.secrets.moodle_password" -}}
+{{- default (randAlphaNum 16) .Values.dbpMoodle.secrets.moodle_password }}
 {{- end -}}
 
-{{- define "dbpMoodle.secrets.db_password" -}}
-{{- default (randAlphaNum 16) .Values.dbpMoodle.secrets.mariadb_password }}
+{{- define "dbpMoodle.secrets.database_password" -}}
+    {{- default (randAlphaNum 16) .Values.dbpMoodle.secrets.database_password }}
 {{- end -}}
 
-{{- define "dbpMoodle.secrets.mariadb_root_password" -}}
+{{- define "dbpMoodle.secrets.database_root_password" -}}
 {{- default (randAlphaNum 16) .Values.dbpMoodle.secrets.mariadb_root_password }}
 {{- end -}}
 
@@ -44,7 +47,7 @@
 {{- end -}}
 
 {{- define "dbpMoodle.secrets.etherpad_api_key" -}}
-{{- default "moodle" .Values.dbpMoodle.secrets.etherpad_api_key }}
+{{- default (randAlphaNum 16) .Values.dbpMoodle.secrets.etherpad_api_key }}
 {{- end -}}
 
 {{- define "dbpMoodle.backup.retention_time" -}}