token is different for each invocation of secret-cluster-agent-token.yaml template

[sarcasticadmin]

Description

The secret-cluster-agent-token.yaml template contains the randAlphaNum function to generate a random token if one is not specific via values. This leads to multiple values of token since its invoked multiple times within the datadog chart during helm install.

The secret-cluster-agent-token.yaml template is called 3 separate times which then causes the generation of 3 separate values for the token value:

1. daemonset.yaml calculating the sha256sum for annotation.checksum/clusteragent_token:

   helm-charts/charts/datadog/templates/daemonset.yaml

   Line 43 in 29ff9d0

   checksum/clusteragent_token: {{ include (print $.Template.BasePath "/secret-cluster-agent-token.yaml") . | sha256sum }}

2. cluster-agent-deployment.yaml calculating the sha256sum for annotation.checksum/clusteragent_token:

   helm-charts/charts/datadog/templates/cluster-agent-deployment.yaml

   Line 52 in 29ff9d0

   checksum/clusteragent_token: {{ include (print $.Template.BasePath "/secret-cluster-agent-token.yaml") . | sha256sum }}

3. Manifest for the results of secret-cluster-agent-token.yaml template for the actual secret datadog-cluster-agent

Helm recommends the following to workaround the limitation of using a unique random string across multiple resources:

Each invocation of the template function will generate a unique random string. This means that if it's necessary to sync the random strings used by multiple resources, all relevant resources will need to be in the same template file.

ref: https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments

Additional discussion around the recommendation for syncing random strings across multiple resources: helm/helm#6456 (comment)

Reproduce

• You can confirm this by noting that the sha256sum value is different for the annotation.checksum/clusteragent_token across both resources in the cluster.

datadog/templates/cluster-agent-deployment.yaml:

...
# Source: datadog/templates/cluster-agent-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: datadog-cluster-agent
  namespace: datadog
  labels:
    helm.sh/chart: 'datadog-3.83.0'
    app.kubernetes.io/name: "datadog"
    app.kubernetes.io/instance: "datadog"
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/version: "7"
    app.kubernetes.io/component: cluster-agent

spec:
  replicas: 1
  revisionHistoryLimit: 10
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  selector:
    matchLabels:
      app: datadog-cluster-agent
  template:
    metadata:
      labels:
        app.kubernetes.io/name: "datadog"
        app.kubernetes.io/instance: "datadog"
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: cluster-agent
        admission.datadoghq.com/enabled: "false"
        app: datadog-cluster-agent

      name: datadog-cluster-agent
      annotations:
        checksum/clusteragent_token: 36c01d9b329101dfe4c6da72bff9d7cd39f48192383d6580f137f902ac44062c <-----------------HERE
        checksum/clusteragent-configmap: 80e84cb320cfd1d77a338751366ed8530ebbd3c074ef913ad7d5621af866405f
        checksum/application_key: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b
        checksum/install_info: f06cede265e1c8d314f29d810a5f72442229263bf7bebafbe12238336e41867b
...

datadog/templates/daemonset.yaml:

...
---
# Source: datadog/templates/daemonset.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: datadog
  namespace: datadog
  labels:
    helm.sh/chart: 'datadog-3.83.0'
    app.kubernetes.io/name: "datadog"
    app.kubernetes.io/instance: "datadog"
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/version: "7"
    app.kubernetes.io/component: agent

spec:
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: datadog
  template:
    metadata:
      labels:
        app.kubernetes.io/name: "datadog"
        app.kubernetes.io/instance: "datadog"
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: agent
        admission.datadoghq.com/enabled: "false"
        app: datadog

      name: datadog
      annotations:
        checksum/clusteragent_token: a0fbd8b46d7e781ec0c35c30a1c8e5099bf962c2df0b5508a0fa962d339aef1f  <-----------------HERE
        checksum/install_info: f06cede265e1c8d314f29d810a5f72442229263bf7bebafbe12238336e41867b
        checksum/autoconf-config: 74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b
        checksum/confd-config: 4cdfa187b0ea0045a3ed56fef7be7339226dd87f62ce3ae48847b6da223adc6e
        checksum/checksd-config: 44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a
...

• Change the sha256sum function to b64enc for each annotation.checksum/clusteragent_token and then use base64 -d to read back out the results of the secret-cluster-agent-token.yaml template. The token does not match the actual token datadog-cluster-agent in secrets.

Fix

Would the team be open to combining the templates for daemonset.yaml, cluster-agent-deployment.yaml, and secret-cluster-agent-token.yaml to fix the behavior?

It would be helpful to correct this behavior so its possible to leverage the lookup function (potentially) to make the chart idempotent: #604 (comment)

Version Info

• datadog chart: 3.83.0
• k8s:

Client Version: v1.31.0
Kustomize Version: v5.4.2
Server Version: v1.31.2-eks-7f9249a