Getting metadata in alerts from effected instances?

[iploooox]

Hey team, I've been trying to figure out how I can essentially retrieve the environment of the deployment in a generic provider as an alert, since I need it for reporting LTFC (Lead Time For Change) for our deployments.

The problem I'm facing is that I cannot find a way to access labels / podAnnotations / podLabels in the alert.

Here is my current, naive approach:

podinfo-values.yaml

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: podinfo
  namespace: podinfo
  labels:
    environment: production
spec:
  chart:
    spec:
      version: ">=1.0.0"
  values:
    podAnnotations:
      environment: production
    podLabels:
      environment: production
    ingress:
      hosts:
        - host: podinfo.production
          paths:
            - path: /
              pathType: ImplementationSpecific

alerts-config.yaml

---
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Provider
metadata:
  name: mdi-provider
  namespace: podinfo
spec:
  type: generic
  address: ******
---
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Provider
metadata:
  name: mdi-provider
  namespace: flux-system
spec:
  type: generic
  address: ******
---
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Alert
metadata:
  name: podinfo-alert
  namespace: podinfo
spec:
  summary: "Reconciliation of app podinfo in production"
  providerRef:
    name: mdi-provider
  eventSeverity: info
  eventSources:
    - kind: HelmRelease
      name: 'podinfo'
    - kind: GitRepository
      name: '*'
    - kind: Kustomization
      name: '*'
  eventMetadata:
    env: production-from-event-metadata
    env-label: '{{ index .Metadata.Labels "environment" }}'
    labels: '{{ .Metadata.Labels }}'
    app-version: '{{ .Metadata.app-version }}'
    revision: '{{ .Metadata.revision }}'
    pod_annotations: '{{ index .InvolvedObject.Spec "values" "podAnnotations" | toJson }}'
    pod_labels: '{{ index .InvolvedObject.Spec "values" "podLabels" | toJson }}'
---
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Alert
metadata:
  name: flux-system-alert
  namespace: flux-system
spec:
  summary: "Reconciliation of flux-system components"
  providerRef:
    name: mdi-provider
  eventSeverity: info
  eventSources:
    - kind: HelmRelease
      name: 'podinfo'
    - kind: GitRepository
      name: '*'
    - kind: Kustomization
      name: '*'
  eventMetadata:
    env: production-from-event-metadata
    env-label: '{{ index .Metadata.Labels "environment" }}'
    labels: '{{ .Metadata.Labels }}'
    app-version: '{{ .Metadata.app-version }}'
    revision: '{{ .Metadata.revision }}'
    pod_annotations: '{{ index .InvolvedObject.Spec "values" "podAnnotations" | toJson }}'
    pod_labels: '{{ index .InvolvedObject.Spec "values" "podLabels" | toJson }}'

I'm receiving env: production-from-event-metadata as expected, but the rest are coming through as strings as configured.

The problem I'm facing is that the alert is per namespace, and since my organization has teams running their own way of doing Flux, I cannot guarantee that this specific namespace will only contain production/development pods. Ideally, I'd like to resolve the metadata/labels directly from the pod itself.