Keda Service Bus Scaler Azure Workload Identity Fails with "sources must contain at least one TokenCredential"

[anthonynguyen394]

I am struggling to try to get Keda to work with Azure AD Workload Identity

My Keda Object with service bus file:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: app-keda-scaled-object
spec:
  scaleTargetRef:
    kind: Deployment
    name: app-deployment
  minReplicaCount: 0
  maxReplicaCount: 10
  cooldownPeriod: 300
  pollingInterval: 5
  triggers:
    - type: azure-servicebus
      metadata:
        namespace: "app-sbns"
        queueName: "app-test-queue"
        messageCount: "1"
      authenticationRef:
        name: app-keda-auth

My Keda Auth file:

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: app-keda-auth
spec:
  podIdentity:
    provider: azure-workload

Error message on keda-operator:

2023-09-04T06:10:38Z	ERROR	azure_servicebus_scaler	error getting service bus entity length	{"type": "ScaledObject", "namespace": "app-namespace", "name": "app-keda-scaled-object", "error": "sources must contain at least one TokenCredential"}
github.com/kedacore/keda/v2/pkg/scalers.(*azureServiceBusScaler).GetMetricsAndActivity
	/workspace/pkg/scalers/azure_servicebus_scaler.go:262
github.com/kedacore/keda/v2/pkg/scaling/cache.(*ScalersCache).GetMetricsAndActivityForScaler
	/workspace/pkg/scaling/cache/scalers_cache.go:130
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).getScaledObjectState
	/workspace/pkg/scaling/scale_handler.go:572
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).checkScalers
	/workspace/pkg/scaling/scale_handler.go:236
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).startScaleLoop
	/workspace/pkg/scaling/scale_handler.go:175

I have configured Azure managed identity, Federated Identity and Oidc for the AKS cluster as according to:
https://learn.microsoft.com/en-us/azure/azure-monitor/containers/integrate-keda

$ az aks show --resource-group $RESOURCE_GROUP --name $AKS_CLUSTER_NAME --query oidcIssuerProfile
{
  "enabled": true,
  "issuerUrl": "https://<region>.oic.prod-aks.azure.com/xxxxxx/xxxxxx/"
}

$ az aks show --resource-group $RESOURCE_GROUP --name $AKS_CLUSTER_NAME --query securityProfile.workloadIdentity
{
  "enabled": true
}

$ az identity federated-credential list --identity-name <keda-managed-id> -g <resource-group-name>
[
  {
    "audiences": [
      "api://AzureADTokenExchange"
    ],
    "id": "/subscriptions/xxxx/resourcegroups/xxxx/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<aks-cluster>/federatedIdentityCredentials/<keda-managed-id>",
    "issuer": "https://<region>..oic.prod-aks.azure.com/xxxx/xxxxx/",
    "name": "kedaOperatorFedIdentity",
    "resourceGroup": "<resource-group-name>",
    "subject": "system:serviceaccount:keda:keda-operator",
    "systemData": null,
    "type": "Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials"
  }
]

I have also assigned RBAC for the managed identity's client ID to the service bus itself.

[hectorm-bmg]

I'm getting a similar error but with ServiceBus and TriggerAuthentication
If I use a connection string through keyvault then the service bus trigger works just fine. And inside my application I can actually use the WorkloadIdentityCredential to access the service bus queue without using a connection string.

[ehrnst]

I too have the same issue. did you figure it out?

[black-snow]

same here

[hectorm-bmg]

I was able to make it work like this:

In the scaledJob:

...
- type: azure-servicebus
    metadata:
      namespace: my-sbns
      queueName: "myqueuename"  
      messageCount: "1"
    authenticationRef:
      name: "azure-servicebus-auth"

service-bus-trigger-auth.yaml

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: "azure-servicebus-auth"
spec:
  podIdentity:
    provider: "azure-workload"
    identityId: {APPLICATION_CLIENT_ID}

APPLICATION_CLIENT_ID is coming from an app registration

export APPLICATION_CLIENT_ID="$(az ad sp list --display-name ${AKS_APP} --query '[0].appId' -otsv)"
export SERVICE_ACCOUNT_ISSUER="$(az aks show -n ${AKS_CLUSTER}  -g ${RESOURCE_GROUP} --query "oidcIssuerProfile.issuerUrl" -otsv)"
export APPLICATION_OBJECT_ID="$(az ad app show --id ${APPLICATION_CLIENT_ID} --query id -otsv)"

that gets created by an Entra administrator:

az ad sp create-for-rbac --name my-aks-app

Administrator needs to Grant Permissions to the app

and then create the federated credentials

cat <<EOF > params.json
{
  "name": "my-kubernetes-federated-credential",
  "issuer": "${SERVICE_ACCOUNT_ISSUER}",
  "subject": "system:serviceaccount:default:workload-identity-sa",
  "description": "Kubernetes service account federated credential",
  "audiences": [
    "api://AzureADTokenExchange"
  ]
}
EOF

az ad app federated-credential create --id ${APPLICATION_OBJECT_ID} --parameters @params.json

cat <<EOF > params.json
{
  "name": "kubernetes-federated-credential-keda-operator",
  "issuer": "${SERVICE_ACCOUNT_ISSUER}",
  "subject": "system:serviceaccount:keda:keda-operator",
  "description": "Kubernetes service account federated credential for keda operator.",
  "audiences": [
    "api://AzureADTokenExchange"
  ]
}
EOF

az ad app federated-credential create --id ${APPLICATION_OBJECT_ID} --parameters @params.json

Hope that helps

[black-snow]

In my case it seems like adding the tenant ID was the secret sauce.

[MitanshiKshatriya]

Hi, @black-snow , I am facing the same issue Keda Service Bus Scaler Azure Workload Identity Fails with "sources must contain at least one TokenCredential". I am trying to authenticate keda using managed identity. I am using AKS add on keda.

Did you add tenantId to triggerAuthentication? I could not find a tenantId parameter under pod identity in trigger authentication.

[black-snow]

@MitanshiKshatriya Adding spec.podIdentity.identityTenantId to TriggerAuthentication did the trick.