Scaling is not performed because triggers are not active

[slengieza]

I am new to Kubernetes and I am trying to manage my standalone Azure Container Instances with Azure Kubernetes Service. I have been able to deploy the application using AKS and I have been following this sample to auto-scale the application based on the number of messages in Azure Service Bus Topics.

When I initially deploy the ScaledObject I can see the deployments scale up correctly (if messages are already added to the Topic) and then they scale back down to 0 when the messages have all been consumed. However, when I add more messages to the Topic or deploy the ScaledObject while there are no messages queued I am not seeing the deployment scale up as it did before.

When I run kubectl describe scaledobject.keda.sh/wwml-aks-canary-scaler I see the following:

Status:
  Conditions:
    Message:  ScaledObject is defined correctly and is ready for scaling
    Reason:   ScaledObjectReady
    Status:   True
    Type:     Ready
    Message:  Scaling is not performed because triggers are not active
    Reason:   ScalerNotActive
    Status:   False
    Type:     Active

I have been looking online to see if I could figure out why the triggers would not be "Active" but I wasn't able to find anything. Any help would be greatly appreciated. Thanks!

This is my autoscale yaml file:

kind: TriggerAuthentication
metadata:
  name: trigger-auth-service-bus-canary
spec:
  secretTargetRef:
  - parameter: connection
    name: secrets-management
    key: servicebus-connectionstring
---
apiVersion: v1
kind: Secret
metadata:
  name: secrets-management
  labels:
    app: wwml-aks-canary
data:
  servicebus-connectionstring: <base 64 encoded connection string>
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: wwml-aks-canary-scaler
spec:
  scaleTargetRef:
    name: wwml-aks-canary
  pollingInterval: 30
  cooldownPeriod:  300
  minReplicaCount: 0
  maxReplicaCount: 100
  advanced:
    restoreToOriginalReplicaCount: true
    horizontalPodAutoscalerConfig:
      behavior:
        scaleDown:
          stabilizationWindowSeconds: 300
          policies:
          - type: Percent
            value: 100
            periodSeconds: 15
  triggers:
  - type: azure-servicebus
    metadata:
      topicName: canary-topic
      subscriptionName: Highest
    authenticationRef:
      name: trigger-auth-service-bus-canary
  - type: azure-servicebus
    metadata:
      topicName: canary-topic
      subscriptionName: High
    authenticationRef:
      name: trigger-auth-service-bus-canary
  - type: azure-servicebus
    metadata:
      topicName: canary-topic
      subscriptionName: Normal
    authenticationRef:
      name: trigger-auth-service-bus-canary
  - type: azure-servicebus
    metadata:
      topicName: canary-topic
      subscriptionName: Low
    authenticationRef:
      name: trigger-auth-service-bus-canary
  - type: azure-servicebus
    metadata:
      topicName: canary-topic
      subscriptionName: Lowest
    authenticationRef:
      name: trigger-auth-service-bus-canary

[tomkerkhove]

However, when I add more messages to the Topic or deploy the ScaledObject while there are no messages queued I am not seeing the deployment scale up as it did before.

When deploying the ScaledObject it will behave the same way as it was before so this will not help, but if messages are being added it should if it matches messageCount. In this case, it is not specified so it will be using 5 (docs; how many did you configure?

Can you talk about the application that you are deploying because now you add triggers for all your subscriptions and will take all these metrics into account.

I see you configure advanced but typically you don't need this unless you really want that control, do you want this?

[tomkerkhove]

I'm not sure if that is correct, because from what I'm reading and is defined KEDA will not require anything from the deployment and purely relies on the Secret that you are providing.

I would say, can you provide some more logs from KEDA because the ones provided above don't make sense because it's referring to different secret & environment variables.

Now if you would rely on the SecretProviderClass which is mounted to your app, then I think scale to 0 will not be possible indeed. That said, we don't support mounting secrets in KEDA as well so that would be a limitation /cc @zroubalik

PS - I saw you are using pod identity, don't forget that you can also enable this with KEDA.

[slengieza]

I am using a SecretProviderClass to mount secrets. Is that something that will be supported in the future?

My workaround for now is having a small container constantly running that also mounts those secrets so that the secret Pod is present when KEDA is trying to scale from 0-1.

[tomkerkhove]

It would be good if you can open an issue for this; that would feel like a new secret provider for TriggerAuthentication, what do you think @zroubalik?

[zroubalik]

Yeah agree with Tom, open an issue if you are interested in this secret provider, SecretProviderClass is unfortunately not supported currently.

[slengieza]

Okay will do, thanks

[slengieza]

Created Issue here: #1407

[jfborie]

In case you have this error go into the keda namespace and look at the log in the keda-operator pod, you might get more useful information.