Fix deployment status propagation when scaling from zero

[skonto]

Fixes #14157

Proposed Changes

• Introduces a new PA condition (PodAutoscalerConditionScaleTargetScaled) that detects failures during scaling to zero, covering the K8s gaps where deployment status is not updated correctly. The condition is set to false just before we scale down to zero (before the deployment update happens) and if pods are crashing. We set it back to true when we scale from zero and we have enough ready pods.

• Previously when deployment was scaled down to zero, revision ready status would be true (and stay that way), but with this patch the pod error is detected and propagated:

Ksvc status:

{
    "lastTransitionTime": "2024-10-04T13:57:35Z",
    "message": "Revision \"revision-failure-00001\" failed with message: Back-off pulling image \"index.docker.io/skonto/revisionfailure@sha256:c7dd34a5919877b89617c3a0df7382e7de0f98318f2c12bf4374bb293f104977\".",
    "reason": "RevisionFailed",
    "status": "False",
    "type": "ConfigurationsReady"
},

Revision:

k  get revision
NAME                     CONFIG NAME        GENERATION   READY   REASON             ACTUAL REPLICAS   DESIRED REPLICAS
revision-failure-00001   revision-failure   1            False   ImagePullBackOff   0                 0

PA status:
{
    "lastTransitionTime": "2024-10-04T13:57:35Z",
    "message": "Back-off pulling image \"index.docker.io/skonto/revisionfailure@sha256:c7dd34a5919877b89617c3a0df7382e7de0f98318f2c12bf4374bb293f104977\"",
    "reason": "ImagePullBackOff",
    "status": "False",
    "type": "ScaleTargetScaled"
}
],

• Updates the pa status propagation logic in the revision reconciler.
• Extends a bit the resource quota e2e test to show that when deployment is scaled to zero we will still report the error. That is irrelevant to this patch but we want to show that we cover certain scenarios more. Probably it would be good to add more e2e tests anyway.
• The steps to test is simply start a skvc, let it scale to zero then remove the image from your registry, block any access (kill internet) and then issue a request.

[codecov]

Codecov Report

Attention: Patch coverage is 36.84211% with 36 lines in your changes missing coverage. Please review.

Project coverage is 84.32%. Comparing base (c8e131b) to head (5ba5209).

Files with missing lines	Patch %	Lines
pkg/reconciler/autoscaling/kpa/scaler.go	30.76%	15 Missing and 3 partials ⚠️
pkg/resources/pods.go	0.00%	7 Missing ⚠️
pkg/apis/autoscaling/v1alpha1/pa_lifecycle.go	33.33%	4 Missing ⚠️
pkg/apis/serving/v1/revision_lifecycle.go	0.00%	2 Missing and 1 partial ⚠️
pkg/reconciler/autoscaling/kpa/kpa.go	60.00%	1 Missing and 1 partial ⚠️
pkg/testing/functional.go	71.42%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #15550      +/-   ##
==========================================
- Coverage   84.49%   84.32%   -0.18%     
==========================================
  Files         219      219              
  Lines       13608    13662      +54     
==========================================
+ Hits        11498    11520      +22     
- Misses       1740     1769      +29     
- Partials      370      373       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[knative-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: skonto
Once this PR has been reviewed and has the lgtm label, please ask for approval from dprotaso. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS [skonto]
• pkg/apis/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[skonto]

We could set ContainerHealthyFalse here too but we need #15503