Support exec readinessProbes for sidecar serving containers

[FloMedja]

Knative version v1.15.0

Expected Behavior

The Knative serving supports the exec readiness probes for serving sidecar containers.

I call serving sidecar containers the containers that didn't have the port defined.

Actual Behavior

The knative serving reconciler triggers an error because the exec readiness probe is not supported in the serving sidecar containers. It is due to the fact that the applyReadinessProbeDefaults is used to compute the probes for the main and sidecar serving containers. The function needs a port to create a TCPProbe in the queue container. That TCPProbe is required for serving containers exec readiness probes.

Question : Why do we need to have a TCP Probe on the queue container for the exec probes of the serving containers ?

Reflection : If ports definition were allowed for sidecar containers, this issue would be trivial to fix. I see that an issue is open for but is on ice for a while.

Contribution: I can help fixing the issue when we will decide on a common implementation.

Steps to Reproduce the Problem

1. Create a knative serving with this yaml :

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  annotations:
    # Add necessary annotations
    serving.knative.dev/istio-gateways: knative-serving/knative-external-ingress-gateway # Change If you don't use istio gateway
  name: helloworld-tcp-error
  namespace: default # Customize your namespace
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/class: kpa.autoscaling.knative.dev
        autoscaling.knative.dev/metric: concurrency
        autoscaling.knative.dev/max-scale: "1"
        autoscaling.knative.dev/min-scale: "1"
    spec:
      containerConcurrency: 0
      containers:
        - image: ghcr.io/knative/helloworld-go:latest
          name: app
          env:
            - name: TARGET
              value: "Go Sample v1"
          ports:
            - containerPort: 8080
              protocol: TCP
          readinessProbe:
            exec:
              command:
                - /bin/sh
                - -c
                - cat /proc/1/status
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - "CAP_SYS_ADMIN"
            runAsNonRoot: true
            runAsUser: 1000
            seccompProfile:
              type: RuntimeDefault
        - name: sidecar
          image: registry.k8s.io/busybox
          args:
            - /bin/sh
            - -c
            - touch /tmp/healthy; sleep 30; rm -f /tmp/healthy; sleep 600
          readinessProbe:
            exec:
              command:
                - /bin/sh
                - -c
                - cat /proc/1/status
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - "CAP_SYS_ADMIN"
            runAsNonRoot: true
            runAsUser: 1000
            seccompProfile:
              type: RuntimeDefault
      timeoutSeconds: 60
  traffic:
    - latestRevision: true
      percent: 100

2. retrieve the container events with kubectl events -n <namespace> you will see an error similar as :

5s (x21 over 38m)        Warning   InternalError           Revision/helloworld-tcp-error-00001                            failed to create deployment "helloworld-tcp-error-00001-deployment": failed to make deployment: failed to create PodSpec: failed to create queue-proxy container: sidecar readiness probe does not define probe port on container: sidecar

[FloMedja]

@skonto Can I have your input on this please ?
Thanks :)