How to configure linkerd with CloudNativePG?

[99hops]

Friends, I am trying to configure the CloudNativePG operator to work with linkerd without luck so far.
Looks like I am missing some config that is unknown to me as there are no error I can debug from and saw things like config.linkerd.io/opaque-ports that may be relavant.

At this point I can confirm adding this will make cnpg work properly (especially the init cluster import part)

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: postgresql
  namespace: dev
spec:
  inheritedMetadata:
    annotations:
      linkerd.io/inject: disabled <--------- HERE

Please advice

[99hops]

Hmm I now think the issue here is linkerd-proxy container not terminating during pg import Job even with config.alpha.linkerd.io/proxy-enable-native-sidecar: enabled it's starying active while the other 3 containers are terminated.

[99hops]

I believe I found the root cause, CloudNativePG have implemeted http://localhost:4191/shutdown hook to kill the proxy which is now disabled by default

curl -X POST http://localhost:4191/shutdown
shutdown endpoint is not enabled

Is there a way to enable it for the given use case and not globally?
Here is a snipped if config I can inject into the cnpg pods

---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: postgresql
  namespace: dev
spec:
  inheritedMetadata:
    annotations:
      config.linkerd.io/proxy-await: enabled
      config.linkerd.io/response-timeout: "10s"
      config.linkerd.io/proxy-enable-shutdown: enabled    <----- LIKE SO?

I confirm I have no issues when enabled globally but this doesn't look optimal?
Here is helm snippet from ArgoCD

---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: linkerd-control-plane
  namespace: argocd
spec:
  project: infra
  source:
    repoURL: 'https://helm.linkerd.io/edge'
    targetRevision: 2024.8.2
    chart: linkerd-control-plane
    helm:
      values: |
         proxy:
           enableShutdownEndpoint: true  # Enable the proxy shutdown endpoint globally

[olix0r]

That sounds correct to me. It seems most desirable to configure the annotation-based solution so that only these workloads have this feature re-enabled.

[99hops]

That sounds correct to me. It seems most desirable to configure the annotation-based solution so that only these workloads have this feature re-enabled.

Right @olix0r but this looks not possible at the moment looking at https://linkerd.io/2.16/reference/proxy-configuration/ there is no such annotation?

[olix0r]

Actually, there is, but the docs haven't been updated yet ;) (sorry, it's a known gap and we'll fix it.) The annotation is config.linkerd.io/proxy-admin-shutdown: enabled.

[99hops]

This is the working solution with annotation, enabling globally is also an option.

---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: postgresql
  namespace: dev
spec:
  inheritedMetadata:
    annotations:
      config.linkerd.io/proxy-admin-shutdown: enabled

cloudnative-pg/cloudnative-pg@f73cc8b

[99hops]

@olix0r maybe it's now the time to ask about config.alpha.linkerd.io/proxy-enable-native-sidecar: enabled
Reading the docs here I got the impression that this flag is enough by itself to not allow linkerd-proxy to be the only container left running but it seems this is not the case? (tested on 1.29.4 and 1.30.4) 😸

Here is sample config

---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: postgresql
  namespace: dev
spec:
  inheritedMetadata:
    annotations:
      config.alpha.linkerd.io/proxy-enable-native-sidecar: enabled

[olix0r]

Yeah, native sidecars are definitely preferable in general, but they're a beta feature in kubernetes and the support has been inconsistent across cloud providers. If your cluster supports them properly, there should be no harm in enabling the feature cluster-wide via the helm chart. Oliver Gould < ***@***.*** >

…

On Thu, Sep 26 2024 at 23:46, 99hops < ***@***.*** > wrote: @olix0r ( https://github.com/olix0r ) maybe it's now the time to ask about config.alpha.linkerd.io/proxy-enable-native-sidecar: enabled Reading the docs here ( https://linkerd.io/2.16/tasks/graceful-shutdown/#native-sidecar ) I got the impression that this flag is enough by itself to not allow linkerd-proxy to be the only container left running but it seems this is not the case? 😸 Here is sample config --- apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: postgresql namespace: dev spec: inheritedMetadata: annotations: config.alpha.linkerd.io/proxy-enable-native-sidecar: enabled — Reply to this email directly, view it on GitHub ( #13105 (comment) ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/AAB2YYVPPKGNPQ64BEMHRADZYT5MPAVCNFSM6AAAAABO4NBLM6VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANZXGE2DOMA ). You are receiving this because you were mentioned. Message ID: <linkerd/linkerd2/repo-discussions/13105/comments/10771470 @ github. com>

[99hops]

Hmm, how to make sure the containers are "native sidecars" aka supported?
I run 1.30.4 on bare metal setup with kubespray 🍻

[olix0r]

The SidecarContainers feature has been enabled by default since Kubernetes v1.29.

We've seen some issues in AKS and EKS where they haven't properly shipped sidecar container support in their API servers in all of these version. If you are running a "vanilla" Kubernetes distribution, I'd expect sidecars to function properly in v1.29 and later.