You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was trying to use linkerd viz tap to debug an internal issue, but kept getting tap authorization failed. After looking at the documentation page in the error message (https://linkerd.io/2.16/tasks/securing-linkerd-tap/), it seemed like the problem might be in Linkerd itself as kubectl can-i works
How can it be reproduced?
I believe this might need an EKS cluster configured with EKS Access Entries for authentication :/
Logs, error output, etc
$ kubectl auth can-i watch deployments.v1alpha1.tap.linkerd.io/[xxx] --namespace [xxx] --subresource tap
yes
$ linkerd viz tap deployment/[xxx] --namespace [xxx]
HTTP error, status Code [403] (unexpected API response: {"error":"tap authorization failed (not authorized to access deployments.tap.linkerd.io), visit https://linkerd.io/tap-rbac for more information"})
output of linkerd check -o short
linkerd-identity
----------------
× trust anchors are using supported crypto algorithm
Invalid trustAnchors:
* 498364598153846236906649535816328601770266843336 One Model Linkerd Root CA must use P-256 curve for public key, instead P-384 was used
see https://linkerd.io/2/checks/#l5d-identity-trustAnchors-use-supported-crypto for hints
linkerd-jaeger
--------------
‼ jaeger extension proxies are up-to-date
some proxies are not running the current version:
* collector-9cb9f47cb-2p4dn (edge-24.10.1)
* collector-9cb9f47cb-s8zg2 (edge-24.10.1)
* jaeger-injector-759b7ff6fb-5xcmz (edge-24.10.1)
* jaeger-injector-759b7ff6fb-q46tg (edge-24.10.1)
see https://linkerd.io/2/checks/#l5d-jaeger-proxy-cp-version for hints
‼ jaeger extension proxies and cli versions match
collector-9cb9f47cb-2p4dn running edge-24.10.1 but cli running edge-24.10.2
see https://linkerd.io/2/checks/#l5d-jaeger-proxy-cli-version for hints
linkerd-viz
-----------
‼ viz extension proxies are up-to-date
some proxies are not running the current version:
* alertmanager-alertmanager-0 (edge-24.10.1)
* metrics-api-fd554c577-8htqz (edge-24.10.1)
* metrics-api-fd554c577-ftj2d (edge-24.10.1)
* prometheus-prometheus-0 (edge-24.10.1)
* tap-5579f7499d-2v4dg (edge-24.10.1)
* tap-5579f7499d-5jf5j (edge-24.10.1)
* tap-5579f7499d-bmdfg (edge-24.10.1)
* tap-injector-5bfbc5b658-fqr7c (edge-24.10.1)
* tap-injector-5bfbc5b658-kvz6s (edge-24.10.1)
* tap-injector-5bfbc5b658-mrlmg (edge-24.10.1)
* web-694c5b64d8-hs76l (edge-24.10.1)
* web-694c5b64d8-kkqtp (edge-24.10.1)
see https://linkerd.io/2/checks/#l5d-viz-proxy-cp-version for hints
‼ viz extension proxies and cli versions match
alertmanager-alertmanager-0 running edge-24.10.1 but cli running edge-24.10.2
see https://linkerd.io/2/checks/#l5d-viz-proxy-cli-version for hints
Status check results are ×
I've made a commit which I will push up in a PR shortly, but I suspect the issue may be because the SubjectAccessReview done by the Tap controller doesn't pass in any of the "extra" user attributes
When looking at the audit logs generated by our Kubernetes control plane and comparing the linkerd viz tap vs. kubectl can-i, I can see that kubectl can-i is passing some additional "extra" fields that seem relevant to the EKS Access Entries authentication.
Kubernetes authorization plugins can rely on extra attributes on a user, and these are provided via `X-Remote-Extra-` headers. Currently the Linkerd Viz `tap` API doesn't include these attributes when making the `SubjectAccessReview` request which means the Tap API cannot be used by end-users who's clusters use such authz plugins.
This change updates the `tap` controller to parse the `X-Remote-Extra-` headers and include them in the SubjectAccessReview request.
Fixedlinkerd#13169
Signed-off-by: David Symons <[email protected]>
What is the issue?
I was trying to use
linkerd viz tap
to debug an internal issue, but kept gettingtap authorization failed
. After looking at the documentation page in the error message (https://linkerd.io/2.16/tasks/securing-linkerd-tap/), it seemed like the problem might be in Linkerd itself askubectl can-i
worksHow can it be reproduced?
I believe this might need an EKS cluster configured with EKS Access Entries for authentication :/
Logs, error output, etc
$ kubectl auth can-i watch deployments.v1alpha1.tap.linkerd.io/[xxx] --namespace [xxx] --subresource tap
yes
$ linkerd viz tap deployment/[xxx] --namespace [xxx]
HTTP error, status Code [403] (unexpected API response: {"error":"tap authorization failed (not authorized to access deployments.tap.linkerd.io), visit https://linkerd.io/tap-rbac for more information"})
output of
linkerd check -o short
Environment
Kubernetes Version: v1.30
Cluster Environment: AWS EKS
Host OS: Amazon Bottlerocket
Linkerd version: Client version: edge-24.10.2 / Server version: edge-24.10.1
Possible solution
I've made a commit which I will push up in a PR shortly, but I suspect the issue may be because the SubjectAccessReview done by the Tap controller doesn't pass in any of the "extra" user attributes
When looking at the audit logs generated by our Kubernetes control plane and comparing the
linkerd viz tap
vs.kubectl can-i
, I can see thatkubectl can-i
is passing some additional "extra" fields that seem relevant to the EKS Access Entries authentication.My commit updates ResourceAuthzForUser in linkerd/pkg/k8s/authz.go to take in the list of extra attributes, retrieved via the X-Remote-Extras- HTTP header
Additional context
No response
Would you like to work on fixing this bug?
yes
The text was updated successfully, but these errors were encountered: