#229 Add OpenTelemetry Collector Server Auth Extensions to Receivers #2203

aidanleuck · 2024-12-02T17:01:33Z

PR Description

Adds server authentication support to otelcol.receivers.* that support the OpenTelemetry collector auth extension model. Some receivers are not wrappers of the collector so they are excluded from this PR.

Receivers that now support server auth:

Datadog
Jaeger
Otlp
Opencensus
Zipkin
Jaeger_Remote_Sampling
InfluxDB

Which issue(s) this PR fixes

Fixes #229

Notes to the Reviewer

TestAuthServer in auth_basic_test.go is consistently failing due to a race condition in startup. Any ideas or potential solutions to this problem would be appreciated. The auth_basic plugin appears to function as expected when running from the binary.

User interface for client auth has not changed.

PR Checklist

[ x ] CHANGELOG.md updated
[ x ] Documentation added
[ x ] Tests updated

CLAassistant · 2024-12-02T17:01:40Z

All committers have signed the CLA.

clayton-cornell · 2024-12-09T22:45:31Z

@grafana/grafana-alloy-maintainers I am going to hold back on reviewing the doc input until after a technical code review has been completed. There are some questions I have about the way things are phrased in the docs that may be resolved with the code review.

aidanleuck · 2024-12-10T04:06:29Z

@grafana/grafana-alloy-maintainers I am going to hold back on reviewing the doc input until after a technical code review has been completed. There are some questions I have about the way things are phrased in the docs that may be resolved with the code review.

Thanks for taking a look! Happy to clarify once the code review is complete.

…eceiver

aidanleuck · 2024-12-16T00:16:38Z

The latest commits should fix the issue with the race condition in auth_basic.go. I have also addressed the outstanding comments on the PR.

clayton-cornell · 2025-01-02T18:50:04Z

docs/sources/reference/components/otelcol/otelcol.auth.oauth2.md

@@ -10,6 +10,8 @@ title: otelcol.auth.oauth2

 `otelcol.auth.oauth2` exposes a `handler` that can be used by other `otelcol` components to authenticate requests using OAuth 2.0.

+This extension only supports client authentication. 


Is this an extension to a component? Or do we mean component here?

docs/sources/reference/components/otelcol/otelcol.auth.sigv4.md

docs/sources/reference/components/otelcol/otelcol.extension.jaeger_remote_sampling.md

aidanleuck · 2025-01-03T02:52:34Z

@clayton-cornell I have made the requested changes to the docs. Based on your previous feedback I also slightly modified each receiver's docs. Let me know if you would like any more changes.

…re/add_auth_to_otlp_receiver

clayton-cornell

Some really minor tweaks to the docs and they look OK to me.

clayton-cornell · 2025-01-03T14:29:27Z

docs/sources/reference/components/otelcol/otelcol.extension.jaeger_remote_sampling.md

+ {{< admonition type="note" >}}
+Not all OpenTelemetry Collector authentication plugins support receiver authentication.
+Refer to the [documentation](https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/components/otelcol/) for each `otelcol.auth.*` component to determine its compatibility.
+ {{< /admonition >}}


Suggested change

{{< admonition type="note" >}}

Not all OpenTelemetry Collector authentication plugins support receiver authentication.

Refer to the [documentation](https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/components/otelcol/) for each `otelcol.auth.*` component to determine its compatibility.

{{< /admonition >}}

{{< admonition type="note" >}}

Not all OpenTelemetry Collector authentication plugins support receiver authentication.

Refer to the [documentation](https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/components/otelcol/) for each `otelcol.auth.*` component to determine its compatibility.

{{< /admonition >}}

Removing extra space

clayton-cornell · 2025-01-03T14:30:31Z

docs/sources/reference/components/otelcol/otelcol.auth.basic.md

@@ -11,7 +11,7 @@ title: otelcol.auth.basic
 `otelcol.auth.basic` exposes a `handler` that can be used by other `otelcol`
 components to authenticate requests using basic authentication.

-This extension supports both server and client authentication.
+This `component` supports both server and client authentication.


Suggested change

This `component` supports both server and client authentication.

This component supports both server and client authentication.

Ooops, my fault. I used the inline code syntax to emphasize in the previous comment.

clayton-cornell · 2025-01-03T14:31:10Z

docs/sources/reference/components/otelcol/otelcol.auth.bearer.md

@@ -11,7 +11,7 @@ title: otelcol.auth.bearer
 `otelcol.auth.bearer` exposes a `handler` that can be used by other `otelcol`
 components to authenticate requests using bearer token authentication.

-This extension supports both server and client authentication.
+This `component` supports both server and client authentication.


Suggested change

This `component` supports both server and client authentication.

This component supports both server and client authentication.

clayton-cornell · 2025-01-03T14:31:20Z

docs/sources/reference/components/otelcol/otelcol.auth.headers.md

@@ -11,7 +11,7 @@ title: otelcol.auth.headers
 `otelcol.auth.headers` exposes a `handler` that can be used by other `otelcol`
 components to authenticate requests using custom headers.

-This extension only supports client authentication. 
+This `component` only supports client authentication. 


Suggested change

This `component` only supports client authentication.

This component only supports client authentication.

clayton-cornell · 2025-01-03T14:31:30Z

docs/sources/reference/components/otelcol/otelcol.auth.oauth2.md

@@ -10,7 +10,7 @@ title: otelcol.auth.oauth2

 `otelcol.auth.oauth2` exposes a `handler` that can be used by other `otelcol` components to authenticate requests using OAuth 2.0.

-This extension only supports client authentication. 
+This `component` only supports client authentication. 


Suggested change

This `component` only supports client authentication.

This component only supports client authentication.

clayton-cornell · 2025-01-03T14:31:41Z

docs/sources/reference/components/otelcol/otelcol.auth.sigv4.md

@@ -12,7 +12,7 @@ title: otelcol.auth.sigv4
 components to authenticate requests to AWS services using the AWS Signature Version 4 (SigV4) protocol.
 For more information about SigV4 see the AWS documentation about [Signing AWS API requests][].

-This extension only supports client authentication. 
+This `component` only supports client authentication. 


Suggested change

This `component` only supports client authentication.

This component only supports client authentication.

…eceiver

aidanleuck · 2025-01-04T21:04:51Z

Documentation changes have been implemented. However, the recent merge with the main branch seems to have caused a test failure. This pull request PR appears to be the culprit. The same test is also failing on the main branch. I wanted to flag this issue here to ensure the failure doesn't hinder the progress of this PR.

clayton-cornell · 2025-01-07T01:10:43Z

@aidanleuck It's probably OK to ignore the failing test here. The Deploy pr preview will fail on forks. The macos test fail is not a mandatory pass to merge.

@grafana/grafana-alloy-maintainers over to someone on the dev team for code review

mattdurham · 2025-01-09T14:32:14Z

internal/component/otelcol/config_http.go

-	// authentication extensions to all of our components.
+	// Auth is a binding to an otelcol.auth.* component extension which handles
+	// authentication.
+	Auth *auth.Handler `alloy:"auth,attr,optional"`


I think we should name this to a authentication in both the var name and alloy name. This is so it doesnt overlap with authentication. Same with the documentation.

What do you think about renaming the var name to Authentication but keeping the alloy name as auth? That way when using client auth or server auth the field is consistent.

client auth looks like this. We can't easily change this without breaking the user interface.

otelcol.exporter.otlp "otlp"{ auth = <component>.handler }

server auth would look like this if we changed auth to authentication.

otelcol.receiver.otlp "receiver"{ authentication = <component>.handler }

As a user I like the consistency between it being auth for both exporters and receivers. Let me know what you think.

I can live with that. Can you add a comment in the code saying why its that way?

mattdurham · 2025-01-09T14:32:33Z

internal/component/otelcol/config_http.go

 	}

 	// Configure the authentication if args.Auth is set.
-	var auth *otelconfigauth.Authentication
+	var authz *otelconfigauth.Authentication


Var name feels like authorization.

mattdurham

Code looks good, minor nits on var naming and alloy configuration naming.

aidanleuck · 2025-01-09T18:13:11Z

internal/component/otelcol/auth/auth.go

 	if err != nil {
 		return err
-	} else if ext != nil {


This merge conflict was already handled in this PR in the function createExtension() on line 388 of this file.

mattdurham · 2025-01-09T19:16:36Z

This is looking great. One last comment to add and I will merge. Likely will pull this into the RC candidate before release.

aidanleuck · 2025-01-09T20:13:17Z

Comments addressed! I also changed the var name for client blocks from Auth to Authentication for consistency.

mattdurham

lgtm thank you for the PR!

* update changelog for rc (#2360) * update changelog for rc * update changelog for rc * update changelog for rc (#2361) * update version (#2362) * update changelog for rc (#2360) * update changelog for rc * update changelog for rc Signed-off-by: matt durham <[email protected]> * fix conversion * Fix changelog main (#2364) * update version * Fix changelog * update the image version to work with the given example (#2358) * docs: fixed kafka config example (#2359) Example shows `loki.source.kafka "local"` pointing to `loki.relabel.kafka.receiver`. This leads to no new label being added. Correct example should have the kafka source pointing directly to `loki.write.local.receiver` * feat(helm): add the ability to deploy extra manifest files (#2347) * feat(helm): add the ability to deploy extra manifest files * docs(helm): run helm-docs * ci(helm): add tests * Update wal queue tls (#2363) * add tls to wal * add alloy config * update version * Add support for TLS doc. * Add changelog. * fix import order * add support and doc for round robin. * fix conversion * Update docs/sources/reference/components/prometheus/prometheus.write.queue.md Co-authored-by: Clayton Cornell <[email protected]> * Add test * fix merge * Update internal/component/prometheus/write/queue/types.go Co-authored-by: William Dumont <[email protected]> --------- Co-authored-by: Clayton Cornell <[email protected]> Co-authored-by: William Dumont <[email protected]> * #229 Add OpenTelemetry Collector Server Auth Extensions to Receivers (#2203) * Work on adding auth so far * Cleanup * Made a ton of progress * Fix test fails? * Refactor * Add auth blocks to implementing extensions * Refactor to use feature flag * Comments * Cleanup * Spacing * Update docs * Update CHANGELOG * Last auth extension missing * We also need grpc auth * Fix opencensus docs * Fix extra comment * Update comment with findings * Properly fix merge conflict * Save file * Spelling error * That has been released now * Add auth support to influxdb receiver * Fix failing auth test/MAIL * Comment cleanup * MAIL for documentation * docs MAIL * MAIL * Move from Auth to Authentication * Update triton-go dependency to avoid embedded RSA key (#2380) * Fix examples for filter and transform processors (#2379) * fix examples filter and transform processors * remove unecessary docs about escaping strings and backticks * fix(loki.secretfilter): Fix partial masking for short secrets and support multiple allowlists per rule (#2320) * Fix partial masking bug and support new allowlist format * Add docs and changelog * Update docs * Add comments * Add comments * Minor docs update * Add more tests * Change criteria for partial redaction * Changes to partial masking rules * Fix comment location * Clarify usage of secret types * Clarify usage of secret types * Update docs/sources/reference/components/loki/loki.secretfilter.md Co-authored-by: Clayton Cornell <[email protected]> * Suggestions * Suggestions --------- Co-authored-by: Clayton Cornell <[email protected]> * Fix only run on fork guard (#2378) * Fix only run on fork guard The previous guard fails because `github.repository` resolves to the base repository on `pull_request` events. * Fix syntax * Fix relabel processed bug (#2394) * Fix issue where alloy_prometheus_relabel_metrics_processed was not being incremented. * Add unit tests * Update WAL to version that supports v2. (#2397) * Update WAL to version that supports v2. * Update WAL to version that supports v2. * Add samples check. * Clean up Alloy component docs (#2387) * First pass at cleanup, pretty tables, sort lists * Sort content, add badge * Fix link * Set link URL correctly * Still fxing link targets * One more tidy pass * database_observability: report health of component and collectors (#2392) Report unhealthy in case of errors when starting up the collectors or of any collector is stopped during operations. * update for rc.1 (#2401) * Update version. * fix version * fix version --------- Signed-off-by: matt durham <[email protected]> Co-authored-by: Adam ABICHOU <[email protected]> Co-authored-by: Jay Clifford <[email protected]> Co-authored-by: dbluxo <[email protected]> Co-authored-by: Clayton Cornell <[email protected]> Co-authored-by: William Dumont <[email protected]> Co-authored-by: Aidan Leuck <[email protected]> Co-authored-by: Sam DeHaan <[email protected]> Co-authored-by: Romain Gaillard <[email protected]> Co-authored-by: Jack Baldry <[email protected]> Co-authored-by: Cristian Greco <[email protected]>

aidanleuck and others added 17 commits November 26, 2024 17:08

Work on adding auth so far

6b7cdbb

Cleanup

d3812c2

Made a ton of progress

c7155c7

Fix test fails?

37e5880

Refactor

3732a54

Add auth blocks to implementing extensions

8e82871

Refactor to use feature flag

93377d0

Comments

497276a

Cleanup

cdb1c86

Spacing

a7d45c7

Update docs

3f21422

Update CHANGELOG

6f477ff

Last auth extension missing

6510731

We also need grpc auth

9998e92

Fix opencensus docs

dade331

Fix extra comment

055bb1b

Update comment with findings

fd537ff

aidanleuck requested review from clayton-cornell and a team as code owners December 2, 2024 17:01

aidanleuck added 2 commits December 2, 2024 10:07

Merge with main to fix conflicts

e8c8e36

Properly fix merge conflict

ddbca7b

clayton-cornell added the type/docs Docs Squad label across all Grafana Labs repos label Dec 9, 2024

aidanleuck added 5 commits December 9, 2024 21:13

Fix changelog

592e535

Save file

883ccee

Spelling error

b6ba127

That has been released now

cf178fa

Add auth support to influxdb receiver

23ab3be

aidanleuck added 3 commits December 15, 2024 16:59

Fix failing auth test/MAIL

c0d3ed0

Merge remote-tracking branch 'origin' into feature/add_auth_to_otlp_r…

1139424

…eceiver

Comment cleanup

4665aaf

clayton-cornell reviewed Jan 2, 2025

View reviewed changes

aidanleuck requested a review from a team as a code owner January 3, 2025 02:52

aidanleuck force-pushed the feature/add_auth_to_otlp_receiver branch from 3913159 to 4665aaf Compare January 3, 2025 03:02

aidanleuck added 2 commits January 2, 2025 20:10

Merge branch 'main' of https://github.com/aidanleuck/alloy into featu…

08f0738

…re/add_auth_to_otlp_receiver

MAIL for documentation

d2afdc8

clayton-cornell approved these changes Jan 3, 2025

View reviewed changes

aidanleuck added 2 commits January 4, 2025 13:39

Merge remote-tracking branch 'origin' into feature/add_auth_to_otlp_r…

345f138

…eceiver

docs MAIL

544d5ca

mattdurham reviewed Jan 9, 2025

View reviewed changes

mattdurham requested changes Jan 9, 2025

View reviewed changes

aidanleuck added 2 commits January 9, 2025 10:52

Fix merge conflicts

c66fb9e

MAIL

92ab0fa

aidanleuck commented Jan 9, 2025

View reviewed changes

Move from Auth to Authentication

c6f4dd3

mattdurham approved these changes Jan 10, 2025

View reviewed changes

mattdurham enabled auto-merge (squash) January 10, 2025 14:15

mattdurham merged commit e8f5bcd into grafana:main Jan 10, 2025
2 checks passed

github-actions bot added the frozen-due-to-age label Feb 10, 2025

github-actions bot locked as resolved and limited conversation to collaborators Feb 10, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

#229 Add OpenTelemetry Collector Server Auth Extensions to Receivers #2203

#229 Add OpenTelemetry Collector Server Auth Extensions to Receivers #2203

aidanleuck commented Dec 2, 2024 •

edited

Loading

CLAassistant commented Dec 2, 2024 •

edited

Loading

clayton-cornell commented Dec 9, 2024 •

edited

Loading

aidanleuck commented Dec 10, 2024 •

edited

Loading

aidanleuck commented Dec 16, 2024

clayton-cornell Jan 2, 2025

aidanleuck commented Jan 3, 2025

clayton-cornell left a comment

clayton-cornell Jan 3, 2025

clayton-cornell Jan 3, 2025

clayton-cornell Jan 3, 2025

clayton-cornell Jan 3, 2025

clayton-cornell Jan 3, 2025

clayton-cornell Jan 3, 2025

aidanleuck commented Jan 4, 2025 •

edited

Loading

clayton-cornell commented Jan 7, 2025

mattdurham Jan 9, 2025

aidanleuck Jan 9, 2025 •

edited

Loading

mattdurham Jan 9, 2025

mattdurham Jan 9, 2025

mattdurham left a comment

aidanleuck Jan 9, 2025 •

edited

Loading

mattdurham commented Jan 9, 2025

aidanleuck commented Jan 9, 2025 •

edited

Loading

mattdurham left a comment

		@@ -10,6 +10,8 @@ title: otelcol.auth.oauth2

		`otelcol.auth.oauth2` exposes a `handler` that can be used by other `otelcol` components to authenticate requests using OAuth 2.0.

		This extension only supports client authentication.

	This `component` supports both server and client authentication.
	This component supports both server and client authentication.

	This `component` only supports client authentication.
	This component only supports client authentication.

#229 Add OpenTelemetry Collector Server Auth Extensions to Receivers #2203

#229 Add OpenTelemetry Collector Server Auth Extensions to Receivers #2203

Conversation

aidanleuck commented Dec 2, 2024 • edited Loading

PR Description

Which issue(s) this PR fixes

Notes to the Reviewer

PR Checklist

CLAassistant commented Dec 2, 2024 • edited Loading

clayton-cornell commented Dec 9, 2024 • edited Loading

aidanleuck commented Dec 10, 2024 • edited Loading

aidanleuck commented Dec 16, 2024

Choose a reason for hiding this comment

aidanleuck commented Jan 3, 2025

clayton-cornell left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aidanleuck commented Jan 4, 2025 • edited Loading

clayton-cornell commented Jan 7, 2025

Choose a reason for hiding this comment

aidanleuck Jan 9, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattdurham left a comment

Choose a reason for hiding this comment

aidanleuck Jan 9, 2025 • edited Loading

Choose a reason for hiding this comment

mattdurham commented Jan 9, 2025

aidanleuck commented Jan 9, 2025 • edited Loading

mattdurham left a comment

Choose a reason for hiding this comment

aidanleuck commented Dec 2, 2024 •

edited

Loading

CLAassistant commented Dec 2, 2024 •

edited

Loading

clayton-cornell commented Dec 9, 2024 •

edited

Loading

aidanleuck commented Dec 10, 2024 •

edited

Loading

aidanleuck commented Jan 4, 2025 •

edited

Loading

aidanleuck Jan 9, 2025 •

edited

Loading

aidanleuck Jan 9, 2025 •

edited

Loading

aidanleuck commented Jan 9, 2025 •

edited

Loading