Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pkg/ottl] Move debug log into Statement.Execute #36456

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

TylerHelmuth
Copy link
Member

@TylerHelmuth TylerHelmuth commented Nov 19, 2024

Description

@tdarwin noticed that the routingconnector wasnt emitting ottl logs when debug logging was enabled. This turned out to be because we added the debug statement to StatementSequence.Execute but the routingconnector uses Statement.Execute, but it only works with single statements.

This PR moves the debug log into the Statement.Execute function so that more components benefit from the debug logs.

Testing

Tested locally using this config:

receivers:
  otlpjsonfile:
    include:
      - /Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json

connectors:
  routing:
    default_pipelines: [traces/a]
    table:
      - statement: route() where attributes["service.name"] == "checkpoint-test"
        pipelines: [traces/b]

processors:
  transform:
    error_mode: ignore
    trace_statements:
      - context: resource
        statements:
          - set(attributes["test"], "pass")


exporters:
  debug/a:
  debug/b:

service:
  telemetry:
    logs:
      level: debug
  pipelines:
    traces:
      receivers:
        - otlpjsonfile
      exporters:
        - routing
    traces/a:
      receivers:
        - routing
      exporters:
        - debug/a
    traces/b:
      receivers:
        - routing
      processors:
        - transform
      exporters:
        - debug/b

The collector spit out these logs:

❯ ./bin/otelcontribcol_darwin_arm64 --config ./local/config.yaml
2024-11-19T12:29:41.671-0700    info    [email protected]/service.go:166 Setting up own telemetry...
2024-11-19T12:29:41.671-0700    info    telemetry/metrics.go:70 Serving metrics {"address": "localhost:8888", "metrics level": "Normal"}
2024-11-19T12:29:41.672-0700    info    builders/builders.go:26 Development component. May change in the future.        {"kind": "exporter", "data_type": "traces", "name": "debug/b"}
2024-11-19T12:29:41.672-0700    debug   builders/builders.go:24 Alpha component. May change in the future.      {"kind": "processor", "name": "transform", "pipeline": "traces/b"}
2024-11-19T12:29:41.672-0700    info    builders/builders.go:26 Development component. May change in the future.        {"kind": "exporter", "data_type": "traces", "name": "debug/a"}
2024-11-19T12:29:41.672-0700    debug   builders/builders.go:24 Alpha component. May change in the future.      {"kind": "connector", "name": "routing", "exporter_in_pipeline": "traces", "receiver_in_pipeline": "traces"}
2024-11-19T12:29:41.673-0700    debug   builders/builders.go:24 Alpha component. May change in the future.      {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces"}
2024-11-19T12:29:41.673-0700    info    [email protected]/service.go:238 Starting otelcontribcol...      {"Version": "0.114.0-dev", "NumCPU": 10}
2024-11-19T12:29:41.673-0700    info    extensions/extensions.go:39     Starting extensions...
2024-11-19T12:29:41.673-0700    info    [email protected]/service.go:261 Everything is ready. Begin running and processing data.
2024-11-19T12:29:41.875-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:41.875-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.076-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.076-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.275-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.275-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.474-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.474-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.675-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.675-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.875-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.875-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.076-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.076-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.274-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.274-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.475-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.475-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.675-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.675-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:44.474-0700    info    fileconsumer/file.go:265        Started watching file   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "path": "/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"}
2024-11-19T12:29:44.475-0700    debug   [email protected]/parser.go:34      TransformContext after statement execution      {"kind": "connector", "name": "routing", "exporter_in_pipeline": "traces", "receiver_in_pipeline": "traces", "statement": "route() where attributes[\"service.name\"] == \"checkpoint-test\"", "condition matched": true, "TransformContext": {"resource": {"attributes": {"service.name": "checkpoint-test", "platform.env": "prd"}, "dropped_attribute_count": 0}, "cache": {}}}
2024-11-19T12:29:44.475-0700    debug   [email protected]/parser.go:340     initial TransformContext before executing StatementSequence     {"kind": "processor", "name": "transform", "pipeline": "traces/b", "TransformContext": {"resource": {"attributes": {"service.name": "checkpoint-test", "platform.env": "prd"}, "dropped_attribute_count": 0}, "cache": {}}}
2024-11-19T12:29:44.475-0700    debug   [email protected]/parser.go:34      TransformContext after statement execution      {"kind": "processor", "name": "transform", "pipeline": "traces/b", "statement": "set(attributes[\"test\"], \"pass\")", "condition matched": true, "TransformContext": {"resource": {"attributes": {"service.name": "checkpoint-test", "platform.env": "prd", "test": "pass"}, "dropped_attribute_count": 0}, "cache": {}}}
2024-11-19T12:29:44.475-0700    info    Traces  {"kind": "exporter", "data_type": "traces", "name": "debug/b", "resource spans": 1, "spans": 1}
^C2024-11-19T12:29:48.931-0700  info    [email protected]/collector.go:328       Received signal from OS {"signal": "interrupt"}
2024-11-19T12:29:48.931-0700    info    [email protected]/service.go:303 Starting shutdown...
2024-11-19T12:29:48.931-0700    info    extensions/extensions.go:66     Stopping extensions...
2024-11-19T12:29:48.931-0700    info    [email protected]/service.go:317 Shutdown complete.

pkg/ottl/parser.go Outdated Show resolved Hide resolved
Copy link
Contributor

@edmocosta edmocosta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants