How many cpu and memory resources uses Open Telemetry Exporter for tracing in Jaeger?

[juanpq02]

Hello everyone

How many cpu and memory resources uses Open Telemetry Exporter for tracing in Jaeger?

Considering this Exporter can be deployed on hosts (K8s workers) and specially used for data tracing howy many resources do i have to consider per host or if applies per pod as Sidecar.

Appreciate your answer

Regards

[dosubot[bot]]

Hola @juanpq02! ¡Bienvenido a mi repositorio! Soy un bot que puede ayudarte a resolver problemas, responder preguntas y convertirte en un colaborador. Estoy aquí para ayudarte mientras esperamos la asistencia de un mantenedor humano.

The Jaeger documentation does not specify the default CPU and memory resource requests and limits for the OpenTelemetry Exporter. However, there are best practices and recommendations for deploying the OpenTelemetry Exporter as a sidecar in Kubernetes, which can help you estimate the resource requirements.

Recommended Configurations for Sidecar Deployment

Here is an example configuration for deploying the OpenTelemetry Collector as a sidecar in a Kubernetes pod:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: my-app-image
        ports:
        - containerPort: 8080
        env:
        - name: OTEL_EXPORTER_OTLP_ENDPOINT
          value: "http://localhost:4317"
        - name: OTEL_TRACES_SAMPLER
          value: "always_on"
      - name: otel-collector
        image: otel/opentelemetry-collector:latest
        ports:
        - containerPort: 4317
        volumeMounts:
        - name: otel-collector-config
          mountPath: /etc/otel-collector-config
        args: ["--config=/etc/otel-collector-config/config.yaml"]
      volumes:
      - name: otel-collector-config
        configMap:
          name: otel-collector-config
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: otel-collector-config
data:
  config.yaml: |
    receivers:
      otlp:
        protocols:
          grpc:
          http:
    processors:
      batch:
    exporters:
      jaeger:
        endpoint: "http://jaeger-collector:14250"
        tls:
          insecure: true
    service:
      pipelines:
        traces:
          receivers: [otlp]
          processors: [batch]
          exporters: [jaeger]

Resource Allocation Recommendations

While specific CPU and memory requirements can vary based on the load and environment, here are some general guidelines:

1. Start with Baseline Resources: Begin with a baseline allocation, such as 100m CPU and 128Mi memory, and adjust based on observed performance and load.
2. Monitor Resource Usage: Use monitoring tools like Prometheus to track resource usage and adjust allocations as needed.
3. Auto-scaling: Implement auto-scaling policies to dynamically adjust resources based on load.

Scaling Recommendations

For high-throughput environments, consider the following scaling strategies:

1. Increase Queue Size: Adjust the internal queue size to handle more spans in memory before processing.
2. Increase Processor Workers: Increase the number of workers to process spans more quickly.
3. Use Intermediate Buffers: Use Apache Kafka as a buffer between the Collector and the storage backend to handle traffic spikes.

By following these guidelines and monitoring resource usage, you can effectively allocate CPU and memory resources for the OpenTelemetry Exporter in Jaeger deployments ^[1][2].

To continue talking to Dosu, mention @dosu.