Native SPM support in Elasticsearch

[yurishkuro]

• Description: Jaeger is an open-source, distributed tracing platform designed to monitor and troubleshoot transactions in distributed systems. In its basic deployment it allows to collect tracing data, store it in a database, and query & analyze individual traces in the UI. This workflow is great for deep-diving into individual requests, but it does not answer some higher level questions like “which endpoints in my service are the slowest?” To address those questions Jaeger has a special feature called SPM (Service Performance Management), which allows the user to see the trends of performance of services and endpoints and to drill down into the outliers. However, this feature requires a more complicated deployment where a special real-time processor is running and extracting metrics from the traces and storing those metrics in a Prometheus-compatible remote storage. Some of the storage backends supported by Jaeger, such as Elasticsearch & OpenSearch, can provide the same aggregate answers directly from the trace data, which can significantly simplify the deployment.
• Expected Outcome:
  • Support SPM functionality directly in Elasticsearch / OpenSearch backends by implementing the metrics query API
  • Enhance existing e2e integration tests to continuously test this new capability
• Recommended Skills: Go, basic familiarity with Elasticsearch
• Expected project size: large (~350 hour projects)