Issue with OpenTelemetry Collector sending Hadoop JMX metrics to SkyWalking

[himanshumaurya09876]

Issue with OpenTelemetry Collector sending Hadoop JMX metrics to SkyWalking

Environment

• SkyWalking Version: v10.2.0
• BanyanDB Version: 0.8 (v10.3.0 is incompatible with BanyanDB 0.8, and BanyanDB 0.9 binaries are not available)
• OpenTelemetry Collector: otelcol-contrib v0.100.0
• Monitoring Target: Hadoop cluster with JMX RMI enabled

Problem Description

I am new to SkyWalking and attempting to set up monitoring for a Hadoop cluster. I have successfully enabled JMX RMI on the Hadoop side and can collect Hadoop metrics using the OpenTelemetry Collector (otel-contrib). However, when pushing these metrics to SkyWalking, I'm encountering:

• Connection failures
• Parsing issues

Current Configuration

OpenTelemetry Collector Configuration:

receivers:
  jmx:
    endpoint: service:jmx:rmi:///jndi/rmi://hostname.domain.com:50070/jmxrmi
    jar_path: /home/downloads/opentelemetry-jmx-metrics.jar
    target_system: hadoop
    collection_interval: 30s

exporters:
  otlp:
    endpoint: hostname.domain.com:11800
    tls:
      insecure: true

service:
  telemetry:
    logs:
      level: debug
      encoding: json
      output_paths: ["/var/log/otelcol.log"]
  pipelines:
    metrics:
      receivers: [jmx]
      exporters: [otlp]

SkyWalking Configuration:

• OpenTelemetry receiver is enabled on the OAP server

Expected Behavior

Metrics should flow from: Hadoop JMX → OpenTelemetry Collector → SkyWalking OAP Server

Questions

1. Are there specific configuration requirements for the SkyWalking OpenTelemetry receiver to accept JMX metrics from the OTLP exporter?
2. Do I need to enable specific handlers in the receiver-otel configuration (e.g., enabledHandlers: "otlp-metrics")?
3. Are there any known compatibility issues between otelcol-contrib v0.100.0 and SkyWalking v10.2.0?
4. Do I need to create custom OAL rules or MAL rules for Hadoop metrics?
5. Should I configure any specific enabledOtelMetricsRules in the receiver configuration?

Additional Information

• The OpenTelemetry Collector logs show metrics are being collected successfully from Hadoop
• The connection failure occurs when attempting to push to SkyWalking endpoint hostname.domain.com:11800
• Debug logs are available if needed

What I've Tried

• Verified that port 11800 is accessible from the OpenTelemetry Collector host
• Confirmed that the OTLP receiver is enabled in SkyWalking
• Checked OpenTelemetry Collector logs for detailed error messages

Any guidance on the correct configuration or troubleshooting steps would be greatly appreciated!

[wu-sheng]

Are there specific configuration requirements for the SkyWalking OpenTelemetry receiver to accept JMX metrics from the OTLP exporter?
Do I need to enable specific handlers in the receiver-otel configuration (e.g., enabledHandlers: "otlp-metrics")?

Yes, you should write them.

Are there any known compatibility issues between otelcol-contrib v0.100.0 and SkyWalking v10.2.0?

We are using 0.102.1 in the e2e, it is good.

skywalking/.github/workflows/skywalking.yaml

Line 340 in bea2941

OTEL_COLLECTOR_VERSION: 0.102.1

Do I need to create custom OAL rules or MAL rules for Hadoop metrics?

Yes, you have to set up MAL. This should be key issue why you can't see data in the database.

Should I configure any specific enabledOtelMetricsRules in the receiver configuration?

Yes, enabledOtelMetricsRules is a MAL script.