addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: Error while dialing: dial tcp [::1]:4317: connect: connection refused" {"grpc_log": true}

[naveenkoch]

Describe the bug
Getting this error in opentelemetry collector container log
addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: Error while dialing: dial tcp [::1]:4317: connect: connection refused" {"grpc_log": true}

Steps to reproduce
I followed this URL
kubectl apply -f https://raw.githubusercontent.com/open-telemetry/opentelemetry-collector/v0.113.0/examples/k8s/otel-config.yaml

What did you expect to see?
I expect, opentelmenty receives traces and export to jeager

What did you see instead?
I see the error
addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: Error while dialing: dial tcp [::1]:4317: connect: connection refused" {"grpc_log": true}

What version did you use?
otel/opentelemetry-collector:latest

What config did you use?
Followed this
https://opentelemetry.io/blog/2023/jaeger-exporter-collector-migration/

exporters:
otlp:
endpoint: "http://localhost:4317" # Replace with a real endpoint.
tls:
insecure: true

Environment
local docker desktop

Additional context

[VihasMakwana]

Hi,
can you paste whole collector log?

[naveenkoch]

2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/service.go:166Setting up own telemetry...
2024-11-08 16:25:28 2024-11-08T15:25:28.325Zinfotelemetry/metrics.go:70Serving metrics{"address": "localhost:8888", "metrics level": "Normal"}
2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/service.go:238Starting otelcol...{"Version": "0.113.0", "NumCPU": 12}
2024-11-08 16:25:28 2024-11-08T15:25:28.328Zinfoextensions/extensions.go:39Starting extensions...
2024-11-08 16:25:28 2024-11-08T15:25:28.329Zinfoextensions/extensions.go:42Extension is starting...{"kind": "extension", "name": "health_check"}
2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/healthcheckextension.go:33Starting health_check extension{"kind": "extension", "name": "health_check", "config": {"Endpoint":"localhost:13133","TLSSetting":null,"CORS":null,"Auth":null,"MaxRequestBodySize":0,"IncludeMetadata":false,"ResponseHeaders":null,"CompressionAlgorithms":null,"ReadTimeout":0,"ReadHeaderTimeout":0,"WriteTimeout":0,"IdleTimeout":0,"Path":"/","ResponseBody":null,"CheckCollectorPipeline":{"Enabled":false,"Interval":"5m","ExporterFailureThreshold":5}}}
2024-11-08 16:25:28 2024-11-08T15:25:28.334Zinfoextensions/extensions.go:59Extension started.{"kind": "extension", "name": "health_check"}
2024-11-08 16:25:28 2024-11-08T15:25:28.335Zinfoextensions/extensions.go:42Extension is starting...{"kind": "extension", "name": "zpages"}
2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/zpagesextension.go:54Registered zPages span processor on tracer provider{"kind": "extension", "name": "zpages"}
2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/zpagesextension.go:64Registered Host's zPages{"kind": "extension", "name": "zpages"}
2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/zpagesextension.go:76Starting zPages extension{"kind": "extension", "name": "zpages", "config": {"Endpoint":"localhost:55679","TLSSetting":null,"CORS":null,"Auth":null,"MaxRequestBodySize":0,"IncludeMetadata":false,"ResponseHeaders":null,"CompressionAlgorithms":null,"ReadTimeout":0,"ReadHeaderTimeout":0,"WriteTimeout":0,"IdleTimeout":0}}
2024-11-08 16:25:28 2024-11-08T15:25:28.339Zinfoextensions/extensions.go:59Extension started.{"kind": "extension", "name": "zpages"}
2024-11-08 16:25:28 2024-11-08T15:25:28.339Zinfoextensions/extensions.go:42Extension is starting...{"kind": "extension", "name": "pprof"}
2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/pprofextension.go:61Starting net/http/pprof server{"kind": "extension", "name": "pprof", "config": {"TCPAddr":{"Endpoint":"localhost:1777","DialerConfig":{"Timeout":0}},"BlockProfileFraction":0,"MutexProfileFraction":0,"SaveToFile":""}}
2024-11-08 16:25:28 2024-11-08T15:25:28.339Zinfoextensions/extensions.go:59Extension started.{"kind": "extension", "name": "pprof"}
2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/clientconn.go:1379[core] [Channel #1 SubChannel #2]grpc: addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: Error while dialing: dial tcp [::1]:4317: connect: connection refused"{"grpc_log": true}
2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/warning.go:40Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks.{"kind": "receiver", "name": "otlp", "data_type": "logs", "documentation": "https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/otlp.go:112Starting GRPC server{"kind": "receiver", "name": "otlp", "data_type": "logs", "endpoint": "0.0.0.0:4317"}
2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/clientconn.go:1379[core] [Channel #3 SubChannel #4]grpc: addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: Error while dialing: dial tcp [::1]:4317: connect: connection refused"{"grpc_log": true}
2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/warning.go:40Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks.{"kind": "receiver", "name": "otlp", "data_type": "logs", "documentation": "https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/otlp.go:169Starting HTTP server{"kind": "receiver", "name": "otlp", "data_type": "logs", "endpoint": "0.0.0.0:4318"}
2024-11-08 16:25:28 2024-11-08T15:25:28.343Zinfohealthcheck/handler.go:132Health Check state change{"kind": "extension", "name": "health_check", "status": "ready"}
2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/service.go:261Everything is ready. Begin running and processing data.
2024-11-08 16:25:28 2024-11-08T15:25:[email protected]/clientconn.go:1379[core] [Channel #5 SubChannel #6]grpc: addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"{"grpc_log": true}
2024-11-08 16:25:29 2024-11-08T15:25:[email protected]/clientconn.go:1379[core] [Channel #1 SubChannel #2]grpc: addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"{"grpc_log": true}
2024-11-08 16:25:29 2024-11-08T15:25:[email protected]/clientconn.go:1379[core] [Channel #3 SubChannel #4]grpc: addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"{"grpc_log": true}
2024-11-08 16:25:29 2024-11-08T15:25:[email protected]/clientconn.go:1379[core] [Channel #5 SubChannel #6]grpc: addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"{"grpc_log": true}
2024-11-08 16:25:30 2024-11-08T15:25:[email protected]/clientconn.go:1379[core] [Channel #1 SubChannel #2]grpc: addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"{"grpc_log": true}
2024-11-08 16:25:30 2024-11-08T15:25:[email protected]/clientconn.go:1379[core] [Channel #5 SubChannel #6]grpc: addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"{"grpc_log": true}
2024-11-08 16:25:30 2024-11-08T15:25:[email protected]/clientconn.go:1379[core] [Channel #3 SubChannel #4]grpc: addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"{"grpc_log": true}
2024-11-08 16:25:33 2024-11-08T15:25:[email protected]/clientconn.go:1379[core] [Channel #1 SubChannel #2]grpc: addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"{"grpc_log": true}
2024-11-08 16:25:33 2024-11-08T15:25:[email protected]/clientconn.go:1379[core] [Channel #3 SubChannel #4]grpc: addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"{"grpc_log": true}
2024-11-08 16:25:33 2024-11-08T15:25:[email protected]/clientconn.go:1379[core] [Channel #5 SubChannel #6]grpc: addrConn.createTransport failed to connect to {Addr: "localhost:4317", ServerName: "localhost:4317", }. Err: connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"{"grpc_log": true}

[ms-jcorley]

The error you are seeing "tls: first record does not look like a TLS handshake" generally happens when something is attempting to call out using TLS and the other side isn't using TLS. Without seeing your full config.yaml, it's hard to give more advice, but I'd careful examine your yaml as the "insecure: true" seems right but doesn't seem to be set for the given log.