0 worker nodes inspite of having pods running

[Subhashini2610]

I am deploying Trino on a K8s environment. The coordinator is working and accessible on https. However, the worker nodes are shown as 0 on the web-ui. However, I can see the worker pod created and running successfully.
Coordinator's config.properties:

  config.properties: |
    coordinator=true
{{- if gt (int .Values.server.workers) 0 }}
    node-scheduler.include-coordinator=false
{{- else }}
    node-scheduler.include-coordinator=true
{{- end }}
    # http-server.http.port={{ .Values.service.port }}
    query.max-memory={{ .Values.server.config.query.maxMemory }}
    query.max-memory-per-node={{ .Values.server.config.query.maxMemoryPerNode }}
    memory.heap-headroom-per-node={{ .Values.server.config.memory.heapHeadroomPerNode }}
    discovery-server.enabled=true
    discovery.uri=https://{{ .Values.service.ip }}:{{ .Values.service.port }}
    http-server.https.enabled=true
    http-server.https.port={{ .Values.service.port }}
    internal-communication.https.required=true
    http-server.https.keystore.path={{ .Values.server.config.https.keystore.path }}
    # needed for Presto ODBC driver
    protocol.v1.alternate-header-name=Presto
    http-server.authentication.type={{ .Values.server.config.authenticationType }}
    http-server.https.keystore.path={{ .Values.server.config.https.keystore.path }}
    internal-communication.shared-secret=***********

Worker's config.properties:

  config.properties: |
    coordinator=false
    internal-communication.https.required=true
    http-server.https.enabled=true
    http-server.https.port={{ .Values.service.port }}
    query.max-memory={{ .Values.server.config.query.maxMemory }}
    query.max-memory-per-node={{ .Values.server.config.query.maxMemoryPerNode }}
    memory.heap-headroom-per-node={{ .Values.server.config.memory.heapHeadroomPerNode }}
    discovery.uri=https://{{ .Values.service.ip }}:{{ .Values.service.port }}
    internal-communication.shared-secret=*************

I am able to see the worker and coordinator pods running status. However, the active workers is shown as 0. What is wrong here?

[Subhashini2610]

Still facing this issue. Please help

[sondn98]

Is there any update on it?

[hashhar]

Check logs of the workers - I'm guessing the discovery.uri is not reachable from the workers.

[hashhar]

Make sure that the versions on all nodes is same and that they are using same value for node.environment and that the node.id is unique for each of them.

These are the only reasons I can think of why a coordinator wouldn't show workers - but I'd also expect in all these cases to have some errors in the worker logs.

[hashhar]

Also share your config.properties and node.properties for both workers and coordinator - I initially assumed it was same as in the initial message of this issue.

[sondn98]

@hashhar The node.properties is shared among workers so the node.environment on all worker are identical , but node.id is not set. Anyway I've had a look into trino's chart and the property node.id is also not set. Is it an required configuration? For more context, My trino cluster is being set up with oauth2 authentication

[sondn98]

It seems that trino would take the worker's hostname as its node id if the node.id property is not set

if ! grep -s -q 'node.id' /etc/trino/node.properties; then
launcher_opts+=("-Dnode.id=${HOSTNAME}")
fi

Therefore, I can make sure that the node ids are unique among workers because the HOSTNAME is unique for each worker pod in k8s

[sondn98]

Btw, my trino cluster is configured with oauth2 and secured by https. But discovery.uri is just a http url pointing to the coordinator node's 8080 (http is also enabled). Is it a valid configuration?

[samanvaysrivastav]

How you finally fixed that issue ? I am facing similar issue.

[PaulLiang1]

had similar issue when coordinator=false was accidentally removed from worker conf