-
Notifications
You must be signed in to change notification settings - Fork 56
Open
Description
Describe the bug
When deploying heat
with openstack-helm, the default manifests configure the heat
service to use publicURL and public interface when authenticating with Keystone. This works fine with Let’s Encrypt or other publicly trusted CAs, but fails when the Keystone public endpoint is using a self-signed TLS certificate. Since pods don’t mount the CA, SSL validation fails with SSLError
, and this results in 500 Internal Server Error
from Heat client.
To Reproduce
Steps to reproduce the behavior:
- Updated the required override config for heat in
/etc/genestack/helm-configs/heat/heat-helm-overrides.yaml
. - Run the Heat deployment Script
/opt/genestack/bin/install-heat.sh
- Check that all the heat pods are running.
- Run debug while validating the Heat service.
Expected behavior
Heat should be able to authenticate with Keystone even when Keystone is using a self-signed cert.
Server (please complete the following information):
- OS: Ubuntu 22.04
Additional context
Observed Behavior:
- All heat pods are running.
- Run the debug while validating the heat service.
# kubectl --namespace openstack exec -ti openstack-admin-client -- \
openstack --os-interface internal orchestration service list --debug
<truncated-output>
Instantiating orchestration client: <class 'heatclient.v1.client.Client'>
REQ: curl -g -i -X GET http://heat-api.openstack.svc.cluster.local:8004/v1/e22812fc202b4f2abf5ddf39757cd3f0/services -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-heatclient" -H "X-Auth-Token: {SHA256}f0820602a9575d0f610cc5c8feacdd9dec641a8a542370730906e6ba5a64c991" -H "X-Region-Name: dev"
Starting new HTTP connection (1): heat-api.openstack.svc.cluster.local:8004
http://heat-api.openstack.svc.cluster.local:8004 "GET /v1/e22812fc202b4f2abf5ddf39757cd3f0/services HTTP/1.1" 500 198
RESP: [500] Connection: close Content-Length: 198 Content-Type: application/json x-openstack-request-id: req-7d96b8c2-05f7-4054-bd4d-6b8f02ec182b
RESP BODY: {"code": 500, "title": "Internal Server Error", "explanation": "The server has either erred or is incapable of performing the requested operation.", "error": {"type": "SSLError", "traceback": null}}
GET call to orchestration for http://heat-api.openstack.svc.cluster.local:8004/v1/e22812fc202b4f2abf5ddf39757cd3f0/services used request id req-7d96b8c2-05f7-4054-bd4d-6b8f02ec182b
ERROR: Internal Error
Traceback (most recent call last):
File "/var/lib/openstack/lib/python3.12/site-packages/cliff/app.py", line 410, in run_subcommand
result = cmd.run(parsed_args)
^^^^^^^^^^^^^^^^^^^^
File "/var/lib/openstack/lib/python3.12/site-packages/osc_lib/command/command.py", line 38, in run
return super(Command, self).run(parsed_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/openstack/lib/python3.12/site-packages/cliff/display.py", line 117, in run
column_names, data = self.take_action(parsed_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/openstack/lib/python3.12/site-packages/heatclient/osc/v1/service.py", line 37, in take_action
services = heat_client.services.list()
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/openstack/lib/python3.12/site-packages/heatclient/v1/services.py", line 33, in list
return self._list(url, "services")
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/openstack/lib/python3.12/site-packages/heatclient/common/base.py", line 113, in _list
body = self.client.get(url).json()
^^^^^^^^^^^^^^^^^^^^
File "/var/lib/openstack/lib/python3.12/site-packages/keystoneauth1/adapter.py", line 393, in get
return self.request(url, 'GET', **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/openstack/lib/python3.12/site-packages/heatclient/common/http.py", line 322, in request
raise exc.from_response(resp)
heatclient.exc.HTTPInternalServerError: ERROR: Internal Error
clean_up ListService: ERROR: Internal Error
END return value: 1
command terminated with exit code 1
- Exec into the heat-api pod and check
/etc/heat/heat.conf
. Under the [keystone_authtoken] section, notice that theinterface
is set topublic
. - I notice that the
endpoint_type
is set topublicURL
andinterface
is set topublic
in the default heat overrideshttps://github.com/rackerlabs/genestack/blob/main/base-helm-configs/heat/heat-helm-overrides.yaml#L93
conf:
heat:
DEFAULT:
num_engine_workers: 2
server_keystone_endpoint_type: public
clients_keystone:
endpoint_type: publicURL
----
keystone_authtoken:
auth_type: password
auth_version: v3
interface: public
- In my lab, I fixed the issue by overriding these values: endpoint_type to internal and interface to internal, and reinstalling the heat.
(genestack) root@controller-01:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack --os-interface internal orchestration service list
+------------------------------+-------------+--------------------------------------+-------------+--------+----------------------------+--------+
| Hostname | Binary | Engine ID | Host | Topic | Updated At | Status |
+------------------------------+-------------+--------------------------------------+-------------+--------+----------------------------+--------+
| heat-engine-6764f5d475-nhtv4 | heat-engine | 30b0b08b-d9b9-49a4-9ed3-47aab0575f7d | heat-engine | engine | 2025-08-28T11:45:07.000000 | up |
| heat-engine-6764f5d475-nhtv4 | heat-engine | 631e6d92-d66f-4223-b57e-cef47c0910d8 | heat-engine | engine | 2025-08-28T11:45:07.000000 | up |
| heat-engine-6764f5d475-znbcz | heat-engine | 38446092-28ee-40f9-8216-003436381178 | heat-engine | engine | 2025-08-28T11:45:06.000000 | up |
| heat-engine-6764f5d475-znbcz | heat-engine | 67f49d6e-6a79-47a7-8b52-d25e40ef771e | heat-engine | engine | 2025-08-28T11:45:06.000000 | up |
+------------------------------+-------------+--------------------------------------+-------------+--------+----------------------------+--------+
Metadata
Metadata
Assignees
Labels
No labels