Airflow Helm - Unable to create a connection

[sararamasamy]

Airflow - Helm - values for the deployment update. I am able to create the connection manually but unable to do it through configuration. Can you please help?

airflow:
connections:
- id: s3_quality
type: s3
extra: |-
{ " XXXX" }
repository: ~

logging:
#remote_logging: '{{- ternary "True" "False" .Values.elasticsearch.enabled }}'
remote_logging: 'True'
encrypt_s3_logs: 'True'
remote_base_log_folder: "s3://XXXX"
remote_log_conn_id: 's3_quality'
colored_console_log: 'False'

Listing the connections in pod:
(airflow)airflow connections get s3_quality
Connection not found.
(airflow)

[potiuk]

You cannot (and should not) create connections when installing airlfow via Helm Chart. Helm Chart is to install or upgrade the application, but Connections (if you want to keep them in the DB) should be ketp outside of the Airlfow Helm Chart.

Helm Chart does not have any features for managin such "database" entries. You can modify them manually and Helm configuration will not keep them in sync. For example if you change password in the future via DB and the override it with Helm Chart configuration, your password changes should be lost.

There are a number of approach you can approach it:

a) you can defin the connection via env vars in which case they won't be editable via DB and you can use Helm Chart as the "source ot truth"
b) you can keep the connection in external Secrets Manager - and for that Helm just needs to connecto to the secrets manager. This is recommended approach.
c) you can keep your connection configuration outside of the chart and use "airflow connection export/import" to upload the connections to the DB (but this leaves you again with the same problem that manually updated configuration can be overwritten - but then you can make sure you always do export -> update -> import workflow if you want to avoid it.