Common Airflow Issues Encountered During Open Metadata Installation

[Prajwal214]

Issue 1: DAGs Run Only a Few Times

Problem: You are able to run the DAGs only a few times. After some minutes, if you try to run the ingestion in OMD, it fails, unable to find the DAG.

Issue 2: Airflow Unable to Locate the Config/JSON File for the DAG

Problem: Airflow is unable to locate the config/json file for the DAG.

Issue 3: Airflow Returns Broken plugin: [openmetadata_managed_apis.plugin] No module named 'metadata'

Problem: Airflow returns the error message: Broken plugin: [openmetadata_managed_apis.plugin] No module named 'metadata'.

Issue 4: Open MetaData Throws Failed to connect to Airflow due to java.net.ConnectException. Is the host available at http://ingestion:8080?

Problem: Open MetaData throws Failed to connect to Airflow due to java.net.ConnectException. Is the host available at http://ingestion:8080?.

[Prajwal214]

Issue 1: DAGs Run Only a Few Times

Solution:

1. Ensure that appropriate ownership or permission is given to /opt/airflow directory.
2. Install openmetadata-managed-apis in all the Airflow Pods(web, worker, scheduler and triggerer) for everything to work, not just the web pod.

Issue 2: Airflow Unable to Locate the Config/JSON File for the DAG

Solution:

1. Confirm that the path to the config/json file is correct in your DAG definition.
2. Make sure the file has the proper read permissions.
3. Verify that the Airflow worker nodes have access to the file location if running in a distributed setup.
4. Verify the custom helm values you have for deploying openmetadata-dependencies helm chart.
5. Verify the Config path, AIRFLOW__OPENMETADATA_AIRFLOW_APIS__DAG_GENERATED_CONFIGS should be pointing to shared path accessible to all the pods.

Issue 3: Airflow Returns Broken plugin: [openmetadata_managed_apis.plugin] No module named 'metadata'

Solution:

1. Install openmetadata-managed-apis in all the Airflow Pods(web, worker, scheduler and triggerer) for everything to work, not just the web pod.
2. Verfiy if any other plugin is not causing any conflict with openmetadata-managed-apis.
3. Verify the version of your OpenMetadata server matches the openmetadata-ingestion client version installed in Airflow.

Issue 4: Open MetaData Throws Failed to connect to Airflow due to java.net.ConnectException. Is the host available at http://ingestion:8080?

Solution:

1. Confirm that the Airflow webserver is running and accessible at the specified URL.
2. Check if there are any network issues or firewall rules blocking the connection.
3. Verify that the ingestion service is properly configured and running in your environment.
4. Ensure that the correct host and port are specified in the Open MetaData configuration.
5. Verify the version of your OpenMetadata server matches the openmetadata-ingestion client version installed in Airflow.
6. Verify if the OpenMetadata server you can reach the Airflow host, and the call to /health gives us the proper response:

$ curl -XGET ${PIPELINE_SERVICE_CLIENT_ENDPOINT}/api/v1/openmetadata/health
{"status": "healthy", "version": "x.y.z"}

Refer to the documentation