On WSL2/Windows, constantly getting "Heartbeat failed with Exception: PID of job runner does not match" warnings

[yarong-lifemap]

Apache Airflow version

2.10.3

If "Other Airflow 2 version" selected, which one?

No response

What happened?

This has happened on a few machines where we have a fresh installation of Airflow/Postgres 16.
The console keeps reporting the following 2 lines:
Heartbeat failed with Exception: PID of job runner does not match
Recorded pid 40205 does not match the current pid 40236

What you think should happen instead?

More information / no warning / how to fix

How to reproduce

Installed fresh Airflow on WSL 2, Ubuntu 24.04, inititalized database

Operating System

Windows 11 Professional / WSL 2

Versions of Apache Airflow Providers

No response

Deployment

Other

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[potiuk]

I think you need to provide a bit more information. Windows/WSL2 is not really a target platform for airflow runtime for "production" and we do not run any tests with it.

But maybe - if you can do some analysis and provide more information we can attempt to help you in your problem.

You can try to gather some more logs and information - i.e. gather more detailed logs from example tasks - to see which processes have been created - if you look and trace it in the logs and send us the info, maybe we can help you to figure out some workarounds. Track what processes are being created. Also if you are using run_as_user - feature, there might be a problem with WSL2 or your impersonation user settings.

There few parameters you can try - https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#execute-tasks-new-python-interpreter might be something worth trying.

Converting to a discussion, as Windows/WSL2 is not a target runtime platform - but there might be a discussion coming out of it.

[yarong-lifemap]

We use it on our development and production machine under Windows/WSL since we're running on .Net. That said, we also run on-prem and single machine implementations, though our DAGs have > 300 tasks in them.

I'll see if I can try to play with the execute_tasks_new_python_interpreter or otherwise try to trace the issue. For now we've just disabled the message since it's polluting the log.

Thanks again for all your hard work - Airflow is an amazing orchestrator that has simplified our lives significantly.