Allow Airflow to run in FIPs enabled environment

[longslvr]

Currently, we are running a forked Airflow 1.10.15 with additional changes to make Airflow runnable in FIPs enabled production environment.

With fips enabled you can't use md5 hash. Related to the issues mentioned here. #14966

This is proven to be very problematic whenever we want to upgrade the airflow version and we would have to reapply the changes with a fresh fork of Airflow.

To avoid the hassle of forking every time we want to upgrade airflow, we can submit a PR to apply the changes we make so that Airflow is runnable in FIPs-enabled environments. Let me know if anyone has any thoughts.

[potiuk]

I believe (please correct me if I am wrong, pointing to specific FIPS requirements), FIPS mandates not using some algorithms if they are used for cryptography, but is does not tell you that it is forbidden to use certain algorithms if they are not used for security.

Even python 3.9 introduced a flag "usedforsecurity" when hashlib is used:

https://docs.python.org/3/library/hashlib.html

Changed in version 3.9: All hashlib constructors take a keyword-only argument usedforsecurity with default value True. A false value 
allows the use of insecure and blocked hashing algorithms in restricted environments. False indicates that the hashing algorithm is not 
used in a security context, e.g. as a non-cryptographic one-way compression function.

I think md5 in general is so universally used for many non-security context, that getting rid of it completely is a bad idea.

So there is nothing wrong to use md5 hash if it is not used in security context. I believe in any kind of certification you can mark certain usages of certain algorithms for security.

We are still Airlfow 3.7 and there is no "useforsecurity" and also we are using md5 probably in some javascript. I am not aware of any "security" context where we would use it, however if you are using it for security, then yeah, it should be changed. But I see no reason why we should change the algorithm if we are not using it in security context.

In this context, I would be very open (but of course it should be revidwed by others) if you provide a PR (or several of them), but I tihnk it should be not straight replacing md5 use but make it more "pluggable" and prepared for "useforsecurity" parameter once we go Python 3.9+.

I think the sequence/scope of implementation shoudl be:

• inventory of all usages and extracting them to a common "util" library
• adding "useforsecurity" parameter in our util, anticipating that we will replace the util in Python 3.9+ directly (and we should actually pass this parameter to hashlib when we are on 3.9+. Likely similar approach for javascript.
• if you have some specific requirements of not using md5 at all, you could add an option to use other algorithms instead in this "util" of ours. I understand there might be some hardware/environment limits (which useforsecurity and 3.9+ should generally address) but maybe it's not perfect
• the behaviour should be controlled by a configuration parameter on a global level if added (security/fips_compliance).

I think if you do it in fully backwards-compatible way so that nothing changes for the users, this should be possible.

[longslvr]

@potiuk you are correct about the FIPS requirement. Our change is mainly to use "useforsecurity=False" to still use MD5 and introduce a CACHE_OPTIONS in flask_caching to use SHA256 instead of MD5.
Thank you for the suggestion on the changes and I think that is a great suggestion! We used a force repository so we can enforce airflow is running on Python 3.9. I will derive a solution that is backward compatible with all Airflow V2 versions and python from 3.7 and up!

[potiuk]

Fantastic! Looking forward to the PR!

[ngearhart]

Any progress on this? I am interested in running in a FIPS-enabled environment.

[ngearhart]

Actually, I was able to get my production environment running with the following modifications:

• Flask caching - add option to use hashlib.sha256 instead of the default md5
• Airflow scheduler - find and replace all hashlib.md5 calls with hashlib.sha256

Obviously, this may not work in every environment, and I have not tested it thoroughly outside of my own environment. Use at your own risk.

You can quickly accomplish this by running:
sed -i "s/cache_config = {'CACHE_TYPE': 'flask_caching.backends.filesystem', 'CACHE_DIR': gettempdir()}/import hashlib\n cache_config = {'CACHE_TYPE': 'flask_caching.backends.filesystem', 'CACHE_DIR': gettempdir(), 'CACHE_OPTIONS': {'hash_method': hashlib.sha256}}/g" ./venv/lib/python3.8/site-packages/airflow/www/app.py \ && find ./venv/lib/python3.8/site-packages/airflow -type f -exec sed -i "s/hashlib.md5/hashlib.sha256/g" {} \;
(if you have Airflow installed at ./venv).

[r0ck37]

latest versions use flask-2.2.5 which still use sha1 for digest.
Flask-3.0.3 has a fix.
pallets/flask#5448

[potiuk]

Airflow 3 is going to get rid of Flask Application Builder as dependency. Currently Even 5.0* alpha/rc releases have flask<3. If you would like to help with Airflow 3 @r0ck37 development , that might be a good idea.

[andrii-korotkov-verkada]

Is getting rid of this dependency still a plan? Do you need help?

[potiuk]

Sure. We need to implement Auth Manager providing the same capabilities - KeyCloak likely is the choice. We might not be able to have a fully Fledged implementation to replace what FAB providers there (i.e. integration with Oauth and others), which means that Airflow 3 without Flask might not be able to provide sophisticated mechanisms of authentication - LDAP/OAUTH and the like.

So yes. Developing a KeyCloak (or similar) Auth manager that will fully replace FAB is a help that is most welcome - more details in
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=311627974#AIP79:RemoveFlaskAppBuilderasCoredependency-Why? (see the Why? section) and @vincbeck @o-nikolas have been running an evaluation earlier to see if it is feasible. It is, and possible, but we need someone who is enterprise-level focused and already needs those enterprise-level authentication to implement the AuthManager.

I am sure if you volunteer for that @o-nikolas and @vincbeck might provide some leads.

[o-nikolas]

We still think Keycloak is a great option. If someone has domain specific knowledge with it or authn/authz in general, we'd be more than happy to collaborate on it!

[andrii-korotkov-verkada]

Would Airflow 3 be completely FIPS-compliant? What needs to be done for this to happen?

[potiuk]

Would Airflow 3 be completely FIPS-compliant? What needs to be done for this to happen?

You tell us what you miss

[andrii-korotkov-verkada]

Okay, will work on it

[longslvr]

Airflow should already be FIPs compliance if you change the encryption suite to one of the allowed suites (SHA256 instead of MD5). If my completely compliance meaning it SHA256 should be the default instead of MD5?

…

On Wed, Nov 27, 2024, 9:11 PM Andrii Korotkov ***@***.***> wrote: Okay, will work on it — Reply to this email directly, view it on GitHub <#25625 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF3GPEWS4SY25NGD7UHZVNT2CZ3URAVCNFSM6AAAAABMU3BV5GVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNBQGEZDSNA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[andrii-korotkov-verkada]

Yeah, I guess we may need SHA256 as default, but need to confirm if that's enough.

[potiuk]

ALso some dependencies of Airflow might still be problematic (see all the discussion above). It also depends on how many providers and which you install).

[longslvr]

We have been running airflow in FIPs environment before I made the change in Airflow repo (we used to have a private fork to apply our changes to be able to run in FIPs enabled AWS cloud) Since 2.7.x, Airflow is able to run in FIPs environment with the suite change.

…

On Wed, Nov 27, 2024, 9:21 PM Andrii Korotkov ***@***.***> wrote: Yeah, I guess we may need SHA256 as default, but need to confirm if that's enough. — Reply to this email directly, view it on GitHub <#25625 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF3GPET7BDACGWQMMFU3XAD2CZ42ZAVCNFSM6AAAAABMU3BV5GVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNBQGEZTIOA> . You are receiving this because you authored the thread.Message ID: ***@***.***>