Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-125498: Update JIT builds to use LLVM 19 and use preserve_none #125499

Merged
merged 48 commits into from
Oct 30, 2024

Conversation

savannahostrowski
Copy link
Member

@savannahostrowski savannahostrowski commented Oct 15, 2024

@savannahostrowski
Copy link
Member Author

So it looks like the Python version installed with brew is failing transitively...doing a bit of spelunking here.

@savannahostrowski
Copy link
Member Author

savannahostrowski commented Oct 22, 2024

Okay, after a bit of digging here, my cursory diagnosis around the failing x86 macOS runs is essentially this (and I am by no means a Homebrew or Ruby expert):
- The default Python version of Homebrew was recently updated to 3.13 (see Homebrew/homebrew-core#182840). As part of this PR, the formula for Python 3.13 seem to vastly differ from 3.12...and 3.12 was also updated.
It seems like Homebrew is no longer handling overwriting files in the way it once did. More specifically, the combo of the setup-python step and then brew installing Python (via LLVM) is not cooperating. I did some testing here, and it looks like this issue only happens with 3.12 onward (preliminary run of https://github.com/savannahostrowski/test_repo_python_install/actions/runs/11453722971 but waiting on the fail-fast: false run for the full output).

Update as of 10/22:
I got a reply from a Homebrew maintainer: Homebrew/homebrew-core#195288. It looks like this is a known GHA issue that has existed since at least May and started popping up recently as a result of some updates that GitHub made to runner images. The workaround is to remove the symlink to the pre-installed Python so that Homebrew Python is used instead. They may also start investigating overwriting the symlinks when in GHA context on their end because this seems to be a prevalent issue.

Copy link
Member

@brandtbucher brandtbucher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a few opportunities to tidy things up:

Tools/jit/_targets.py Show resolved Hide resolved
Include/internal/pycore_jit.h Outdated Show resolved Hide resolved
Tools/jit/jit.h Outdated Show resolved Hide resolved
Tools/jit/jit.h Outdated Show resolved Hide resolved
@savannahostrowski
Copy link
Member Author

Oooo, thanks for the review @brandtbucher. Great catches for cleaning things up a bit. Much simpler.

@savannahostrowski savannahostrowski changed the title gh-125498: Replace ghccc with preserve_none in JIT builds gh-125498: Update JIT builds to use LLVM 19 and use preserve_none Oct 30, 2024
Copy link
Member

@brandtbucher brandtbucher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! @mdboom, I'm landing this, so we should probably also update the benchmark runners that need it. Let me know if you want help.

@brandtbucher brandtbucher merged commit c29bbe2 into python:main Oct 30, 2024
71 checks passed
@mdboom
Copy link
Contributor

mdboom commented Oct 30, 2024

Awesome! @mdboom, I'm landing this, so we should probably also update the benchmark runners that need it. Let me know if you want help.

Yep -- I'll do that now.

Eclips4 added a commit to mpage/cpython that referenced this pull request Nov 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants