-
Notifications
You must be signed in to change notification settings - Fork 354
gha/add llvmlite osx 64 wheel builder #1187
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
seibert
merged 15 commits into
numba:main
from
swap357:gha/add_llvmlite_osx-64_wheel_builder
Apr 17, 2025
Merged
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
d8c4ff1
add llvmlite osx-64 wheel build and test workflow
swap357 731645c
add debug output for target directory and dist contents
swap357 bb96b58
add debugging for osx library checks on runner
swap357 a1b8e78
select xcode 14.1 and set macos deployment target 13.0
swap357 33d4c20
remove debugging checks
swap357 702fc33
apply bash flags -elx suggestions from code review
swap357 1840367
address pre-commit check issues
swap357 b9cec3f
update wheel selection method on wheel builder workflow to use stat
swap357 8417ecb
add echo command to display workflow run ID
swap357 10c0db5
Update .github/workflows/llvmlite_osx-64_wheel_builder.yml
swap357 e9cea7b
remove unnecessary bash flags
swap357 b1087c2
remove whitespaces on wheel builder workflow
swap357 8c5de94
add missing newline at end of file on the wheel builder workflow
swap357 e6c9898
add osx-64-upload job to wheel builder workflow to numba channel with…
swap357 1cecbd2
update anaconda upload command in wheel builder workflow to use API t…
swap357 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,270 @@ | ||
| name: llvmlite_osx-64_wheel_builder | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - .github/workflows/llvmlite_osx-64_wheel_builder.yml | ||
| workflow_dispatch: | ||
| inputs: | ||
| llvmdev_run_id: | ||
| description: 'llvmdev workflow run ID (optional)' | ||
| required: false | ||
| type: string | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }} | ||
| cancel-in-progress: true | ||
|
|
||
| env: | ||
| LOCAL_LLVMDEV_ARTIFACT_PATH: ${{ github.workspace }}/llvmdev_conda_packages | ||
| FALLBACK_LLVMDEV_VERSION: "15" | ||
| CONDA_CHANNEL_NUMBA: numba/label/osx_wheel | ||
| VALIDATION_PYTHON_VERSION: "3.12" | ||
| ARTIFACT_RETENTION_DAYS: 7 | ||
|
|
||
| jobs: | ||
| osx-64-build: | ||
| name: osx-64-build | ||
| runs-on: macos-13 | ||
| defaults: | ||
| run: | ||
| shell: bash -elx {0} | ||
| strategy: | ||
| matrix: | ||
| python-version: ["3.10", "3.11", "3.12", "3.13"] | ||
| fail-fast: false | ||
|
|
||
| steps: | ||
| - name: Clone repository | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Setup Miniconda | ||
| uses: conda-incubator/setup-miniconda@v3 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
| conda-remove-defaults: true | ||
| auto-update-conda: true | ||
| auto-activate-base: true | ||
|
|
||
| - name: Download llvmdev Artifact | ||
| if: ${{ inputs.llvmdev_run_id != '' }} | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: llvmdev_for_wheel_osx-64 | ||
| path: llvmdev_conda_packages | ||
| run-id: ${{ inputs.llvmdev_run_id }} | ||
| repository: ${{ github.repository }} | ||
| github-token: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Install build dependencies | ||
| run: | | ||
| if [ "${{ inputs.llvmdev_run_id }}" != "" ]; then | ||
| CHAN="file:///${{ env.LOCAL_LLVMDEV_ARTIFACT_PATH }}" | ||
| else | ||
| CHAN="${{ env.CONDA_CHANNEL_NUMBA }}" | ||
| fi | ||
| conda install -c "$CHAN" llvmdev=${{ env.FALLBACK_LLVMDEV_VERSION }} cmake libxml2 python-build | ||
|
|
||
| - name: Select Xcode 14.1 | ||
| run: sudo xcode-select -s /Applications/Xcode_14.1.0.app | ||
|
|
||
| - name: Build wheel | ||
| env: | ||
| MACOSX_DEPLOYMENT_TARGET: "13.0" | ||
| run: python -m build | ||
|
|
||
| - name: Fix macOS wheel library paths | ||
| run: | | ||
| cd dist | ||
| whl=$(stat -f "%m %N" ./*.whl | sort -n | tail -n 1 | cut -d' ' -f2-) | ||
| echo "Processing wheel: $whl" | ||
|
|
||
| # Unpack the wheel | ||
| wheel unpack "$whl" | ||
|
|
||
| # Get into the unpacked directory | ||
| target_dir=$(echo "$whl" | cut -d "-" -f1 -f2) | ||
| cd "$target_dir" | ||
|
|
||
| # Find the dylib | ||
| target=$(find . -iname "libllvmlite.dylib") | ||
| echo "Found dylib at: $target" | ||
|
|
||
| echo "=== BEFORE ===" | ||
| otool -L "$target" | ||
|
|
||
| # Fix all @rpath libraries | ||
| install_name_tool -change "@rpath/libz.1.dylib" "/usr/lib/libz.1.dylib" "$target" | ||
| install_name_tool -change "@rpath/libc++.1.dylib" "/usr/lib/libc++.1.dylib" "$target" | ||
| install_name_tool -change "@rpath/libunwind.1.dylib" "/usr/lib/libunwind.1.dylib" "$target" | ||
|
|
||
| echo "=== AFTER ===" | ||
| otool -L "$target" | ||
|
|
||
| echo "=== $target_dir contents ===" | ||
| find . -ls | ||
| echo "=== Write back to wheel ===" | ||
| cd .. | ||
| wheel pack "$target_dir" | ||
|
|
||
| echo "=== Final wheel filename ===" | ||
| ls -la ./*.whl # List wheel in dist dir | ||
|
|
||
| - name: Upload wheel | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: llvmlite-osx-64-py${{ matrix.python-version }} | ||
| path: dist/*.whl | ||
| compression-level: 0 | ||
| retention-days: ${{ env.ARTIFACT_RETENTION_DAYS }} | ||
| if-no-files-found: error | ||
|
|
||
| - name: Show Workflow Run ID | ||
| run: "echo \"Workflow Run ID: ${{ github.run_id }}\"" | ||
|
|
||
| osx-64-validate: | ||
| name: osx-64-validate | ||
| needs: osx-64-build | ||
| runs-on: macos-13 | ||
| defaults: | ||
| run: | ||
| shell: bash -elx {0} | ||
| strategy: | ||
| matrix: | ||
| python-version: ["3.10", "3.11", "3.12", "3.13"] | ||
| fail-fast: false | ||
| steps: | ||
| - name: Clone repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Miniconda | ||
| uses: conda-incubator/setup-miniconda@v3 | ||
| with: | ||
| python-version: ${{ env.VALIDATION_PYTHON_VERSION }} | ||
| conda-remove-defaults: true | ||
| auto-update-conda: true | ||
| auto-activate-base: true | ||
|
|
||
| - name: Install validation dependencies | ||
| run: conda install -c defaults wheel twine keyring rfc3986 | ||
|
|
||
| - name: Download llvmlite wheels | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: llvmlite-osx-64-py${{ matrix.python-version }} | ||
| path: dist | ||
|
|
||
| - name: Validate wheels | ||
| run: | | ||
| cd dist | ||
|
|
||
| for WHL_FILE in *.whl; do | ||
| echo "=== Validating $WHL_FILE ===" | ||
|
|
||
| # Check wheel structure | ||
| twine check "$WHL_FILE" | ||
|
|
||
| # Check dylib paths | ||
| wheel unpack "$WHL_FILE" | ||
| WHEEL_DIR=$(echo "$WHL_FILE" | cut -d "-" -f1 -f2) | ||
| DYLIB=$(find "$WHEEL_DIR" -iname "libllvmlite.dylib") | ||
|
|
||
| if [ -z "$DYLIB" ]; then | ||
| echo "Error: libllvmlite.dylib not found in $WHL_FILE" | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "=== Checking dynamic library dependencies ===" | ||
| otool -L "$DYLIB" | ||
|
|
||
| # Verify library paths | ||
| LIBS=("libz.1.dylib" "libc++.1.dylib") | ||
| for LIB in "${LIBS[@]}"; do | ||
| if ! otool -L "$DYLIB" | grep -q "/usr/lib/$LIB"; then | ||
| echo "Error: $LIB path is incorrect in $WHL_FILE" | ||
| exit 1 | ||
| fi | ||
| done | ||
| done | ||
|
|
||
| osx-64-test: | ||
| name: osx-64-test | ||
| needs: osx-64-build | ||
| runs-on: macos-13 | ||
| defaults: | ||
| run: | ||
| shell: bash -elx {0} | ||
| strategy: | ||
| matrix: | ||
| python-version: ["3.10", "3.11", "3.12", "3.13"] | ||
| fail-fast: false | ||
|
|
||
| steps: | ||
| - name: Setup Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
|
|
||
| - name: Download llvmlite wheel | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: llvmlite-osx-64-py${{ matrix.python-version }} | ||
| path: dist | ||
|
|
||
| - name: Install and test | ||
| run: | | ||
| # Get the Python executable path | ||
| PYTHON_PATH=$(which python${{ matrix.python-version }}) | ||
|
|
||
| # Upgrade pip and install wheel | ||
| $PYTHON_PATH -m pip install --upgrade pip wheel | ||
|
|
||
| # Install wheel and run tests | ||
| cd dist | ||
| whl=$(stat -f "%m %N" ./*.whl | sort -n | tail -n 1 | cut -d' ' -f2-) | ||
| echo "Using wheel: $whl" | ||
| $PYTHON_PATH -m pip install -v "$whl" | ||
|
|
||
| # Run tests | ||
| $PYTHON_PATH -m llvmlite.tests | ||
|
|
||
| osx-64-upload: | ||
| name: osx-64-upload | ||
| needs: osx-64-test | ||
| if: github.event_name == 'workflow_dispatch' | ||
| runs-on: macos-13 | ||
| defaults: | ||
| run: | ||
| shell: bash -elx {0} | ||
| strategy: | ||
| matrix: | ||
| python-version: ["3.10", "3.11", "3.12", "3.13"] | ||
| fail-fast: false | ||
| steps: | ||
| - name: Setup Miniconda | ||
| uses: conda-incubator/setup-miniconda@v3 | ||
| with: | ||
| python-version: ${{ env.VALIDATION_PYTHON_VERSION }} | ||
| conda-remove-defaults: true | ||
| auto-update-conda: true | ||
| auto-activate-base: true | ||
|
|
||
| - name: Install anaconda-client | ||
| run: conda install -c anaconda anaconda-client | ||
|
|
||
| - name: Download llvmlite wheel | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: llvmlite-osx-64-py${{ matrix.python-version }} | ||
| path: dist | ||
|
|
||
| - name: Upload wheel to Anaconda Cloud | ||
| env: | ||
| ANACONDA_API_TOKEN: ${{ secrets.NUMBA_CHANNEL_WHEEL_UPLOAD }} | ||
| run: | | ||
| cd dist | ||
| whl=$(stat -f "%m %N" ./*.whl | sort -n | tail -n 1 | cut -d' ' -f2-) | ||
| echo "Uploading wheel: $whl" | ||
| anaconda -t $ANACONDA_API_TOKEN upload --force -u numba -l dev "$whl" | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.