Skip to content

Commit ecf3ce8

Browse files
AlexTMjugadorshssoichiro
authored and
Pr0methean
committed
Revamp CI workflow to upload artifacts, cross-compile ARM64 binaries, and more (shssoichiro#534)
As commented in issues shssoichiro#444 and shssoichiro#518, there is some user interest for distributing binaries for each unstable commit, and target ARM64 platforms. Personally, I think both suggestions are useful for the project, as uploading binary artifacts for each commit might help interested users to catch regressions and give feedback earlier, and powerful ARM64 platforms are becoming increasingly popular due to some cloud services (e.g., Amazon EC2, Azure VMs, Oracle Cloud) offering cheaper plans for this hardware, in addition to the well-known push for ARM by Apple with their custom M1 chips. These changes make the CI target ARM64 as a first-class citizen. Because the public GitHub actions runners can only be hosted on x64 for now, I resorted to cross-compilation, [Debian's multiarch](https://elinux.org/images/d/d8/Multiarch_and_Why_You_Should_Care-_Running%2C_Installing_and_Crossbuilding_With_Multiple_Architectures.pdf), and QEMU to build, get ARM64 C library dependencies, and run tests, respectively. When the CI workflow finishes, a release CLI binary artifact is now uploaded, which can be downloaded from the workflow run page on the GitHub web interface. In addition, these changes also introduce some cleanup and miscellaneous improvements and changes to the CI workflow: - Tests are run using [`nextest`](https://nexte.st/) instead of `cargo test`, which substantially speeds up their execution. (On my development workstation, `cargo test --release` takes around 10.67 s, while `cargo nextest run --release` takes around 6.02 s.) - The dependencies on unmaintained `actions-rs` actions were dropped in favor of running Cargo commands directly, or using `giraffate/clippy-action` for pretty inline annotations for Clippy. This gets rid of the deprecation warnings for each workflow run. - Most CI steps are run with a nightly Rust toolchain now, which allows to take advantage of the latest Clippy lints and codegen improvements. In my experience, when not relying on specific nightly features or compiler internals, Rust does a pretty good job at making it possible to rely on a rolling-release compiler for CI, as breakage is extremely rare and thus offset by the improved features. - The MSRV check was moved to a separate job with less steps, so that it takes less of a toll on total workflow run minutes. - [x] Generate universal macOS binaries with `lipo` (i.e., containing both `aarch64` and `x64` code) - [x] Tirelessly fix the stupid errors that tend to happen when deploying a new CI workflow for the first time - [x] Think what to do with the `deploy.yml` workflow. Should it fetch artifacts from the CI job instead of building them again? - [x] Maybe bring back 32-bit Windows binaries. Are they actually useful for somebody, or just a way to remember the good old days? --------- Co-authored-by: Josh Holmer <[email protected]>
1 parent 4fe1e45 commit ecf3ce8

File tree

7 files changed

+217
-197
lines changed

7 files changed

+217
-197
lines changed

‎.cargo/config.toml

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Remove this if targeting AArch64 from an AArch64 Linux box
2+
[target.'cfg(all(target_os = "linux", target_arch = "aarch64"))']
3+
runner = 'qemu-aarch64'
4+
5+
[target.aarch64-unknown-linux-gnu]
6+
linker = 'aarch64-linux-gnu-gcc'
7+
8+
[target.aarch64-unknown-linux-musl]
9+
linker = 'aarch64-linux-musl-gcc'

‎.github/workflows/deploy.yml

+57-138
Original file line numberDiff line numberDiff line change
@@ -5,157 +5,76 @@ on:
55
tags:
66
- 'v*.*.*'
77

8-
jobs:
9-
create-windows-binaries:
10-
runs-on: windows-latest
11-
if: github.repository == 'shssoichiro/oxipng'
12-
13-
strategy:
14-
matrix:
15-
conf: [x86_64]
16-
# Temporarily disable i686 binaries, they are failing on linking libdeflate
17-
# and I don't have a Windows machine set up to experiment with fixing it.
18-
# conf: [x86_64, i686]
19-
20-
steps:
21-
- uses: actions/checkout@v3
22-
with:
23-
persist-credentials: false
24-
25-
- name: Install stable
26-
uses: actions-rs/toolchain@v1
27-
with:
28-
profile: minimal
29-
toolchain: stable
30-
target: ${{ matrix.conf }}-pc-windows-msvc
31-
override: true
32-
33-
- name: Build oxipng
34-
run: |
35-
cargo build --release --target ${{ matrix.conf }}-pc-windows-msvc
36-
37-
- name: Get the version
38-
shell: bash
39-
id: tagName
40-
run: |
41-
VERSION=$(cargo pkgid | cut -d# -f2 | cut -d: -f2)
42-
echo "::set-output name=tag::$VERSION"
8+
permissions:
9+
actions: read
10+
contents: write
4311

44-
- name: Build package
45-
id: package
46-
shell: bash
47-
run: |
48-
ARCHIVE_TARGET="${{ matrix.conf }}-pc-windows-msvc"
49-
ARCHIVE_NAME="oxipng-${{ steps.tagName.outputs.tag }}-$ARCHIVE_TARGET"
50-
ARCHIVE_FILE="${ARCHIVE_NAME}.zip"
51-
mv LICENSE LICENSE.txt
52-
7z a ${ARCHIVE_FILE} \
53-
./target/${{ matrix.conf }}-pc-windows-msvc/release/oxipng.exe \
54-
./CHANGELOG.md ./LICENSE.txt ./README.md
55-
echo "::set-output name=file::${ARCHIVE_FILE}"
56-
echo "::set-output name=name::${ARCHIVE_NAME}.zip"
12+
jobs:
13+
deploy:
14+
name: Deploy release
5715

58-
- name: Upload artifacts
59-
uses: actions/upload-artifact@v3
60-
with:
61-
name: ${{ steps.package.outputs.name }}
62-
path: ${{ steps.package.outputs.file }}
16+
runs-on: ubuntu-latest
17+
timeout-minutes: 30
6318

64-
create-unix-binaries:
65-
runs-on: ${{ matrix.os }}
66-
if: github.repository == 'shssoichiro/oxipng'
19+
# Prevent job from running on forks
20+
if: ${{ !github.event.repository.fork }}
6721

6822
strategy:
6923
matrix:
70-
os: [ubuntu-latest, macos-latest]
71-
include:
72-
- os: ubuntu-latest
73-
target: x86_64-unknown-linux-musl
74-
- os: macos-latest
75-
target: x86_64-apple-darwin
24+
target:
25+
- x86_64-unknown-linux-gnu
26+
- x86_64-unknown-linux-musl
27+
- aarch64-unknown-linux-gnu
28+
- aarch64-unknown-linux-musl
29+
- x86_64-pc-windows-msvc
30+
- i686-pc-windows-msvc
31+
- x86_64-apple-darwin
32+
- aarch64-apple-darwin
7633

7734
steps:
78-
- uses: actions/checkout@v3
79-
with:
80-
persist-credentials: false
81-
82-
- name: Install Rust
83-
uses: actions-rs/toolchain@v1
84-
with:
85-
profile: minimal
86-
toolchain: stable
87-
target: ${{ matrix.target }}
88-
override: true
89-
90-
- name: Install musl
91-
if: contains(matrix.target, 'linux-musl')
92-
run: |
93-
sudo apt-get install musl-tools
94-
95-
- name: Build oxipng
96-
run: |
97-
cargo build --release --target ${{ matrix.target }}
98-
99-
- name: Strip binary
100-
run: |
101-
strip target/${{ matrix.target }}/release/oxipng
102-
103-
- name: Get the version
104-
id: tagName
105-
run: |
106-
VERSION=$(cargo pkgid | cut -d# -f2 | cut -d: -f2)
107-
echo "::set-output name=tag::$VERSION"
108-
109-
- name: Build package
110-
id: package
111-
run: |
112-
ARCHIVE_TARGET=${{ matrix.target }}
113-
ARCHIVE_NAME="oxipng-${{ steps.tagName.outputs.tag }}-$ARCHIVE_TARGET"
114-
ARCHIVE_FILE="${ARCHIVE_NAME}.tar.gz"
115-
mkdir "/tmp/${ARCHIVE_NAME}"
116-
cp README.md CHANGELOG.md LICENSE \
117-
target/${{ matrix.target }}/release/oxipng \
118-
/tmp/${ARCHIVE_NAME}
119-
tar -czf ${PWD}/${ARCHIVE_FILE} -C /tmp/ ${ARCHIVE_NAME}
120-
echo ::set-output "name=file::${ARCHIVE_FILE}"
121-
echo ::set-output "name=name::${ARCHIVE_NAME}.tar.gz"
122-
123-
- name: Upload artifacts
124-
uses: actions/upload-artifact@v3
125-
with:
126-
name: ${{ steps.package.outputs.name }}
127-
path: ${{ steps.package.outputs.file }}
35+
- name: Checkout source
36+
uses: actions/checkout@v3
12837

38+
- name: Get the Oxipng version
39+
id: oxipngMeta
40+
run: echo "version=$(cargo metadata --format-version 1 --no-deps | jq -r '.packages[] | select(.name == "oxipng").version')"
41+
>> "$GITHUB_OUTPUT"
12942

130-
deploy:
131-
runs-on: ubuntu-latest
132-
if: github.repository == 'shssoichiro/oxipng'
133-
needs: [create-windows-binaries, create-unix-binaries]
134-
135-
steps:
136-
- uses: actions/checkout@v3
43+
- name: Retrieve ${{ matrix.target }} binary
44+
uses: dawidd6/action-download-artifact@v2
13745
with:
138-
persist-credentials: false
46+
workflow: oxipng.yml
47+
commit: ${{ env.GITHUB_SHA }}
48+
name: Oxipng binary (${{ matrix.target }})
49+
path: target
13950

140-
- name: Get version and release description
141-
id: tagName
51+
- name: Build archives
52+
working-directory: target
14253
run: |
143-
VERSION=$(cargo pkgid | cut -d# -f2 | cut -d: -f2)
144-
tail -n +2 CHANGELOG.md | sed -e '/^$/,$d' > CHANGELOG.txt
145-
echo "::set-output name=tag::$VERSION"
146-
147-
- name: Download artifacts
148-
uses: actions/download-artifact@v3
149-
with:
150-
path: ./binaries
151-
152-
- name: Create a release
54+
ARCHIVE_NAME="oxipng-${{ steps.oxipngMeta.outputs.version }}-${{ matrix.target }}"
55+
56+
mkdir "$ARCHIVE_NAME"
57+
cp ../CHANGELOG.md ../README.md "$ARCHIVE_NAME"
58+
59+
case '${{ matrix.target }}' in
60+
*-windows-*)
61+
cp ../LICENSE "$ARCHIVE_NAME/LICENSE.txt"
62+
cp oxipng.exe "$ARCHIVE_NAME"
63+
zip "${ARCHIVE_NAME}.zip" "$ARCHIVE_NAME"/*;;
64+
*)
65+
cp ../LICENSE "$ARCHIVE_NAME"
66+
cp oxipng "$ARCHIVE_NAME"
67+
tar -vczf "${ARCHIVE_NAME}.tar.gz" "$ARCHIVE_NAME"/*;;
68+
esac
69+
70+
- name: Create release notes
71+
run: tail -n +2 CHANGELOG.md | sed -e '/^$/,$d' > RELEASE_NOTES.txt
72+
73+
- name: Create release
15374
uses: softprops/action-gh-release@v1
15475
with:
155-
name: v${{ steps.tagName.outputs.tag }}
156-
body_path: CHANGELOG.txt
76+
name: v${{ steps.oxipngMeta.outputs.version }}
77+
body_path: RELEASE_NOTES.txt
15778
files: |
158-
./binaries/**/*.zip
159-
./binaries/**/*.tar.gz
160-
env:
161-
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
79+
target/*.zip
80+
target/*.tar.gz

0 commit comments

Comments
 (0)