Skip to content

Commit

Permalink
Update to Swift 5.5.2 compiler and tweak the doc
Browse files Browse the repository at this point in the history
  • Loading branch information
@finagolfin
finagolfin committed Dec 29, 2021
1 parent 7e41522 commit 6ff952d
Show file tree
Hide file tree
Showing 5 changed files with 54 additions and 52 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/sdks.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -153,8 +153,8 @@ jobs:
if [ ${{ matrix.version }} = 'trunk' ]; then
echo "::set-output name=sdk-name::$SDK_NAME/usr/swiftpm-android"
else
sed -i "s%/home/butta/swift-5.5.1-RELEASE%`pwd`/${{ steps.version.outputs.tag }}%" android-${{ matrix.arch }}.json
sed -i "s%/home/butta/swift-5.5-android-${{ matrix.arch }}-${ANDROID_API_LEVEL}-sdk%`pwd`/${SDK_NAME}%" android-${{ matrix.arch }}.json
sed -i "s%/home/butta/swift-5.5.2-RELEASE%`pwd`/${{ steps.version.outputs.tag }}%" android-${{ matrix.arch }}.json
sed -i "s%/home/butta/swift-5.5.2-android-${{ matrix.arch }}-${ANDROID_API_LEVEL}-sdk%`pwd`/${SDK_NAME}%" android-${{ matrix.arch }}.json
sed -i "s%/home/butta/android-ndk-r23b%${ANDROID_NDK_LATEST_HOME}%" android-${{ matrix.arch }}.json
echo "::set-output name=sdk-name::android"
fi
Expand Down
90 changes: 46 additions & 44 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ of the Swift toolchain for AArch64, armv7, and x86_64, builds several Swift pack
against those SDKs, and then runs their tests in the Android x86_64
emulator](https://github.com/buttaface/swift-android-sdk/blob/main/.github/workflows/sdks.yml).

To build with a Swift 5.5 SDK, first download [the latest Android LTS NDK
23b](https://developer.android.com/ndk/downloads) and [Swift 5.5.1
To build with a Swift 5.5.2 SDK, first download [the latest Android LTS NDK
23b](https://developer.android.com/ndk/downloads) and [Swift 5.5.2
compiler](https://swift.org/download/#releases) (make sure to install the Swift
compiler's dependencies listed there). Unpack these archives and the SDK.

Expand All @@ -20,42 +20,42 @@ The SDK will need to be modified with the path to your NDK and Swift compiler
in the following ways (I'll show aarch64 below, the same will need to be done
for the armv7 or x86_64 SDKs):

1. Change all paths in `swift-5.5-android-aarch64-24-sdk/usr/lib/swift/android/aarch64/glibc.modulemap`
from `/home/butta/android-ndk-r23` to the path to your NDK, ie something
1. Change all paths in `swift-5.5.2-android-aarch64-24-sdk/usr/lib/swift/android/aarch64/glibc.modulemap`
from `/home/butta/android-ndk-r23b` to the path to your NDK, ie something
like `/home/yourname/android-ndk-r23b`.

2. There's a single line pointing to a header in the SDK itself, so change it
from `/home/butta/swift-5.5-android-aarch64-24-sdk` in `glibc.modulemap` to the
path where you unpacked this SDK, such as `/home/yourname/swift-5.5-android-aarch64-24-sdk`.
from `/home/butta/swift-5.5.2-android-aarch64-24-sdk` in `glibc.modulemap` to the
path where you unpacked this SDK, such as `/home/yourname/swift-5.5.2-android-aarch64-24-sdk`.

3. Change the symbolic link at `swift-5.5-android-aarch64-24-sdk/usr/lib/swift/clang`
3. Change the symbolic link at `swift-5.5.2-android-aarch64-24-sdk/usr/lib/swift/clang`
to point to the clang headers next to your swift compiler, eg

```
ln -sf /home/yourname/swift-5.5.1-RELEASE-ubuntu20.04/usr/lib/clang/10.0.0
swift-5.5-android-aarch64-24-sdk/usr/lib/swift/clang
ln -sf /home/yourname/swift-5.5.2-RELEASE-ubuntu20.04/usr/lib/clang/10.0.0
swift-5.5.2-android-aarch64-24-sdk/usr/lib/swift/clang
```

Finally, modify the cross-compilation JSON file in this repo similarly:

1. All paths to the NDK should change from `/home/butta/android-ndk-r23b`
to the path to your NDK, `/home/yourname/android-ndk-r23b`.

2. The path to the compiler should change from `/home/butta/swift-5.5.1-RELEASE-ubuntu20.04`
to the path to your Swift compiler, `/home/yourname/swift-5.5.1-RELEASE-centos8`.
2. The path to the compiler should change from `/home/butta/swift-5.5.2-RELEASE-ubuntu20.04`
to the path to your Swift compiler, `/home/yourname/swift-5.5.2-RELEASE-centos8`.

3. The path to the Android SDK should change from `/home/butta/swift-5.5-android-aarch64-24-sdk`
to the path where you unpacked the Android SDK, `/home/yourname/swift-5.5-android-aarch64-24-sdk`.
3. The path to the Android SDK should change from `/home/butta/swift-5.5.2-android-aarch64-24-sdk`
to the path where you unpacked the Android SDK, `/home/yourname/swift-5.5.2-android-aarch64-24-sdk`.

Now you're ready to cross-compile a Swift package with the cross-compilation
configuration JSON file, android-aarch64.json, and run its tests on Android.
I'll demonstrate with the swift-argument-parser package:
```
git clone --depth 1 https://github.com/apple/swift-argument-parser.git
cd swift-argument-parser/
/home/yourname/swift-5.5.1-RELEASE-ubuntu20.04/usr/bin/swift build --build-tests
/home/yourname/swift-5.5.2-RELEASE-ubuntu20.04/usr/bin/swift build --build-tests
--enable-test-discovery --destination ~/swift-android-sdk/android-aarch64.json
-Xlinker -rpath -Xlinker \$ORIGIN/swift-5.5-android-aarch64-24-sdk/usr/lib/swift/android
-Xlinker -rpath -Xlinker \$ORIGIN/swift-5.5.2-android-aarch64-24-sdk/usr/lib/swift/android
```
This will cross-compile the package for Android aarch64 and produce a test
runner executable with the `.xctest` extension, in this case at
Expand Down Expand Up @@ -88,10 +88,10 @@ uname -m # check if you're running on the right architecture, should say `aarch6
cd # move to the Termux app's home directory
pkg install openssh
scp [email protected]:{swift-5.5-android-aarch64-24-sdk.tar.xz,
scp [email protected]:{swift-5.5.2-android-aarch64-24-sdk.tar.xz,
swift-argument-parserPackageTests.xctest,math,repeat,roll} .
tar xf swift-5.5-android-aarch64-24-sdk.tar.xz
tar xf swift-5.5.2-android-aarch64-24-sdk.tar.xz
./swift-argument-parserPackageTests.xctest
```
Expand All @@ -106,11 +106,11 @@ mode.

# Building the Android SDKs

Download the Swift 5.5.1 compiler and Android NDK 23b as above. Check out this
Download the Swift 5.5.2 compiler and Android NDK 23b as above. Check out this
repo and run
`SWIFT_TAG=swift-5.5.1-RELEASE ANDROID_ARCH=aarch64 swift get-packages-and-swift-source.swift`
`SWIFT_TAG=swift-5.5.2-RELEASE ANDROID_ARCH=aarch64 swift get-packages-and-swift-source.swift`
to get some prebuilt Android libraries and the Swift source to build the SDK. If
you pass in a different tag like `swift-DEVELOPMENT-SNAPSHOT-2021-09-18-a`
you pass in a different tag like `swift-DEVELOPMENT-SNAPSHOT-2021-12-23-a`
for the latest Swift trunk snapshot and pass in the path to the corresponding
official prebuilt Swift toolchain to `build-script` below, you can build a Swift
trunk SDK too, as seen on the CI.
Expand All @@ -123,7 +123,8 @@ git apply swift-android-release.patch
wget -q https://patch-diff.githubusercontent.com/raw/apple/swift/pull/40633.diff
wget -q https://patch-diff.githubusercontent.com/raw/apple/swift/pull/40707.diff
cd swift
git apply ../40633.diff ../40707.diff
patch -p1 < ../40633.diff
patch -p1 < ../40707.diff
cd ..
```

Expand All @@ -133,18 +134,19 @@ substituted instead:
```
./swift/utils/build-script -RA --skip-build-cmark --build-llvm=0 --android
--android-ndk /home/butta/android-ndk-r23b/ --android-arch aarch64 --android-api-level 24
--android-icu-uc /home/butta/swift-5.5.1-android-aarch64-24-sdk/usr/lib/libicuuc.so
--android-icu-uc-include /home/butta/swift-5.5.1-android-aarch64-24-sdk/usr/include/
--android-icu-i18n /home/butta/swift-5.5.1-android-aarch64-24-sdk/usr/lib/libicui18n.so
--android-icu-i18n-include /home/butta/swift-5.5.1-android-aarch64-24-sdk/usr/include/
--android-icu-data /home/butta/swift-5.5.1-android-aarch64-24-sdk/usr/lib/libicudata.so
--build-swift-tools=0 --native-swift-tools-path=/home/butta/swift-5.5.1-RELEASE-ubuntu20.04/usr/bin/
--native-clang-tools-path=/home/butta/swift-5.5.1-RELEASE-ubuntu20.04/usr/bin/
--android-icu-uc /home/butta/swift-5.5.2-android-aarch64-24-sdk/usr/lib/libicuuc.so
--android-icu-uc-include /home/butta/swift-5.5.2-android-aarch64-24-sdk/usr/include/
--android-icu-i18n /home/butta/swift-5.5.2-android-aarch64-24-sdk/usr/lib/libicui18n.so
--android-icu-i18n-include /home/butta/swift-5.5.2-android-aarch64-24-sdk/usr/include/
--android-icu-data /home/butta/swift-5.5.2-android-aarch64-24-sdk/usr/lib/libicudata.so
--build-swift-tools=0 --native-swift-tools-path=/home/butta/swift-5.5.2-RELEASE-ubuntu20.04/usr/bin/
--native-clang-tools-path=/home/butta/swift-5.5.2-RELEASE-ubuntu20.04/usr/bin/
--host-cc=/usr/bin/clang-11 --host-cxx=/usr/bin/clang++-11
--cross-compile-hosts=android-aarch64 --cross-compile-deps-path=/home/butta/swift-5.5.1-android-aarch64-24-sdk
--cross-compile-hosts=android-aarch64 --cross-compile-deps-path=/home/butta/swift-5.5.2-android-aarch64-24-sdk
--skip-local-build --xctest --swift-install-components='clang-resource-dir-symlink;license;stdlib;sdk-overlay'
--install-swift --install-libdispatch --install-foundation --install-xctest
--install-destdir=/home/butta/swift-5.5.1-android-aarch64-24-sdk -j9
--install-destdir=/home/butta/swift-5.5.2-android-aarch64-24-sdk
--disable-append-cross-compile-hostname-to-destdir -j9
```
Make sure you have an up-to-date CMake and not something old like 3.16. The
`--host-cc` and `--host-cxx` flags are not needed if you have a `clang` and
Expand All @@ -156,8 +158,8 @@ Finally, copy `libc++_shared.so` from the NDK and modify the cross-compiled
`libdispatch.so` and Swift corelibs to include `$ORIGIN` and other relative
directories in their rpaths:
```
cp /home/yourname/android-ndk-r23b/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++_shared.so swift-5.5.1-android-aarch64-24-sdk/usr/lib
patchelf --set-rpath \$ORIGIN/../..:\$ORIGIN swift-5.5.1-android-aarch64-24-sdk/usr/lib/swift/android/lib*.so
cp /home/yourname/android-ndk-r23b/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++_shared.so swift-5.5.2-android-aarch64-24-sdk/usr/lib
patchelf --set-rpath \$ORIGIN/../..:\$ORIGIN swift-5.5.2-android-aarch64-24-sdk/usr/lib/swift/android/lib*.so
```

Here is a description of what the above Swift script is doing:
Expand All @@ -169,29 +171,29 @@ API 24. Specifically, it downloads the libicu, libicu-static, libandroid-spawn,
libcurl, and libxml2 packages from the [Termux package
repository](https://packages.termux.org/apt/termux-main/pool/main/).

Each one is unpacked with `ar x libicu_69.1_aarch64.deb; tar xf data.tar.xz` and
the resulting files moved to a newly-created Swift 5.5.1 SDK directory:
Each one is unpacked with `ar x libicu_70.1_aarch64.deb; tar xf data.tar.xz` and
the resulting files moved to a newly-created Swift 5.5.2 SDK directory:
```
mkdir swift-5.5.1-android-aarch64-24-sdk
mv data/data/com.termux/files/usr swift-5.5.1-android-aarch64-24-sdk
mkdir swift-5.5.2-android-aarch64-24-sdk
mv data/data/com.termux/files/usr swift-5.5.2-android-aarch64-24-sdk
```
It removes two config scripts in `usr/bin`, runs `patchelf` to remove the
Termux rpath from all Termux shared libraries, and modifies the ICU libraries
to get rid of the versioning and symlinks (three libicu libraries are removed
since they're unused by Swift):
```
rm swift-5.5.1-android-aarch64-24-sdk/usr/bin/*-config
cd swift-5.5.1-android-aarch64-24-sdk/usr/lib
rm swift-5.5.2-android-aarch64-24-sdk/usr/bin/*-config
cd swift-5.5.2-android-aarch64-24-sdk/usr/lib
rm libicu{io,test,tu}*
patchelf --set-rpath \$ORIGIN libandroid-spawn.so libcurl.so libicu*so.69.1 libxml2.so
patchelf --set-rpath \$ORIGIN libandroid-spawn.so libcurl.so libicu*so.70.1 libxml2.so
# repeat the following for libicui18n.so and libicudata.so, as needed
rm libicuuc.so libicuuc.so.69
readelf -d libicuuc.so.69.1
mv libicuuc.so.69.1 libicuuc.so
rm libicuuc.so libicuuc.so.70
readelf -d libicuuc.so.70.1
mv libicuuc.so.70.1 libicuuc.so
patchelf --set-soname libicuuc.so libicuuc.so
patchelf --replace-needed libicudata.so.69 libicudata.so libicuuc.so
patchelf --replace-needed libicudata.so.70 libicudata.so libicuuc.so
```
The libcurl and libxml2 packages are [only needed for the FoundationNetworking
and FoundationXML libraries respectively](https://github.com/apple/swift-corelibs-foundation/blob/release/5.5/Docs/ReleaseNotes_Swift5.md),
Expand All @@ -211,7 +213,7 @@ instead, so this Swift SDK for Android could be built without using
any prebuilt Termux packages, if you're willing to put in the effort to
cross-compile them yourself, for example, against a different Android API.

Finally, it gets [the 5.5.1 source](https://github.com/apple/swift/releases/tag/swift-5.5.1-RELEASE)
Finally, it gets [the 5.5.2 source](https://github.com/apple/swift/releases/tag/swift-5.5.2-RELEASE)
tarballs for five Swift repos and renames them to `llvm-project/`, `swift/`,
`swift-corelibs-libdispatch`, `swift-corelibs-foundation`, and
`swift-corelibs-xctest`, as required by the Swift `build-script`, and creates
Expand Down
4 changes: 2 additions & 2 deletions android-aarch64.json
View file
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
{
"version": 1,
"target": "aarch64-unknown-linux-android",
"toolchain-bin-dir": "/home/butta/swift-5.5.1-RELEASE-ubuntu20.04/usr/bin",
"toolchain-bin-dir": "/home/butta/swift-5.5.2-RELEASE-ubuntu20.04/usr/bin",
"sdk": "/home/butta/android-ndk-r23b/toolchains/llvm/prebuilt/linux-x86_64/sysroot",
"extra-cc-flags": [
"-fPIC"
],
"extra-swiftc-flags": [
"-resource-dir", "/home/butta/swift-5.5-android-aarch64-24-sdk/usr/lib/swift",
"-resource-dir", "/home/butta/swift-5.5.2-android-aarch64-24-sdk/usr/lib/swift",
"-tools-directory", "/home/butta/android-ndk-r23b/toolchains/llvm/prebuilt/linux-x86_64/bin",
"-Xclang-linker", "--target=aarch64-linux-android24", "-use-ld=lld",
],
Expand Down
4 changes: 2 additions & 2 deletions android-armv7.json
View file
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
{
"version": 1,
"target": "armv7-unknown-linux-androideabi",
"toolchain-bin-dir": "/home/butta/swift-5.5.1-RELEASE-ubuntu20.04/usr/bin",
"toolchain-bin-dir": "/home/butta/swift-5.5.2-RELEASE-ubuntu20.04/usr/bin",
"sdk": "/home/butta/android-ndk-r23b/toolchains/llvm/prebuilt/linux-x86_64/sysroot",
"extra-cc-flags": [
"-fPIC"
],
"extra-swiftc-flags": [
"-resource-dir", "/home/butta/swift-5.5-android-armv7-24-sdk/usr/lib/swift",
"-resource-dir", "/home/butta/swift-5.5.2-android-armv7-24-sdk/usr/lib/swift",
"-tools-directory", "/home/butta/android-ndk-r23b/toolchains/llvm/prebuilt/linux-x86_64/bin",
"-Xclang-linker", "--target=armv7-linux-android24", "-use-ld=lld",
],
Expand Down
4 changes: 2 additions & 2 deletions android-x86_64.json
View file
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
{
"version": 1,
"target": "x86_64-unknown-linux-android",
"toolchain-bin-dir": "/home/butta/swift-5.5.1-RELEASE-ubuntu20.04/usr/bin",
"toolchain-bin-dir": "/home/butta/swift-5.5.2-RELEASE-ubuntu20.04/usr/bin",
"sdk": "/home/butta/android-ndk-r23b/toolchains/llvm/prebuilt/linux-x86_64/sysroot",
"extra-cc-flags": [
"-fPIC"
],
"extra-swiftc-flags": [
"-resource-dir", "/home/butta/swift-5.5-android-x86_64-24-sdk/usr/lib/swift",
"-resource-dir", "/home/butta/swift-5.5.2-android-x86_64-24-sdk/usr/lib/swift",
"-tools-directory", "/home/butta/android-ndk-r23b/toolchains/llvm/prebuilt/linux-x86_64/bin",
"-Xclang-linker", "--target=x86_64-linux-android24", "-use-ld=lld",
],
Expand Down

0 comments on commit 6ff952d

Please sign in to comment.