[android] build and run ctest-based tests on x64

[weliveindetail]

The Swift SDK contains components that use CTest, e.g. swift-corelibs-libdispatch

These tests have unit-test character: Test cases are executables that are invoked and timed. Exit code determines success

$ file dispatch_apply
dispatch_apply: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /system/bin/linker64, not stripped

The binary directory contains a file CTestTestfile.cmake that lists all test cases and defines properties:

add_test(dispatch_apply "S:/a/402/bsdtestharness" "S:/a/402/dispatch_apply")
set_tests_properties(dispatch_apply PROPERTIES  DEPENDS "bsdtestharness" ENVIRONMENT "NOLEAKS=1" TIMEOUT "120" WORKING_DIRECTORY "S:/a/402/tests" _BACKTRACE_TRIPLES "S:/SourceCache/swift-corelibs-libdispatch/tests/CMakeLists.txt;105;add_test;S:/SourceCache/swift-corelibs-libdispatch/tests/CMakeLists.txt;174;add_unit_test;S:/SourceCache/swift-corelibs-libdispatch/tests/CMakeLists.txt;0;")

The ctest executable loads this file and drives test execution:

S:\a\102>ctest
Test project S:/a/102
      Start  1: dispatch_apply
 1/22 Test  #1: dispatch_apply ...................   Passed    0.16 sec
...

100% tests passed, 0 tests failed out of 22

Total Test time (real) =  40.61 sec

[weliveindetail]

Building these test for android is straightforward in case of libdispatch. Build-CMakeProject in build.ps1 has all the logic for cross-platform compilation.

[weliveindetail]

Once we have the binaries, we can upload them to an emulator:

S:\a\702>file dispatch_apply
dispatch_apply: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /system/bin/linker64, not stripped

S:\a\702>adb push bsdtestharness /data/local/tmp
bsdtestharness: 1 file pushed, 0 skipped. 6.7 MB/s (26016 bytes in 0.004s)

S:\a\702>adb push dispatch_apply /data/local/tmp
dispatch_apply: 1 file pushed, 0 skipped. 98.5 MB/s (30856 bytes in 0.000s)

S:\a\702>adb push libdispatch.so /data/local/tmp
libdispatch.so: 1 file pushed, 0 skipped. 78.4 MB/s (406320 bytes in 0.005s)

S:\a\702>adb push libBlocksRuntime.so /data/local/tmp
libBlocksRuntime.so: 1 file pushed, 0 skipped. 29.1 MB/s (10768 bytes in 0.000s)

And execute them through a remote shell:

> adb shell
generic_x86_64:/data/local/tmp $ cd /data/local/tmp
generic_x86_64:/data/local/tmp $ chmod +x bsdtestharness
generic_x86_64:/data/local/tmp $ chmod +x dispatch_apply
generic_x86_64:/data/local/tmp $ LD_LIBRARY_PATH=$(pwd) ./bsdtestharness ./dispatch_apply
==================================================
[TEST] Dispatch Apply
[PID] 3821
==================================================

[BEGIN] dispatch_get_global_queue
        Actual: 0x71a985fa97c0
        Expected: 0x71a985fa97c0
[PASS] dispatch_get_global_queue
...

[BEGIN] Process exited
        Actual: 0
        Expected: 0
[PASS] Process exited
[PERF]  wall time: 0.173572
[PERF]  user time: 0.043217
[PERF]  system time: 0.054021
[PERF]  max resident set size: 11436
[PERF]  page faults: 0
[PERF]  swaps: 0
[PERF]  voluntary context switches: 38
[PERF]  involuntary context switches: 8

generic_x86_64:/data/local/tmp $ echo $?
0

This seems like the most pragmatic solution. We just need to replicate the features we need from the ctest driver in a shell script. A timeout seems like a reasonable feature to replicate and a way to hide the output when tests pass.

[weliveindetail]

ctest is a standalone binary and cross-compiling it from source doesn't seem too complicated: https://github.com/Kitware/CMake/blob/master/Source/ctest.cxx That would give us the same kind of aggregation we have on the host machine.

CMake also has an integrated way to offload tests to Android, but it doesn't seem very popular: https://cmake.org/cmake/help/latest/module/AndroidTestUtilities.html