Skip to content

Run the compiler validation suite for Android #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 62 commits into from
Jun 29, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
69800e6
Run the compiler validation suite for Android
marcprux May 26, 2025
e350942
Add --build-compiler option
marcprux May 27, 2025
fcde0f1
Add --cross-compile-build-swift-tools=0 from https://github.com/swift…
marcprux May 27, 2025
53aa633
Build with --build-llvm=0
marcprux May 28, 2025
60c3370
Check out Yams for Swift 6.1.1 build
marcprux May 28, 2025
63f3750
Add --llvm-ninja-targets-for-cross-compile-hosts=help
marcprux May 28, 2025
bfb9e50
Install pre-requisites
marcprux May 29, 2025
c527b1d
Install build prerequisites
marcprux Jun 2, 2025
ef3f51e
Only setup local toolchain if build-compiler is not 0
marcprux Jun 2, 2025
84e885a
Fix Yams version checkout
marcprux Jun 2, 2025
cdbd187
Quote arguments to build scripts
marcprux Jun 2, 2025
060c871
Permit empty host-toolchain argument in build.sh
marcprux Jun 2, 2025
163220d
Fix check for BUILD_COMPILER
marcprux Jun 2, 2025
f8db216
Re-order Docker PATH to system clang is used before Swift toolchain c…
marcprux Jun 3, 2025
a9942a2
Use --host-test to skip attempt to test on connected device/emulator
marcprux Jun 3, 2025
1cbf01e
Install clang in Dockerfile
marcprux Jun 4, 2025
383fb46
Add --skip-test-linux flag to build
marcprux Jun 5, 2025
af85033
Remove more folders to free up space
marcprux Jun 6, 2025
d78eb9e
Update patches
marcprux Jun 6, 2025
f7d1437
Add docker-specific CI variants that run the compiler validation tests
marcprux Jun 7, 2025
4e6f935
meaningless edit
finagolfin Jun 7, 2025
1692659
Update pull_request.yml to stop the 6.1 release builds
finagolfin Jun 7, 2025
287e81d
Update pull_request.yml to really only build the full docker compiler…
finagolfin Jun 7, 2025
e438467
Update README.md with meaningless edit to bump build
finagolfin Jun 8, 2025
324a3b7
Update build.sh to skip testing XCTest for linux, as a handful of the…
finagolfin Jun 9, 2025
02485be
Update build.sh to disable building libTesting and for 16K memory pag…
finagolfin Jun 9, 2025
d070103
Update build.sh to skip testing Foundation for linux, which requires …
finagolfin Jun 9, 2025
ee9529b
Update build.sh to not build each arch in a separate build directory
finagolfin Jun 10, 2025
4b99b11
Update Dockerfile to use clang 19 instead
finagolfin Jun 10, 2025
896ba60
Add self-hosted runner CI variant
marcprux Jun 13, 2025
9b9d56b
Fix CI workflow syntax
marcprux Jun 13, 2025
a157ab3
Update CI runner config
marcprux Jun 13, 2025
f8c0ffd
Update CI runner config
marcprux Jun 13, 2025
73ccf2e
Fix name of self-hosted CI runner
marcprux Jun 13, 2025
4af2892
Re-order arch run sequence
marcprux Jun 14, 2025
60e2543
Update CI for self-hosted runner
marcprux Jun 14, 2025
87b297a
Build for self-hosted runner
marcprux Jun 14, 2025
67a5291
Add swift-6.2-branch to self-hosted run matrix
marcprux Jun 15, 2025
6c3a1ba
Update build.sh to disable aarch64 temporarily, as we know it built fine
finagolfin Jun 15, 2025
b30c138
Update pull_request.yml to disable non-compiler builds, as they all w…
finagolfin Jun 15, 2025
3a0189b
Run compiler validation on self-hosted with increased timeout
marcprux Jun 15, 2025
929ff18
Update build-docker to only build for armv7
finagolfin Jun 16, 2025
28fbe55
Update pull_request.yml to disable self-hosted builds temporarily
finagolfin Jun 16, 2025
eaa140e
Update pull_request.yml to build the full compiler in github runners …
finagolfin Jun 16, 2025
f0b6057
Update build-docker to build for aarch64 first
finagolfin Jun 23, 2025
1161e56
Update toolchain-vars.sh to use older trunk snapahot toolchain that d…
finagolfin Jun 24, 2025
52108a8
Update toolchain-vars.sh to only download the latest release compiler…
finagolfin Jun 25, 2025
718b96c
Update build-docker to only build for armv7 again
finagolfin Jun 25, 2025
8eb389a
Update toolchain-vars.sh to fix setting branch variable
finagolfin Jun 25, 2025
86dff6b
Update build-docker to build for all three supported arches
finagolfin Jun 25, 2025
16c19f5
Update pull_request.yml to try self-hosted runs again
finagolfin Jun 25, 2025
ae32e31
Update build.sh to only install SDK components and remove linux stdlib
finagolfin Jun 27, 2025
0346d7f
Tolerate missing linux folder when attempting to clean up unnecessary…
marcprux Jun 29, 2025
c6e85b8
Fix extraCLIOptions in swift-toolset.json
marcprux Jun 29, 2025
c01a313
gcpdw
marcprux Jun 29, 2025
d0d85bb
Fix patch application
marcprux Jun 29, 2025
341bf54
Update apply-patches.sh to remove unused changes
finagolfin Jun 29, 2025
b77748a
Update swift-android-testing-except-release.patch to add Testing fix
finagolfin Jun 29, 2025
1bbe96b
Update swift-android.patch to remove unneeded patches
finagolfin Jun 29, 2025
3006f75
Update build.sh to try and fix Testing and clean up the bundle more
finagolfin Jun 29, 2025
d413bdb
marcprux Jun 29, 2025
266ecdb
Fix README.md
marcprux Jun 29, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 35 additions & 14 deletions .github/workflows/pull_request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,27 +23,38 @@ jobs:
*.log

android-build:
name: Build Android ${{ matrix.build-type }} ${{ matrix.swift-version }} ${{ matrix.arch }} SDK
name: Android ${{ matrix.build-type }} ${{ matrix.swift-version }} ${{ matrix.arch }} ${{ matrix.runner }} (compiler=${{ matrix.build-compiler }})
strategy:
fail-fast: false
matrix:
#build-type: ['docker']
build-type: ['docker', 'local']
# blank arch builds all (aarch64,x86_64,armv7)
#arch: ['']
# builds only x86_64 to speed up the validation
#arch: ['x86_64']
# build both the quick (x86_64) and complete (aarch64,x86_64,armv7) SDKs
arch: ['x86_64', '']
swift-version: ['release', 'swift-6.2-branch', 'development']
runs-on: ubuntu-24.04
include:
- swift-version: 'swift-6.2-branch'
build-type: 'docker'
build-compiler: '1'
runner: 'self-hosted'
- swift-version: 'development'
build-type: 'docker'
build-compiler: '1'
runner: 'self-hosted'
- swift-version: 'swift-6.2-branch'
build-type: 'docker'
build-compiler: '0'
runner: 'ubuntu-24.04'
- swift-version: 'development'
build-type: 'docker'
build-compiler: '0'
runner: 'ubuntu-24.04'
runs-on: ${{ matrix.runner }}
# 15 hour timeout
timeout-minutes: 900
steps:
- name: Free Disk Space
if: ${{ matrix.runner != 'self-hosted' }}
run: |
df -h
# brings available space from 25G to 32G
# otherwise we sometimes run out of space during the build
sudo rm -rf /usr/share/miniconda /usr/share/az* /usr/share/glade* /usr/local/share/chromium /usr/local/share/powershell /usr/share/dotnet /opt/ghc /opt/hostedtoolcache/CodeQL
sudo rm -rf /usr/share/miniconda /usr/share/az* /usr/share/glade* /usr/local/share/chromium /usr/local/share/powershell /usr/share/dotnet /opt/ghc /opt/hostedtoolcache /usr/local/graalvm/ /usr/local/.ghcup/ /usr/local/lib/node_modules /usr/local/share/boost
sudo docker image prune --all --force
sudo docker builder prune -a
df -h
Expand All @@ -53,6 +64,8 @@ jobs:
# these variabes are used by build-docker and build-local
# to determine which Swift version to build for
echo "BUILD_SCHEME=${{ matrix.swift-version }}" >> $GITHUB_ENV
# pass the build-compiler matrix through to the build script
echo "BUILD_COMPILER=${{ matrix.build-compiler }}" >> $GITHUB_ENV
echo "TARGET_ARCHS=${{ matrix.arch }}" >> $GITHUB_ENV
echo "WORKDIR=${{ runner.temp }}/swift-android-sdk" >> $GITHUB_ENV
- name: Checkout repository
Expand All @@ -61,7 +74,7 @@ jobs:
if: ${{ matrix.build-type == 'local' }}
working-directory: swift-ci/sdks/android
run: |
sudo apt install -q ninja-build patchelf
sudo apt install -q patchelf build-essential cmake ninja-build python3 golang git gnupg2 libcurl4-openssl-dev libedit-dev libicu-dev libncurses5-dev libpython3-dev libsqlite3-dev libxml2-dev rsync uuid-dev uuid-runtime tzdata curl unzip
./build-local ${BUILD_SCHEME} ${WORKDIR}
- name: Build Android SDK (Docker)
if: ${{ matrix.build-type == 'docker' }}
Expand Down Expand Up @@ -99,6 +112,9 @@ jobs:
if [[ "${{ matrix.build-type }}" == 'local' ]]; then
ARTIFACT_NAME="${ARTIFACT_NAME}-local"
fi
if [[ "${{ matrix.build-compiler }}" == '1' ]]; then
ARTIFACT_NAME="${ARTIFACT_NAME}-hostbuild"
fi
# artifacts need a unique name so we suffix with the matrix arch(s)
if [[ ! -z "${{ matrix.arch }}" ]]; then
ARTIFACT_NAME="${ARTIFACT_NAME}-$(echo ${{ matrix.arch }} | tr ',' '-')"
Expand All @@ -117,6 +133,7 @@ jobs:
name: ${{ steps.info.outputs.artifact-name }}
path: ${{ steps.info.outputs.artifact-path }}
- name: Cleanup
if: ${{ matrix.runner != 'self-hosted' }}
run: |
# need to free up some space or else when installing we get: No space left on device
df -h
Expand All @@ -125,6 +142,7 @@ jobs:
sudo docker builder prune -a
df -h
- name: Install artifactbundle
if: ${{ matrix.runner != 'self-hosted' }}
shell: bash
run: |
set -ex
Expand All @@ -135,6 +153,7 @@ jobs:
echo "ANDROID_NDK_ROOT=" >> $GITHUB_ENV

- name: Create Demo Project
if: ${{ matrix.runner != 'self-hosted' }}
run: |
cd ${{ runner.temp }}
mkdir DemoProject
Expand All @@ -149,6 +168,7 @@ jobs:
echo 'import Android' >> Sources/DemoProject/DemoProject.swift
- name: Test Demo Project on Android
uses: skiptools/swift-android-action@main
if: ${{ matrix.runner != 'self-hosted' }}
with:
# only test for the complete arch SDK build to speed up CI
#run-tests: ${{ matrix.arch == '' }}
Expand All @@ -157,14 +177,15 @@ jobs:
installed-swift: ${{ steps.info.outputs.swift-root }}

- name: Checkout swift-algorithms
if: ${{ matrix.runner != 'self-hosted' }}
uses: actions/checkout@v4
with:
repository: apple/swift-algorithms
path: swift-algorithms
- name: Test swift-algorithms
if: ${{ matrix.runner != 'self-hosted' }}
uses: skiptools/swift-android-action@main
with:
run-tests: ${{ matrix.arch == '' }}
package-path: swift-algorithms
installed-sdk: ${{ steps.info.outputs.sdk-id }}
installed-swift: ${{ steps.info.outputs.swift-root }}
Expand Down
3 changes: 2 additions & 1 deletion swift-ci/sdks/android/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ ARG SWIFT_TOOLCHAIN_URL=
RUN apt-get -q update \
&& DEBIAN_FRONTEND=noninteractive apt-get -q install -y \
build-essential \
clang-19 \
cmake \
ninja-build \
python3 \
Expand Down Expand Up @@ -54,7 +55,7 @@ ENV SWIFT_SIGNING_KEY=$SWIFT_SIGNING_KEY \
COPY scripts/install-swift.sh /scripts/install-swift.sh
RUN chmod ugo+x /scripts/install-swift.sh
RUN /scripts/install-swift.sh /usr/local/swift
ENV PATH="/usr/local/swift/bin:${PATH}"
ENV PATH="/usr/lib/llvm-19/bin:/usr/local/swift/bin:${PATH}"

ARG ANDROID_NDK_VERSION=

Expand Down
15 changes: 8 additions & 7 deletions swift-ci/sdks/android/build-docker
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ source ./scripts/toolchain-vars.sh

# Check-out and patch the sources
./scripts/fetch-source.sh --source-dir ${WORKDIR}/source --swift-tag ${SWIFT_TAG}
${WORKDIR}/source/swift-android-patches/apply-patches.sh ${WORKDIR}/source/swift-project
./patches/apply-patches.sh ${WORKDIR}/source/swift-project

mkdir -p ${WORKDIR}/products
chmod ugo+rwx ${WORKDIR}/products
Expand Down Expand Up @@ -59,9 +59,10 @@ $DOCKER run -i --rm \
-v ${WORKDIR}/products:/products:rw \
${CONTAINER_NAME} \
/scripts/build.sh \
--source-dir /source \
--products-dir /products \
--host-toolchain /usr/local/swift \
--android-api ${ANDROID_API} \
--ndk-home /usr/local/ndk/${ANDROID_NDK_VERSION} \
--archs ${TARGET_ARCHS}
--source-dir "/source" \
--products-dir "/products" \
--host-toolchain "/usr/local/swift" \
--build-compiler "${BUILD_COMPILER}" \
--android-api "${ANDROID_API}" \
--ndk-home "/usr/local/ndk/${ANDROID_NDK_VERSION}" \
--archs "${TARGET_ARCHS}"
35 changes: 19 additions & 16 deletions swift-ci/sdks/android/build-local
Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,18 @@ WORKDIR=$(realpath ${WORKDIR})
HOST_OS=ubuntu$(lsb_release -sr)
source ./scripts/toolchain-vars.sh

SWIFT_ROOT=${WORKDIR}/host-toolchain
HOST_TOOLCHAIN=$SWIFT_ROOT/$SWIFT_BASE/usr
if [[ ! -d "$HOST_TOOLCHAIN" ]]; then
./scripts/install-swift.sh ${HOST_TOOLCHAIN}
fi
if [[ "${BUILD_COMPILER}" != "1" ]]; then
SWIFT_ROOT=${WORKDIR}/host-toolchain
HOST_TOOLCHAIN=$SWIFT_ROOT/$SWIFT_BASE/usr
if [[ ! -d "$HOST_TOOLCHAIN" ]]; then
./scripts/install-swift.sh ${HOST_TOOLCHAIN}
fi

$HOST_TOOLCHAIN/bin/swift --version
$HOST_TOOLCHAIN/bin/swift --version

# ensure the correct Swift is first in the PATH
export PATH=$HOST_TOOLCHAIN/bin:$PATH
# ensure the correct Swift is first in the PATH
export PATH=$HOST_TOOLCHAIN/bin:$PATH
fi

export ANDROID_NDK_HOME=${WORKDIR}/ndk/${ANDROID_NDK_VERSION}

Expand All @@ -50,15 +52,16 @@ fi

# Check-out and patch the sources
./scripts/fetch-source.sh --source-dir ${WORKDIR}/source --swift-tag ${SWIFT_TAG}
${WORKDIR}/source/swift-android-patches/apply-patches.sh ${WORKDIR}/source/swift-project
./patches/apply-patches.sh ${WORKDIR}/source/swift-project

mkdir -p ${WORKDIR}/products

./scripts/build.sh \
--source-dir ${WORKDIR}/source \
--products-dir ${WORKDIR}/products \
--build-dir ${WORKDIR}/build \
--host-toolchain ${HOST_TOOLCHAIN} \
--android-api ${ANDROID_API} \
--ndk-home ${ANDROID_NDK_HOME} \
--archs ${TARGET_ARCHS}
--source-dir "${WORKDIR}/source" \
--products-dir "${WORKDIR}/products" \
--build-dir "${WORKDIR}/build" \
--build-compiler "${BUILD_COMPILER}" \
--host-toolchain "${HOST_TOOLCHAIN}" \
--android-api "${ANDROID_API}" \
--ndk-home "${ANDROID_NDK_HOME}" \
--archs "${TARGET_ARCHS}"
23 changes: 23 additions & 0 deletions swift-ci/sdks/android/patches/apply-patches.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/bin/bash -ex

patches_dir=$(dirname $(realpath -- "${BASH_SOURCE[0]}"))
cd ${1}

case "${BUILD_SCHEME}" in
swift-*-branch)
git apply -v -C1 ${patches_dir}/swift-android.patch
git apply -v -C1 ${patches_dir}/swift-android-testing-except-release.patch
;;
development)
git apply -v -C1 ${patches_dir}/swift-android.patch
git apply -v -C1 ${patches_dir}/swift-android-trunk-libdispatch.patch
git apply -v -C1 ${patches_dir}/swift-android-testing-except-release.patch
;;
*)
echo "$0: invalid BUILD_SCHEME=${BUILD_SCHEME}"
exit 1
;;
esac

# disable backtrace() for Android (needs either API33+ or libandroid-execinfo, or to manually add in backtrace backport)
perl -pi -e 's;os\(Android\);os\(AndroidDISABLED\);g' swift-testing/Sources/Testing/SourceAttribution/Backtrace.swift
Loading
Loading