ContainerizedAndroidBuilder is a tool that encapsulates the entire Android ROM building process within a Docker container. To simplify your life even further, it features an intuitive terminal user interface (TUI) that organizes common actions into menus you will need when building any Android-based ROM. It's particularly suitable if you want to manage your trees with ease on your not-very-powerful machine (e.g., smartphone), but perform the actual build on another more capable machine (e.g., remote server).
This tool is designed to be shipped as a Git submodule, which allows to reuse the Android sources and other common directories when building for multiple devices/flavors.
Easy to use
This tool is perfect for lazy or inexperienced users who simply want to build a ROM without too much hassle. All required dependencies are already bundled in the Docker image.
Clean system
Since all files/directories are in one place and dependencies are "containerized", your current system won't get dirty. Whenever you want, use the Self-destroy menu option to remove Docker image entirely and free up allocated disk space.
N.B.: source trees, compiled Android and cache files will remain unaffected, and will stay intact on your disk. You have to remove them manually.
Platform-agnostic
Containerization allows to reduce to minimal machine-dependent failures and dependency conflicts when building, which makes this tool work seamlessly in any environment.
Optimized disk usage
You can easily move around the Android sources and compiled output files, even across different disks using --src-dir and --out-dir arguments.
-
Create a directory for your future Android build:
mkdir AndroidBuilder && cd $_ -
Init a Git project in it:
git init -b device_codenameReplace
device_codenamewith your device codename.Note: the best practice is to always create a branch per device. This will allow to reuse the submodule, source tree and other common directories.
-
Add this repo as a submodule:
git submodule add https://github.com/iusmac/ContainerizedAndroidBuilder.git -
Create a wrapper script (e.g.,
builder.sh) for the submodule:touch builder.sh && chmod +x $_Place submodule execution in it:
#!/usr/bin/env bash ./ContainerizedAndroidBuilder/run.sh \ --android <Android version> \ --repo-url <Repo URL> \ --repo-revision <Repo revision> \ --lunch-system <Lunch system> \ --lunch-device <Lunch device> \ --lunch-flavor <Lunch flavor>
Note: replace the placeholders with the appropriate values, example:
#!/usr/bin/env bash ./ContainerizedAndroidBuilder/run.sh \ --android 15.0 \ --repo-url https://android.googlesource.com/platform/manifest \ --repo-revision main \ --lunch-system aosp \ --lunch-device redfin \ --lunch-flavor userdebug -
(Optional) Create local_manifests/ directory for additional .xml files:
mkdir local_manifestsDiscover more about Local Manifests.
-
Make sure your final project has the following directory structure:
AndroidBuilder/ ├── .git/ ├── .gitmodules ├── builder.sh ├── ContainerizedAndroidBuilder/ └── local_manifests/ -
(Recommended) Commit the changes and push to your repository:
git add -A git commit -m "Init" git push
This way you and others can obtain the pre-configured project with the submodule just by cloning it:
git clone --recursive https://github.com/.../AndroidBuilder.git(!) Pay attention to the --recursive option. It is necessary to 'hook' the ContainerizedAndroidBuilder module from
.gitmodulesfile during cloning. -
Get Docker on your system
curl -fsSL https://get.docker.com | sudo sh -Note: strictly follow the installation guide. It's fully automatic for Unix-like systems, but requires some manual intervention on Windows or MacOS systems.
Ensure Docker is installed:
docker --version Docker version 20.10.17, build 100c701
-
Well done! Run the
builder.shand start building now! :)
$ ./run.sh --help
Usage: ./run.sh
--android ANDROID
--repo-url REPO_URL
--repo-revision REPO_REVISION
--lunch-system LUNCH_SYSTEM
--lunch-device LUNCH_DEVICE
--lunch-flavor LUNCH_FLAVOR
[--email EMAIL]
[--src-dir SRC_DIR]
[--out-dir OUT_DIR]
[--zips-dir ZIPS_DIR]
[--move-zips MOVE_ZIPS]
[--ccache-dir CCACHE_DIR]
[--ccache-disabled]
[--ccache-size CCACHE_SIZE]
[--timezone TIMEZONE]
[--help] [--version]
Description of required arguments:
| Argument | Description |
|---|---|
--android ANDROID |
Android version to build. Possible values: 10.0, 11.0, 12.0, 12.1, 13.0, 14.0 |
--repo-url URL |
URL for repo init -u {URL} command. Example: https://android.googlesource.com/platform/manifest |
--repo-revision REVISION_NAME |
Branch/revision for repo init -u {URL} -b {REVISION_NAME} command. Example: main |
--lunch-system SYSTEM |
System name for lunch {SYSTEM}_device-flavor command. Example: aosp |
--lunch-device DEVICE |
Device codename for lunch system_{DEVICE}-flavor command. Example: redfin |
--lunch-flavor FLAVOR |
Build type/flavor for lunch system_device-{FLAVOR} command. Possibile values: user, userdebug, or eng. |
Description of optional arguments:
| Argument | Description |
|---|---|
--email EMAIL |
E-mail for git config user.email {EMAIL} command. Defaults to docker@localhost. |
--src-dir SRC_DIR |
Absolute path to source tree. Defaults to src/ in current working directory. |
--out-dir OUT_DIR |
Absolute path to output directory. Defaults to out/ in current working directory. |
--zips-dir ZIPS_DIR |
Absolute path to zips directory. Defaults to zips/ in current working directory. |
--move-zips |
Whether the .zip and the boot.img files need to be moved from {OUT_DIR}/target/product/{LUNCH_DEVICE} directory to zips/ directory. |
--ccache-dir CCACHE_DIR |
Absolute path to compiler cache directory. Defaults to ccache/ in current working directory. |
--ccache-disabled |
Whether the compiler cache should be disabled. |
--ccache-size CCACHE_SIZE |
Maximum compiler cache size. Defaults to 30GB. |
--timezone TIMEZONE |
Timezone used inside Docker container. Defaults to the $TZ environment variable.The value shall respect the timezone syntax, example: America/New_York. See list of tz database time zonesNote: in case your shell doesn't declare $TZ variable, the script will try to detectTZ using timedatectl command. If it fails, the script will attempt to fetchtimezone from URL http://ip-api.com/line?fields=timezone. |
This is the final directory structure you should expect after running the TUI app at least ones.
| Directory | Description |
|---|---|
cache/ |
Contains the entire /home/android directory with caches & shell config files extracted from the container and other temporary files. |
ccache/ |
Contains ccache.conf config file and compiler cache files. |
ContainerizedAndroidBuilder/ |
Contains the builder itself shipped as submodule. |
local_manifests/ |
Contains additional remotes and projects copied to {SRC_DIR}/.repo/local_manifests. |
logs/ |
Contains some useful build infos. |
misc/ |
Contains miscellaneous files (e.g., configs) needed for customizations. |
src/ |
Contains Android source code. |
out/ |
Contains compiled Android files. |
zips/ |
Contains .zip and boot.img files if --move-zips option is passed. |
| Key | Description |
|---|---|
Arrows (←↑↓→) |
Move between options. |
Enter (↵) |
Confirm action. |
Tab (↹) |
Change focus. |
Space (␣) |
Select option(s) from the checkbox & radio lists. |
Escape (Esc) |
Return or exit. |
If you'd like to build as fast as possible, try to apply as many of these tips as you can.
Avoid
java.lang.OutOfMemoryError: Java heap spaceerrorAlthough Google suggests you to have at least 16GB of RAM in order to build Android, you may still occasionally observe the build being killed by the Out of Memory (OOM) killer.To be able to build on 16GB (even 8GB) RAM machine without being killed by the Out of Memory (OOM) killer, apply these patches in [SRC_DIR]/build/soong directory:
For Android 15:
diff --git a/java/droidstubs.go b/java/droidstubs.go index b32b754..3a37aa1 100644 --- a/java/droidstubs.go +++ b/java/droidstubs.go @@ -784,6 +784,7 @@ func metalavaCmd(ctx android.ModuleContext, rule *android.RuleBuilder, srcs andr cmd.BuiltTool("metalava").ImplicitTool(ctx.Config().HostJavaToolPath(ctx, "metalava.jar")). Flag(config.JavacVmFlags). Flag(config.MetalavaAddOpens). + Flag("-J-Xmx6114m"). FlagWithArg("--java-source ", params.javaVersion.String()). FlagWithRspFileInputList("@", android.PathForModuleOut(ctx, fmt.Sprintf("%s.metalava.rsp", params.stubsType.String())), srcs). FlagWithInput("@", srcJarList)For Android 14:
diff --git a/java/droidstubs.go b/java/droidstubs.go index 9556e956ac..b46eb58798 100644 --- a/java/droidstubs.go +++ b/java/droidstubs.go @@ -722,6 +722,7 @@ func metalavaCmd(ctx android.ModuleContext, rule *android.RuleBuilder, javaVersi cmd.BuiltTool("metalava").ImplicitTool(ctx.Config().HostJavaToolPath(ctx, "metalava.jar")). Flag(config.JavacVmFlags). Flag(config.MetalavaAddOpens). + Flag("-J-Xmx6114m"). FlagWithArg("--java-source ", javaVersion.String()). FlagWithRspFileInputList("@", android.PathForModuleOut(ctx, "metalava.rsp"), srcs). FlagWithInput("@", srcJarList)For Android 12.x/13:
diff --git a/java/droidstubs.go b/java/droidstubs.go index c756815c62..37cb4c8d9b 100644 --- a/java/droidstubs.go +++ b/java/droidstubs.go @@ -415,6 +415,7 @@ func metalavaCmd(ctx android.ModuleContext, rule *android.RuleBuilder, javaVersi cmd.BuiltTool("metalava").ImplicitTool(ctx.Config().HostJavaToolPath(ctx, "metalava.jar")). Flag(config.JavacVmFlags). + Flag("-J-Xmx6114m"). Flag("-J--add-opens=java.base/java.util=ALL-UNNAMED"). FlagWithArg("-encoding ", "UTF-8"). FlagWithArg("-source ", javaVersion.String()).For Android 11:
diff --git a/java/droiddoc.go b/java/droiddoc.go index b564fea014..ddc0682a1c 100644 --- a/java/droiddoc.go +++ b/java/droiddoc.go @@ -1474,6 +1474,7 @@ func metalavaCmd(ctx android.ModuleContext, rule *android.RuleBuilder, javaVersi cmd.BuiltTool(ctx, "metalava"). Flag(config.JavacVmFlags). + Flag("-J-Xmx6114m"). FlagWithArg("-encoding ", "UTF-8"). FlagWithArg("-source ", javaVersion.String()). FlagWithRspFileInputList("@", srcs).For Android 10:
diff --git a/java/droiddoc.go b/java/droiddoc.go index 30a968d6..362acf91 100644 --- a/java/droiddoc.go +++ b/java/droiddoc.go @@ -73,7 +73,7 @@ var ( Command: `rm -rf "$outDir" "$srcJarDir" "$stubsDir" && ` + `mkdir -p "$outDir" "$srcJarDir" "$stubsDir" && ` + `${config.ZipSyncCmd} -d $srcJarDir -l $srcJarDir/list -f "*.java" $srcJars && ` + - `${config.JavaCmd} -jar ${config.MetalavaJar} -encoding UTF-8 -source $javaVersion @$out.rsp @$srcJarDir/list ` + + `${config.JavaCmd} -Xmx6114m -jar ${config.MetalavaJar} -encoding UTF-8 -source $javaVersion @$out.rsp @$srcJarDir/list ` + `$bootclasspathArgs $classpathArgs $sourcepathArgs --no-banner --color --quiet --format=v2 ` + `$opts && ` + `${config.SoongZipCmd} -write_if_changed -jar -o $out -C $stubsDir -D $stubsDir && ` + diff --git a/java/config/config.go b/java/config/config.go index 7263205b..327db935 100644 --- a/java/config/config.go +++ b/java/config/config.go @@ -51,7 +51,7 @@ var ( func init() { pctx.Import("github.com/google/blueprint/bootstrap") - pctx.StaticVariable("JavacHeapSize", "2048M") + pctx.StaticVariable("JavacHeapSize", "1024M") pctx.StaticVariable("JavacHeapFlags", "-J-Xmx${JavacHeapSize}") pctx.StaticVariable("DexFlags", "-JXX:OnError='cat hs_err_pid%p.log' -JXX:CICompilerCount=6 -JXX:+UseDynamicNumberOfGCThreads") diff --git a/java/config/makevars.go b/java/config/makevars.go index 6881caff..fb660748 100644 --- a/java/config/makevars.go +++ b/java/config/makevars.go @@ -47,7 +47,7 @@ func makeVarsProvider(ctx android.MakeVarsContext) { ctx.Strict("COMMON_JDK_FLAGS", "${CommonJdkFlags}") ctx.Strict("DX", "${D8Cmd}") - ctx.Strict("DX_COMMAND", "${D8Cmd} -JXms16M -JXmx2048M") + ctx.Strict("DX_COMMAND", "${D8Cmd} -JXms16M -JXmx1024M") ctx.Strict("R8_COMPAT_PROGUARD", "${R8Cmd}") ctx.Strict("TURBINE", "${TurbineJar}")Compress RAM with zRAM
zRAM, formerly called compcache, is a Linux kernel module for creating a compressed block device in RAM, i.e. a RAM disk with on-the-fly disk compression. With zRAM you can store in RAM ~50% more of data.
Install zRAM
- Debian:
sudo apt install zram-config
- Fedora:
sudo dnf install zram-generator-defaults
- Arch:
sudo pacman -S zram-generatorNote: don't forget to reboot afterwards. Also, check that zRAM is enabled using
swapon -scommand.Create a swap file/partition on disk
If you're building on a 8GB RAM machine, it may require additional 20-30GB of space on the disk when building Soong components. Prefer SSDs over HDDs if possible. Without a fast random I/O access, it could take days to finish the build from scratch.Note: make sure your swap priority is smaller than zRAM priority, if the latter is used:
$ swapon -s Filename Type Size Used Priority /dev/sdb2 partition 8387580 0 -2 /dev/zram0 partition 507264 256164 5 [...]Adjust swappiness
Setting the right value for swappiness allows to get the most of zRAM and swap on disk.
- For swap on disk:
We need to tell the kernel to avoid swapping on disk as much as possible and fill RAM as we all know disk I/O is a lot slower than RAM.
- For zRAM:
We need to tell the kernel to compress RAM as soon as possible, and thus avoid bottleneck due to long compressing/decompressing moving loop when running out of RAM.
In the command below, replace the
Nwith the appropriate value for your case:
N = 1if you only have swap on disk
N = 100if you have zRAM and less than 16GB of RAM
N = 200if you have zRAM and at least 16GB of RAM# This will apply swappiness now & make it persistent across reboots sudo sysctl -w vm.swappiness=N | sudo tee -a /etc/sysctl.confNote: these swappiness values are suitable only for high load, as we know 100 percent that the RAM will be filled up quickly. So, if you plan to use your PC for routine work, you need to revert manually the value to
N = 60, which is the default.Use a lightweight Linux distro
For example, Ubuntu Server (TTY only) consumes about 100-200 MB of RAM after boot up.
Make a load balancing between disks
Use RAID0 system if more than one disk can be used. Or simply split and store the Android sources and output directories on separate disks using
--src-dirand--out-dirarguments.Switch to BTRFS file system
Switching to the BTRFS file system, by itself, makes no difference, but enabling compression allows to reduce the size of the Android sources and output directories by more than 50%. That is, it's good in itself (especially when using low-capacity SSDs), but if you also have a fast CPU, that will speed up the build as you have to read/write less to/from disk.
misc/.bash_profile file
Use this script to customize the shell environment or run some custom scripts. The file will be sourced by .bashrc whenever you jump inside the container or before you start any action from the main menu, such as ROM building or source syncing.
DOCKER_BUILD_IMAGE
Set the DOCKER_BUILD_IMAGE=1 environment variable to force re-build Docker image locally using the default config file that you can find in Dockerfile/ directory.
DOCKER_BUILD_PATH
Set the DOCKER_BUILD_PATH="/path/to/project" environment variable to build using a different build context. This will read the Dockerfile from the same directory.
Requires the DOCKER_BUILD_IMAGE=1 environment variable to be set.
NOTE: in order to reduce building time and image size, it's recommended to place the Dockerfile file inside an empty directory to prevent the Docker CLI from sending the entire context to the docker daemon.
Contributions are welcome! Any ideas and suggestions are appreciated. For now, only recent Android versions are tested and supported. Feel free to try the builder on older Android versions and submit adaptations via Pull Request.