Refactor image builds to put all of the hardware and platform selection logic into one place. #720
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,6 @@ | ||
| FROM quay.io/fedora/fedora:41 | ||
|
|
||
| COPY build_llama_and_whisper.sh / | ||
|
|
||
| RUN chmod +x /build_llama_and_whisper.sh ; \ | ||
| /build_llama_and_whisper.sh asahi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| # Bash does not easily pass arrays as a single arg to a function. So, make this a global var in the script. | ||
| CMAKE_FLAGS="" | ||
| LLAMA_CPP_CMAKE_FLAGS="" | ||
|
There was a problem hiding this comment. If we need a global variables, could we put local variables in main instead? Trying to keep everything scoped. There was a problem hiding this comment. The styling changes aren't a blocker for merge, but why change it? We don't put "function" in front of the functions in any of the other shell scripts in the project, we don't use camelCase in any of the other shell scripts either. Now this is the odd shell script in terms of consistency 😄 There was a problem hiding this comment. Yeah, I don't want to be the oddball... |
||
| WHISPER_CPP_CMAKE_FLAGS="" | ||
|
|
||
| function cmakeCheckWarnings() { | ||
| awk -v rc=0 '/CMake Warning:/ { rc=1 } 1; END {exit rc}' | ||
| } | ||
|
|
||
| function cloneAndBuild() { | ||
| local git_repo=${1} | ||
| local git_sha=${2} | ||
| local install_prefix=${3} | ||
| local work_dir=$(mktemp -d) | ||
|
|
||
| git clone ${git_repo} ${work_dir} | ||
| cd ${work_dir} | ||
| git submodule update --init --recursive | ||
| git reset --hard ${git_sha} | ||
| cmake -B build ${CMAKE_FLAGS[@]} 2>&1 | cmakeCheckWarnings | ||
| cmake --build build --config Release -j$(nproc) -v 2>&1 | cmakeCheckWarnings | ||
| cmake --install build --prefix ${install_prefix} 2>&1 | cmakeCheckWarnings | ||
| cd - | ||
| rm -rf ${work_dir} | ||
| } | ||
|
|
||
| function dnfPrepUbi() { | ||
|
|
||
| local url="https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm" | ||
| local uname_m=$(uname -m) | ||
|
|
||
| dnf install -y ${url} | ||
| crb enable # this is in epel-release, can only install epel-release via url | ||
| dnf copr enable -y slp/mesa-krunkit epel-9-${uname_m} | ||
| url="https://mirror.stream.centos.org/9-stream/AppStream/${uname_m}/os/" | ||
| dnf config-manager --add-repo ${url} | ||
| url="http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-Official" | ||
| curl --retry 8 --retry-all-errors -o /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-Official ${url} | ||
| rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-Official | ||
| } | ||
|
|
||
| function main() { | ||
| set -ex | ||
|
|
||
| local container_image=${1} | ||
| local install_prefix=/usr | ||
| local package_list | ||
| local whisper_cpp_sha=${WHISPER_CPP_SHA:-8a9ad7844d6e2a10cddf4b92de4089d7ac2b14a9} | ||
| local llama_cpp_sha=${LLAMA_CPP_SHA:-aa6fb1321333fae8853d0cdc26bcb5d438e650a1} | ||
| local common_rpms=("python3" "python3-pip" "python3-argcomplete" "python3-dnf-plugin-versionlock" "gcc-c++" "cmake" "vim" "procps-ng" "git" "dnf-plugins-core" "libcurl-devel") | ||
| local vulkan_rpms=("vulkan-headers" "vulkan-loader-devel" "vulkan-tools" "spirv-tools" "glslc" "glslang") | ||
| local intel_rpms=("intel-oneapi-mkl-sycl-devel" "intel-oneapi-dnnl-devel" "intel-oneapi-compiler-dpcpp-cpp" "intel-level-zero" "oneapi-level-zero" "oneapi-level-zero-devel" "intel-compute-runtime") | ||
|
|
||
| LLAMA_CPP_CMAKE_FLAGS+=("-DGGML_CCACHE=OFF" "-DGGML_NATIVE=OFF" "-DBUILD_SHARED_LIBS=NO" "-DLLAMA_CURL=ON") | ||
|
There was a problem hiding this comment. This isn't right, we want BUILD_SHARE_LIBS for llama.cpp but not whisper.cpp There was a problem hiding this comment. I think -DBUILD_SHARED_LIBS breaks for the intel-gpu build. I'll double check that. There was a problem hiding this comment. I bet that's why the rocm image got really big... There was a problem hiding this comment. Yeah, I think you got it, -DBUILD_SHARED_LIBS=NO end up in the libraries being duplicated in every single binary produced. The reason we do it for whisper.cpp is it uses some of the same libraries, with the same names, but with different versions sometimes, so one install will replace the library of the other, breaking whichever was installed first. whisper.cpp is a small project so we statically link that one to fix the issue. |
||
| WHISPER_CPP_CMAKE_FLAGS+=("-DGGML_CCACHE=OFF" "-DGGML_NATIVE=OFF" "-DBUILD_SHARED_LIBS=NO") | ||
|
|
||
| case ${container_image} in | ||
| ramalama) | ||
| dnf --enablerepo=ubi-9-appstream-rpms install -y "${common_rpms[@]}" | ||
| dnfPrepUbi | ||
| dnf --enablerepo=ubi-9-appstream-rpms install -y mesa-vulkan-drivers "${vulkan_rpms[@]}" | ||
| LLAMA_CPP_CMAKE_FLAGS+=("-DGGML_KOMPUTE=ON" "-DKOMPUTE_OPT_DISABLE_VULKAN_VERSION_CHECK=ON") | ||
| ;; | ||
| rocm) | ||
| dnf --enablerepo=ubi-9-appstream-rpms install -y "${common_rpms[@]}" | ||
| dnfPrepUbi | ||
| dnf install -y "${vulkan_rpms[@]}" rocm-dev hipblas-devel rocblas-devel | ||
| LLAMA_CPP_CMAKE_FLAGS+=("-DGGML_HIP=ON" "-DAMDGPU_TARGETS=${AMDGPU_TARGETS:-gfx1010,gfx1030,gfx1032,gfx1100,gfx1101,gfx1102}") | ||
| WHISPER_CPP_CMAKE_FLAGS+=("-DGGML_HIP=ON" "-DAMDGPU_TARGETS=${AMDGPU_TARGETS:-gfx1010,gfx1030,gfx1032,gfx1100,gfx1101,gfx1102}") | ||
| ;; | ||
| cuda) | ||
| dnf install -y "${common_rpms[@]}" gcc-toolset-12 | ||
| . /opt/rh/gcc-toolset-12/enable | ||
| LLAMA_CPP_CMAKE_FLAGS+=("-DGGML_CUDA=ON" "-DCMAKE_EXE_LINKER_FLAGS=-Wl,--allow-shlib-undefined") | ||
| WHISPER_CPP_CMAKE_FLAGS+=("-DGGML_CUDA=ON" "-DCMAKE_EXE_LINKER_FLAGS=-Wl,--allow-shlib-undefined") | ||
| install_prefix=/llama-cpp | ||
| ;; | ||
| vulkan) | ||
| dnf --enablerepo=ubi-9-appstream-rpms install -y "${common_rpms[@]}" | ||
| dnfPrepUbi | ||
| dnf --enablerepo=ubi-9-appstream-rpms install -y mesa-vulkan-drivers "${vulkan_rpms[@]}" | ||
| LLAMA_CPP_CMAKE_FLAGS+=("-DGGML_VULKAN=1") | ||
| WHISPER_CPP_CMAKE_FLAGS+=("-DGGML_VULKAN=1") | ||
| ;; | ||
| asahi) | ||
| dnf copr enable -y @asahi/fedora-remix-branding | ||
| dnf install -y asahi-repos | ||
| dnf install -y mesa-vulkan-drivers "${vulkan_rpms[@]}" "${common_rpms[@]}" | ||
| LLAMA_CPP_CMAKE_FLAGS+=("-DGGML_VULKAN=1") | ||
| WHISPER_CPP_CMAKE_FLAGS+=("-DGGML_VULKAN=1") | ||
| ;; | ||
| intel-gpu) | ||
| dnf install -y ${common_rpms[@]} ${intel_rpms[@]} | ||
| LLAMA_CPP_CMAKE_FLAGS+=("-DGGML_SYCL=ON" "-DCMAKE_C_COMPILER=icx" "-DCMAKE_CXX_COMPILER=icpx") | ||
| WHISPER_CPP_CMAKE_FLAGS+=("-DGGML_SYCL=ON" "-DCMAKE_C_COMPILER=icx" "-DCMAKE_CXX_COMPILER=icpx") | ||
| install_prefix=/llama-cpp | ||
| source /opt/intel/oneapi/setvars.sh | ||
| ;; | ||
| esac | ||
|
|
||
| CMAKE_FLAGS+=("${WHISPER_CPP_CMAKE_FLAGS[@]}") | ||
| cloneAndBuild https://github.com/ggerganov/whisper.cpp ${whisper_cpp_sha} ${install_prefix} | ||
| CMAKE_FLAGS+=(${LLAMA_CPP_CMAKE_FLAGS[@]}) | ||
| cloneAndBuild https://github.com/ggerganov/llama.cpp ${llama_cpp_sha} ${install_prefix} | ||
| dnf -y clean all | ||
| rm -rf /var/cache/*dnf* /opt/rocm-*/lib/*/library/*gfx9* | ||
|
There was a problem hiding this comment. This was where the bulk of the trimming was done for rocm in the past. Not sure why it's ballooned again. I'd revert this shell script back to it's original state and introduce just enough changes to get "intel-gpu" working with this script. There's no need to rewrite the whole script, it's asking for breakages. It's not making a reviewers job easy re-writing the whole thing 😄 There was a problem hiding this comment. Understood. It was a wild hair 🤓 There was a problem hiding this comment. My intent is to see if we can prevent the build script from becoming a Rube Goldberg machine. The more logic that we have to drop into places based on different builds may get unmanageable. IMO, it's already a bit challenging to read. Natural entropy of code. There was a problem hiding this comment. Not sure my attempt is much easier to read either though... |
||
| ldconfig # needed for libraries | ||
|
|
||
| } | ||
|
|
||
| main "$@" | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,32 +1,23 @@ | ||
| FROM quay.io/fedora/fedora:41 as builder | ||
|
|
||
| COPY intel-gpu/oneAPI.repo /etc/yum.repos.d/ | ||
| COPY build_llama_and_whisper.sh / | ||
|
|
||
| RUN chmod +x /build_llama_and_whisper.sh ; \ | ||
| /build_llama_and_whisper.sh intel-gpu | ||
|
|
||
| FROM quay.io/fedora/fedora:41 | ||
|
|
||
| COPY --from=builder /llama-cpp/ /usr/ | ||
| COPY intel-gpu/oneAPI.repo /etc/yum.repos.d/ | ||
| COPY --chown=0:0 intel-gpu/entrypoint.sh / | ||
|
|
||
| RUN dnf install -y procps-ng python3 python3-pip python3-devel intel-level-zero oneapi-level-zero intel-compute-runtime libcurl lspci clinfo intel-oneapi-runtime-compilers intel-oneapi-mkl-core intel-oneapi-mkl-sycl-blas intel-oneapi-runtime-dnnl ; \ | ||
| chown 0:0 /etc/passwd ; \ | ||
| chown 0:0 /etc/group ; \ | ||
| chmod g=u /etc/passwd /etc/group /home ; \ | ||
| chmod +x /entrypoint.sh | ||
|
|
||
| USER 10000 | ||
|
|
||
| ENTRYPOINT ["/entrypoint.sh"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| if [ -z ${HOME} ] | ||
| then | ||
| export HOME=/home/llama-user | ||
| fi | ||
|
|
||
| # Create Home directory | ||
| if [ ! -d "${HOME}" ] | ||
| then | ||
| mkdir -p "${HOME}" | ||
| fi | ||
|
|
||
| # Create User ID | ||
| if ! whoami &> /dev/null | ||
| then | ||
| if [ -w /etc/passwd ] && [ -w /etc/group ] | ||
| then | ||
| echo "${USER_NAME:-llama-user}:x:$(id -u):0:${USER_NAME:-llama-user} user:${HOME}:/bin/bash" >> /etc/passwd | ||
| echo "${USER_NAME:-llama-user}:x:$(id -u):" >> /etc/group | ||
| render_group="$(cat /etc/group | grep 'render:x')" | ||
| video_group="$(cat /etc/group | grep 'video:x')" | ||
| render_group_new="${render_group}${USER_NAME:-llama-user}" | ||
| video_group_new="${video_group}${USER_NAME:-llama-user}" | ||
| sed "s|${render_group}|${render_group_new}|g" /etc/group > /tmp/group | ||
| cat /tmp/group > /etc/group | ||
| sed "s|${video_group}|${video_group_new}|g" /etc/group > /tmp/group | ||
| cat /tmp/group > /etc/group | ||
| fi | ||
| fi | ||
|
|
||
| # Configure Z shell | ||
| if [ ! -f ${HOME}/.zshrc ] | ||
| then | ||
| (echo "source /opt/intel/oneapi/setvars.sh") > ${HOME}/.zshrc | ||
| fi | ||
|
|
||
| # Configure Bash shell | ||
| if [ ! -f ${HOME}/.bashrc ] | ||
| then | ||
| (echo "source /opt/intel/oneapi/setvars.sh") > ${HOME}/.bashrc | ||
| fi | ||
|
|
||
| source /opt/intel/oneapi/setvars.sh | ||
|
|
||
| exec "$@" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,6 @@ | ||
| FROM registry.access.redhat.com/ubi9/ubi:9.5-1736404036 | ||
|
|
||
| COPY build_llama_and_whisper.sh / | ||
|
|
||
| RUN chmod +x /build_llama_and_whisper.sh ; \ | ||
| /build_llama_and_whisper.sh ramalama |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,10 +1,12 @@ | ||
| FROM registry.access.redhat.com/ubi9/ubi:9.5-1736404036 | ||
| ARG AMDGPU_TARGETS= | ||
| ENV AMDGPU_TARGETS=${AMDGPU_TARGETS} | ||
|
|
||
| COPY rocm/amdgpu.repo /etc/yum.repos.d/ | ||
| COPY rocm/rocm.repo /etc/yum.repos.d/ | ||
|
|
||
| COPY build_llama_and_whisper.sh / | ||
|
|
||
| RUN chmod +x /build_llama_and_whisper.sh ; \ | ||
| /build_llama_and_whisper.sh rocm | ||
|
|
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't need to pass local variables as parameters in bash, they propagate to called functions, they just don't have global scope if that makes sense.