CUDA Toolkit isn't needed to support GPU

[A2va]

In the troubleshooting GPU guide you describe that the user needs to install CUDA Toolkit.
It's essential to clarify that the CUDA Toolkit is a comprehensive toolchain that includes a compiler, and library. However, for end-users, installing a compiler is typically unnecessary for using an application. It's the only the important to package.

In most cases, incorporating the necessary CUDA libraries with the application itself is more practical and user-friendly. This can be achieved in two ways. The first method involves linking CUDA in a shared manner, provided that the essential .dll or .so files are placed in the same directory as the application.
Alternatively, static linking with CUDA is another approach but it's worth noting that this CUDA Toolkit for Windows doesn't support it.

https://github.com/ggerganov/llama.cpp/blob/f172de03f11465dc6c5a0fc3a22f8ec254c6832c/CMakeLists.txt#L329-L331
If one of the options is made the CUDA Toolkit is not needed.