-
Notifications
You must be signed in to change notification settings - Fork 10.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Misc. bug: Rpc-server does not use opencl backend on Android. #11957
Comments
Please ask me if you need any other information. |
OpenCL is not included in rpc_server.cpp. It needs to be updated. |
I tried to add everything related to opencl according to rpcserver code, but eventually a compilation error occurred. |
I modified the code and now it compiles successfully, but an error is thrown when the model is loaded. version: 4746 (4806498) |
When I use the --rpc parameter of LLMA-CLI, it will not use the gpu (terminal using LLMA-CLI) of opencl, but vulkan will use both LLMA-CLI and rpc-server. This seems to be a problem with the ggml-opencl header file. Maybe opencl is not supported by the upstream or not updated in time. |
Name and Version
ggml_opencl: using kernels optimized for Adreno (GGML_OPENCL_USE_ADRENO_KERNELS)
version: 4727 (c2ea16f)
built with Android (11349228, +pgo, +bolt, +lto, -mlgo, based on r487747e) clang version 17.0.2 (https://android.googlesource.com/toolchain/llvm-project d9f89f4d16663d5012e5c09495f3b30ece3d2362) for x86_64-unknown-linux-gnu
Operating systems
Other? (Please let us know in description)
Which llama.cpp modules do you know to be affected?
Other (Please specify in the next section)
Command line
Problem description & steps to reproduce
If you use vulkan, it can be used normally in Android, but opencl will prompt that you are using the cpu backend.
Llama-cli can use gpu when using opencl.
First Bad Commit
No response
Relevant log output
The text was updated successfully, but these errors were encountered: