Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Misc. bug: Rpc-server does not use opencl backend on Android. #11957

Open
belog2867 opened this issue Feb 19, 2025 · 5 comments
Open

Misc. bug: Rpc-server does not use opencl backend on Android. #11957

belog2867 opened this issue Feb 19, 2025 · 5 comments

Comments

@belog2867
Copy link

Name and Version

ggml_opencl: using kernels optimized for Adreno (GGML_OPENCL_USE_ADRENO_KERNELS)
version: 4727 (c2ea16f)
built with Android (11349228, +pgo, +bolt, +lto, -mlgo, based on r487747e) clang version 17.0.2 (https://android.googlesource.com/toolchain/llvm-project d9f89f4d16663d5012e5c09495f3b30ece3d2362) for x86_64-unknown-linux-gnu

Operating systems

Other? (Please let us know in description)

Which llama.cpp modules do you know to be affected?

Other (Please specify in the next section)

Command line

./rpc-server --port 8880

Starting RPC server on 127.0.0.1:8880, backend memory: 15160 MB

Problem description & steps to reproduce

If you use vulkan, it can be used normally in Android, but opencl will prompt that you are using the cpu backend.

Llama-cli can use gpu when using opencl.

First Bad Commit

No response

Relevant log output

create_backend: using CPU backend
@belog2867
Copy link
Author

Please ask me if you need any other information.

@foldl
Copy link
Contributor

foldl commented Feb 20, 2025

OpenCL is not included in rpc_server.cpp. It needs to be updated.

@belog2867
Copy link
Author

belog2867 commented Feb 20, 2025

I tried to add everything related to opencl according to rpcserver code, but eventually a compilation error occurred.

@belog2867
Copy link
Author

belog2867 commented Feb 20, 2025

I modified the code and now it compiles successfully, but an error is thrown when the model is loaded.
tensor->extra populated by the backend, this is currently unsupported.

version: 4746 (4806498)
built with clang version 19.1.7 for aarch64-unknown-linux-android24

rpc-server.cpp.txt

@belog2867
Copy link
Author

When I use the --rpc parameter of LLMA-CLI, it will not use the gpu (terminal using LLMA-CLI) of opencl, but vulkan will use both LLMA-CLI and rpc-server. This seems to be a problem with the ggml-opencl header file. Maybe opencl is not supported by the upstream or not updated in time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants