Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why does my GPU not getting utilized, encodec only utilizes CPU backend #215

Open
naren200 opened this issue Feb 10, 2025 · 0 comments
Open

Comments

@naren200
Copy link

bark.cpp/build/examples/main/main binary doesn't use GPU, it only uses CPU as far as i can see.
I am not sure, why my GPU is not utilized to any extent...

Image

root@thom:~/app/bark.cpp# ./build/examples/main/main -m ./models/bark-small/ggml_weights.bin -p "this is an audio generated by bark.cpp" -t 4

██████╗  █████╗ ██████╗ ██╗  ██╗    ██████╗██████╗ ██████╗ 
██╔══██╗██╔══██╗██╔══██╗██║ ██╔╝   ██╔════╝██╔══██╗██╔══██╗
██████╔╝███████║██████╔╝█████╔╝    ██║     ██████╔╝██████╔╝
██╔══██╗██╔══██║██╔══██╗██╔═██╗    ██║     ██╔═══╝ ██╔═══╝ 
██████╔╝██║  ██║██║  ██║██║  ██╗██╗╚██████╗██║     ██║     
╚═════╝ ╚═╝  ╚═╝╚═╝  ╚═╝╚═╝  ╚═╝╚═╝ ╚═════╝╚═╝     ╚═╝     

encodec_load_model_weights: using CPU backend
encodec_load_model_weights: model size = 44.36 MB

bark_tokenize_input: prompt: 'this is an audio generated by bark.cpp'
bark_tokenize_input: number of tokens in prompt = 513, first 8 tokens: 20579 20172 20199 33733 58966 20203 28169 20222 

Generating semantic tokens... 78%

bark_print_statistics:   sample time =    75.15 ms / 600 tokens
bark_print_statistics:  predict time =  5390.09 ms / 8.98 ms per token
bark_print_statistics:    total time =  5476.48 ms

Building process for GPU access; btw yet GPU is not working

Also, I changed two lines manually in the following files.

  1. File: bark.cpp/bark.cpp
    You also need to modify bark.cpp line 826
    model.backend = ggml_backend_cuda_init();
    by model.backend = ggml_backend_cuda_init(0);

  2. File: bark.cpp/encodec.cpp/encodec.cpp
    You also need to modify encodec.cpp line 215
    model.backend = ggml_backend_cuda_init();
    by model.backend = ggml_backend_cuda_init(0);

After the above changes I built through the following GPU command,

mkdir build
cd build
cmake -DGGML_CUBLAS=ON -DGGML_CUDA=1 -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-11.8 ..
cmake --build . --config Release
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant