Llama3 8B IQ 1 M takes 22 hours for perplexity testing on wiki text 2

[Abhranta]

I have compiled llama.cpp with GGML_USE_CUDA. I am running Meta-Llama-3-8B-Instruct-IQ1_M.gguf via llama-perplexity. The ETA shows 22 hours (on intel CPU) on wiki-text-2. Meanwhile, when I am running Meta-Llama-3-8B-Instruct-IQ4_NL.gguf, the ETA shows 6 hours. Any idea why?

[joseph777111]

https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md#cuda

GGML_USE_CUDA is not a flag.

Use Make

make GGML_CUDA=1

or

Use CMake

cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release

[Abhranta]

Got it ! Thanks !