How to setup OpenBlas on windows?

[x02Sylvie]

Supposedly there's performance uplift with linking of OpenBLAS

Any hints how to set it up on windows? I have no issues on linux but windows cmake does not seem to want to cooperate with me at the moment in case of OpenBLAS

• Thanks

[cmp-nct]

cmake -G "Visual Studio 17 2022"
cmake --build . --config Release -- /m:24 /verbosity:minimal
Admin: cmake --build . --config Release --target install
by default it creates the .lib, you can change static off so it will create a dll.

Hope it helps, though I don't know how you'll make ggml use the lib/dll. If you get into that would be nice to update here for others.

[LostRuins]

I have already integrated OpenBLAS for windows in my llamacpp for kobold, if you'd like you can use my prebuilt binaries in my fork https://github.com/LostRuins/llamacpp-for-kobold , they are smaller than the official ones. Or you can use the official builds at https://github.com/xianyi/OpenBLAS/releases/download/v0.3.22/OpenBLAS-0.3.22-x64.zip

Note that it only really helps with prompt ingestion (about 2-3x speedup) not the subsequent generation with stochastic sampling.

[x02Sylvie]

Thanks,
I've managed OpenBlas running with ggml. At the very least it says BLAS =1 now when running main.exe and it seems to be calling blas libs functions when i peek in profiler

What i did:

1. Adding this to CMakeList.txt
   add_compile_definitions(GGML_USE_OPENBLAS)
   add_link_options("D:/Projects/LLama/llama.cpp/win64/lib/libopenblas.lib")

2. Putting OpenBlas .h headers into llama.cpp folder directly

3. Putting libopenblas.dll into folder with main.exe

It's messy but i just wanted to see if i'd get any performance improvements

[wmx-github]

Blas is accelerated by compiling GGML provided, so in the llama.cpp/ggml/src/CMakeLists.txt file , add the following code in this line of code if GGML_BLAS front

# add custom blas
if (CUSTOM_BLAS)
  set(BLAS_ROOT "C:/workspace/program/openblas")
  set(BLAS_INCLUDE_DIRS
        "${BLAS_ROOT}/include/"
        "${BLAS_ROOT}/include/openblas"
    )
  set(BLAS_LIBRARIES "${BLAS_ROOT}/lib/openblas.lib")
  list(APPEND GGML_CDEF_PUBLIC GGML_USE_BLAS)
  
  set(GGML_HEADERS_BLAS .. /include/ggml-blas.h)
  set(GGML_SOURCES_BLAS ggml-blas.cpp)
  
  set(GGML_EXTRA_LIBS     ${GGML_EXTRA_LIBS}     ${BLAS_LIBRARIES})
  set(GGML_EXTRA_INCLUDES ${GGML_EXTRA_INCLUDES} ${BLAS_INCLUDE_DIRS})
endif()

Then compile, specifying CUSTOM_BLAS=ON :

cmake -B build -DGGML_BLAS=OFF  -DCUSTOM_BLAS=ON
cmake --build build --config Release