Using Llama.cpp as a dependency in another c++ project.

[psych0v0yager]

I am currently working on a project where I would like to integrate llama.cpp into a game as a dependency. I have a background in python, but I am still pretty new to c++.

As far as I understand, in order to use llama.cpp as a dependency, I need to specify which headers and which source (.cpp) files to include in my xmake.lua file (or CMakeLists.txt).

Based on main.cpp and simple.cpp I only need to include "common.h" and "llama.h". However, there are many more folders in the project whose headers are not mentioned in these examples.

If I wish to use llama.cpp as a dependency, are these two headers enough for my project? If I wish to ship a .dll file, how do I handle the cuda files if I did not mention the ggml-cuda.h header within my xmake.lua/CMakeLists.txt file?

Thanks

[dranger003]

You will need to compile llama.cpp with BUILD_SHARED_LIBS=1. Including llama.h and common.h should work but then you'll need to link with libllama.a and libcommon.a as well (i.e. target_link_libraries in CMakeLists.txt). This is for a static build.

[psych0v0yager]

Thanks for the fast reply.

I did some research and found if I build with BUILD_SHARED_LIBS=1 I will get a shared library .so/.dll. But if I build with LLAMA_STATIC=1 I will get the .a/.lib file. Is this correct or will BUILD_SHARED_LIBS=1 also build the .lib files

Is there an advantage to using the .lib files over the .dll files? I plan to build this on windows? You mention the instructions for a static build. What is the alternative build style?

Also, the other library in my project only works with c++23, whereas llama.cpp is built in c++11. Can I safely build llama.cpp in c++23 or do I need to build llama.cpp as a static library in c++11.

Thanks for your help

[dranger003]

You're right, I meant for a shared build. I haven't been able to get the static build to work, it seems the llama.cpp build currently produces a mix of static and shared libraries, and a static build requires all library files to be built using /MT. There is probably a way to fix this, but I haven't looked into it.

Using a static build means you are packaging all the library code within a single exe file, making it a much larger file but you don't have to bring the library files along with the exe. And using a shared build means you have to bring library files along with the exe but the exe is much smaller in size.

Using BUILD_SHARED_LIBS, you should have common.lib statically linked with llama.dll and you can use it as a shared build in your project so you only need llama.dll along with the exe.

As for the C++ version, I'm using C++23 and it's working fine linking with the library files compiled with C++11.

[psych0v0yager]

I appreciate the reply. Thanks for clearing up the difference between shared and static builds.

I am attempting to build an agent toolkit mod for the Bethesda Games Skyrim and Fallout 4. Both games have a script extender that allows modders to write code in c++. The code is compiled and the .dll is side-loaded into the game via the script extender.

Based on my understanding, I can package the llama.dll file into my mod without needing any additional .lib files. The end user will not have any .lib files or any other llama.cpp build files on their computer. Is the .dll really all I need?

In a more general case, how does static linking work? I assume the common.lib file includes the ggml tensor library as well as the other llama related stuff. How are you able to release the .dll`` file without the .libfiles? And if the.dllfile contains the relevant information from the.libfiles, why do you need the.lib``` files?

Thanks for taking the time to answer my questions.

[dranger003]

That's right, use BUILD_SHARED_LIBS for building llama.cpp then in your project link common.lib and llama.lib and package llama.dll with your plugin dll and you should be all set. So llama.cpp ggml and common are statically linked and you link to llama.lib dynamically. It's all done automatically with the default settings in VS. You can use the VS project settings to add the lib files, or you can do it in your code using #pragma like this:

#pragma comment(lib, "common.lib")
#pragma comment(lib, "llama.lib")

I have a script extender plugin for SFSE, take a look if you want as it would be similar for Skyrim/FO4: https://github.com/dranger003/sfse_plugin_console_api/blob/master/inc/plugin.h

[Orrexon]

Hello! I am trying the same thing in linux. I've built using BUILD_SHARED_LIBS and I've got the libllama.so and libcommon.a.

I am trying to call the functions from a custom program. (obviously :) ) How would I link those files instead of the windows dll/lib ?

[DibyaXPP]

please can you share your CMAKE LIST? I tried this, and it didn't work
cmake -B build -DBUILD_SHARED_LIBS=ON -G "Visual Studio 2022"
cmake --build build --config Release

[dranger003]

Z:\llama.cpp>cmake.exe -S . -B build -DGGML_CCACHE=OFF -DBUILD_SHARED_LIBS=ON
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.26100.0 to target Windows 10.0.22631.
-- The C compiler identification is MSVC 19.42.34435.0
-- The CXX compiler identification is MSVC 19.42.34435.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Professional/VC/Tools/MSVC/14.42.34433/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Professional/VC/Tools/MSVC/14.42.34433/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.47.0.windows.2")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
-- CMAKE_SYSTEM_PROCESSOR: AMD64
-- CMAKE_GENERATOR_PLATFORM:
-- Including CPU backend
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- x86 detected
-- Performing Test HAS_AVX_1
-- Performing Test HAS_AVX_1 - Success
-- Performing Test HAS_AVX2_1
-- Performing Test HAS_AVX2_1 - Success
-- Performing Test HAS_FMA_1
-- Performing Test HAS_FMA_1 - Success
-- Performing Test HAS_AVX512_1
-- Performing Test HAS_AVX512_1 - Failed
-- Performing Test HAS_AVX512_2
-- Performing Test HAS_AVX512_2 - Failed
-- Adding CPU backend variant ggml-cpu: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C
-- Configuring done (16.9s)
-- Generating done (0.8s)
-- Build files have been written to: Z:/llama.cpp/build

Z:\llama.cpp>cmake.exe --build build --config Release -j
...

Z:\llama.cpp>dir *.lib /s /b
Z:\llama.cpp\build\common\build_info.dir\Release\build_info.lib
Z:\llama.cpp\build\common\Release\common.lib
Z:\llama.cpp\build\examples\gguf-hash\sha1.dir\Release\sha1.lib
Z:\llama.cpp\build\examples\gguf-hash\sha256.dir\Release\sha256.lib
Z:\llama.cpp\build\examples\gguf-hash\xxhash.dir\Release\xxhash.lib
Z:\llama.cpp\build\examples\llava\llava.dir\Release\llava.lib
Z:\llama.cpp\build\examples\llava\Release\llama-llava-cli.lib
Z:\llama.cpp\build\examples\llava\Release\llama-minicpmv-cli.lib
Z:\llama.cpp\build\examples\llava\Release\llama-qwen2vl-cli.lib
Z:\llama.cpp\build\examples\llava\Release\llava_shared.lib
Z:\llama.cpp\build\examples\llava\Release\llava_static.lib
Z:\llama.cpp\build\ggml\src\Release\ggml-base.lib
Z:\llama.cpp\build\ggml\src\Release\ggml-cpu.lib
Z:\llama.cpp\build\ggml\src\Release\ggml.lib
Z:\llama.cpp\build\src\Release\llama.lib

Z:\llama.cpp>dir *.dll /s /b
Z:\llama.cpp\build\bin\Release\ggml-base.dll
Z:\llama.cpp\build\bin\Release\ggml-cpu.dll
Z:\llama.cpp\build\bin\Release\ggml.dll
Z:\llama.cpp\build\bin\Release\llama.dll
Z:\llama.cpp\build\bin\Release\llava_shared.dll