This repository was archived by the owner on Aug 7, 2025. It is now read-only.

Llamacpp with cpp backend#2527

Closed

shrinath-suresh wants to merge 11 commits intopytorch:cpp_backendfrom

shrinath-suresh:llm-cpp-backend

Commits on Aug 16, 2023

Version1 of llm inference with cpp backend
shrinath-suresh
committed

Commits on Aug 17, 2023

Updating llm handler - loadmodel, preprocess, inference methods
shrinath-suresh
committed

Commits on Aug 21, 2023

Fixed infinite lock by adding request ids to the preprocess method
shrinath-suresh
committed
Adding test script for finding tokens per second llama-7b-chat and ggml version
shrinath-suresh
committed

Commits on Sep 15, 2023

Processing all the items in the batch
shrinath-suresh
committed

Commits on Jan 12, 2024

Adopted llama.cpp api changes
mreso
committed