Simplified llama.dll

[JohnClaw]

Llama.dll that can be downloaded from llama.cpp repo is suitable mostly for programming languages that have abilities to work with rather difficult (for novice coder) concepts like pointers, structures etc. Does anybody know any simplified implementation of llama.dll? For example, it would be wonderful if it will be possible to work with this dll in such way:

CONSOLE

Dim LLama_DLL as Integer

Load "llama.dll" as LLama_DLL

Call function Load_Local_Gguf ("Llama-2-7b-q8.0.gguf") from LLama_DLL

Call function Set_Context_Size_For_Loaded_Gguf (4096) from LLama_DLL

Dim Answer$ As String

Dim Question$ As String

Label #Begin

Input Question$

Answer$ = Call function SentPromptToLoadedGguf (Question$) from LLama_DLL

Print Answer$

Goto #Begin

[martindevans]

A simplified/higher level wrapper around llama.dll is basically the entire point of this project!

For example here's all of the code required to load a set of model weights and to generate a response to a prompt (taken from this example).

// Load model
var parameters = new ModelParams("the/path/to/your/model.gguf");
using var model = LLamaWeights.LoadFromFile(parameters);

// Create "executor"
var ex = new StatelessExecutor(model, parameters);

// Setup some parameters for exactly how you want to use the model
var inferenceParams = new InferenceParams() { Temperature = 0.6f, MaxTokens = 50 };

// What's the input
var input = "cats are cute because";

// Generate the output
await foreach (var text in ex.InferAsync(input, inferenceParams))
{
    Console.Write(text);
}

[JohnClaw]

I meant low-api functions that should be called directly from simplified llama.dll. I should make clear what i mean. For example, i use BCX to code my apps: https://bcxbasiccoders.com/ BCX is a dialect of Basic. It supports simple work with dll-s like this:

DECLARE FUNCTION Example_Function$ LIB "EXAMPLE_DLL.dll" ALIAS "Hello_World"(Param$)
PRINT "Calling function from dll says: ", Example_Function$("hello")

And the function in .dll looks like this:

FUNCTION Hello_World$(A$) EXPORT
LOCAL T$
T$ = A$ & ",World"
FUNCTION = T$
END FUNCTION

BCX is a simple language so it doesn't support complicated coding concepts available in C++ or C#. That's why BCX can't call functions from original llama.dll that was coded and compiled in C++.

If you don't mind i will attach BCX help file (it's small and can be read very quickly) to make clear what BCX can do with dll-s and what it can't.
BCXHelp.zip

[SignalRT]

The project goal is to be able to execute LLMs from .NET It seems that you are searching for bindings for BCX language.

[JohnClaw]

Yes, i'm searching for bindings/wrapper to any dialect of Basic or for a universal simplified llama.dll that can be used with any programming language.