Failure on most examples I have tried.

[lastrosade]

Trtying "example_agent_retrieval_memory.py" yielded:

pydantic.errors.PydanticImportError: `BaseSettings` has been moved to the `pydantic-settings` package. See https://docs.pydantic.dev/2.5/migration/#basesettings-has-moved-to-pydantic-settings for more details.

Trying "example_agent_auto_coder.py" gave me

<|im_start|>user
Implement the existing development plan in the "./" folder, for a chat bot frontend in HTML, CSS and Javascript with a dark UI.<|im_end|>
<|im_start|>assistant
{
  "function": "send-message-to-user",
  "function-parameters": {
    "message": "Starting the development plan implementation."
  }
Traceback (most recent call last):
  File "A:\AI_MODELS\LLM\llama-cpp-agent\example_agent_auto_coder.py", line 65, in <module>
    user_input = coder_agent.get_chat_response(
  File "C:\Users\pedr0\AppData\Roaming\Python\Python310\site-packages\llama_cpp_agent\llm_agent.py", line 327, in get_chat_response
    full_response = function_tool_registry.handle_function_call(full_response)
  File "C:\Users\pedr0\AppData\Roaming\Python\Python310\site-packages\llama_cpp_agent\function_calling.py", line 183, in handle_function_call
    function_call = parse_json_response(function_call_response)
  File "C:\Users\pedr0\AppData\Roaming\Python\Python310\site-packages\llama_cpp_agent\output_parser.py", line 42, in parse_json_response
    json_object = json.loads(sanitized)
  File "C:\Program Files\Python310\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files\Python310\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Program Files\Python310\lib\json\decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 130 (char 129)

Trying "example_multi_file_generation.py" gives me

      "file_content": "from swarm_ant import predict_ant_behavior\nfrom swarm_bee import predict_bee_behavior\n\n# Define a function to predict the behavior of the swarm\ndef predict_swarm_behavior(ant_data, bee_data):\n    # Predict the behavior of the ant\n    ant_label = predict_ant_behavior(ant_data)\n\n    # Predict the behavior of the bee\n    bee_label = predict_bee_behavior(bee_data)\n\n    # Combine the labels\n    combined_label = (ant_label + bee_label) / 2\n\n    return combined_label\n\n"
    }]
}<|im_end|>
Traceback (most recent call last):
  File "A:\AI_MODELS\LLM\llama-cpp-agent\example_multi_file_generation.py", line 81, in <module>
    program = develop(
  File "A:\AI_MODELS\LLM\llama-cpp-agent\example_multi_file_generation.py", line 75, in develop
    json_obj = json.loads(response)
  File "C:\Program Files\Python310\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files\Python310\lib\json\decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 19 column 2 (char 2392)

This happens with both Llama() and LlamaCppEndpointSettings() with two different models.

The simple example works, though.

Am I missing something? Are these dependency problems? Maybe python version? Model quants?

[Maximilian-Winter]

No, I think I have broken some things with my update for function calling, will push update with working examples ASAP.
Should be done later today.

[Maximilian-Winter]

Sorry for your problems, but I fixed all problems. Could you test again? And thank you for reporting the problems.

[Maximilian-Winter]

@lastrosade Make sure to have the latest version of the llama.cpp server and Pydantic!

[lastrosade]

Thank you for your answers.

I just updated llama-cpp-agent, running example_agent_auto_coder.py yields ModuleNotFoundError: No module named 'docstring_parser' which is missing from the dependencies

[lastrosade]

With docstring_parser installed, example_agent_auto_coder.py seems to run fine. Although it loops indefinitely when finished.

[lastrosade]

both example_agent_structured_output.py and example_book_dataset_creation.py return

Traceback (most recent call last):
  File "A:\AI_MODELS\LLM\llama-cpp-agent\example_book_dataset_creation.py", line 38, in <module>
    gbnf_grammar, documentation = generate_gbnf_grammar_and_documentation([Book])
  File "C:\Users\pedr0\AppData\Roaming\Python\Python310\site-packages\llama_cpp_agent\gbnf_grammar_generator\gbnf_grammar_from_pydantic_models.py", line 1078, in generate_gbnf_grammar_and_documentation
    documentation = generate_markdown_documentation(
  File "C:\Users\pedr0\AppData\Roaming\Python\Python310\site-packages\llama_cpp_agent\gbnf_grammar_generator\gbnf_grammar_from_pydantic_models.py", line 719, in generate_markdown_documentation
    documentation += generate_field_markdown(
  File "C:\Users\pedr0\AppData\Roaming\Python\Python310\site-packages\llama_cpp_agent\gbnf_grammar_generator\gbnf_grammar_from_pydantic_models.py", line 793, in generate_field_markdown
    if isclass(field_type) and issubclass(field_type, BaseModel):
  File "C:\Program Files\Python310\lib\abc.py", line 123, in __subclasscheck__
    return _abc_subclasscheck(cls, subclass)
TypeError: issubclass() arg 1 must be a class

[lastrosade]

example_agent_retrieval_memory.py still returns the same error even after upgrading pydantic.
Fixed by upgrading chromadb; pip is the worst package manager I have ever used.

Running example_agent_retrieval_memory.py now yields:

Traceback (most recent call last):
  File "A:\AI_MODELS\LLM\llama-cpp-agent\example_agent_retrieval_memory.py", line 15, in <module>
    function_tool_registry = LlamaCppAgent.get_function_tool_registry(function_tools)
  File "C:\Users\pedr0\AppData\Roaming\Python\Python310\site-packages\llama_cpp_agent\llm_agent.py", line 86, in get_function_tool_registry
    function_tool_registry.finalize()
  File "C:\Users\pedr0\AppData\Roaming\Python\Python310\site-packages\llama_cpp_agent\function_calling.py", line 130, in finalize
    gbnf_grammar, documentation = generate_gbnf_grammar_and_documentation(
  File "C:\Users\pedr0\AppData\Roaming\Python\Python310\site-packages\llama_cpp_agent\gbnf_grammar_generator\gbnf_grammar_from_pydantic_models.py", line 1078, in generate_gbnf_grammar_and_documentation
    documentation = generate_markdown_documentation(
  File "C:\Users\pedr0\AppData\Roaming\Python\Python310\site-packages\llama_cpp_agent\gbnf_grammar_generator\gbnf_grammar_from_pydantic_models.py", line 719, in generate_markdown_documentation
    documentation += generate_field_markdown(
  File "C:\Users\pedr0\AppData\Roaming\Python\Python310\site-packages\llama_cpp_agent\gbnf_grammar_generator\gbnf_grammar_from_pydantic_models.py", line 752, in generate_field_markdown
    field_info = model.model_fields.get(field_name)
AttributeError: type object 'SendMessageToUser' has no attribute 'model_fields'