Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about Llama3.2 tool call prompt/training (pythonic output) #169

Open
gregnr opened this issue Oct 4, 2024 · 0 comments
Open

Question about Llama3.2 tool call prompt/training (pythonic output) #169

gregnr opened this issue Oct 4, 2024 · 0 comments

Comments

@gregnr
Copy link

gregnr commented Oct 4, 2024

Amazing work on Llama3.2.

In text prompt format you mention that 3.2 uses a new tool calling format that is pythonic in the form:

[func1(params_name=params_value, params_name2=params_value2...), func2(params)]

which is different from the previous JSON format in 3.1:

<|python_tag|>{
    "type": "function",
    "name": "trending_songs",
    "parameters": {
        "n": "10",
        "genre": "all"
    }
}<|eom_id|>

or

<function=trending_songs>{"n": 10}</function><|eot_id|>

Questions

  1. Is it accurate to say 3.2-instruct was trained to produce this pythonic form (over JSON)? Therefore you will get better performance by prompting for this pythonic format over JSON?
  2. You mention that this format is designed to be more flexible and powerful than the previous format. Would you mind sharing the rationale behind this? One benefit I see is that the format is more terse, and so it should be faster to generate than JSON. And perhaps if the model is trained on lots of Python code, it will perform better using a pythonic tool calling format. But JSON is easier to work with in non-python environments, so I was curious if there were other benefits to this new format.

My use case

I'm using this model in a non-python (JavaScript) environment so I would normally stick to a more standard JSON output format, but don't want to compromise performance if the above is true.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant