Skip to content

Support parallel tool calling #457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Support parallel tool calling #457

wants to merge 1 commit into from

Conversation

hadley
Copy link
Member

@hadley hadley commented Apr 16, 2025

Fixes #356

@simonpcouch can you please kick the tires a bit?

@hadley hadley requested a review from simonpcouch April 16, 2025 15:47
@simonpcouch
Copy link
Contributor

HELL YEAH

Will convert vitals back to $chat_parallel() in a moment and leave a review tomorrow after some stress-testing.

@simonpcouch
Copy link
Contributor

simonpcouch commented Apr 17, 2025

Still working on a more minimal reprex, but bringing this up before I need to hop into a couple meetings: It seems like the responses in the output are sometimes "scrambled" following uneven numbers tool calls.

Here's a whittled down version of the vignette where this came up:

library(ellmer)
library(btw)
# just for the `are` dataset
library(vitals)

ch <- chat_anthropic(model = "claude-3-7-sonnet-latest")
btw_register_tools(ch, tools = "docs")

res <- ch$chat_parallel(as.list(are$input[1:5]))

For example, here's one chat transcript where the model tool calls for some relevant documentation, and then, in its response, provides an answer to a different question from the list of prompts:

<Chat Anthropic/claude-3-7-sonnet-latest turns=4 tokens=5842/352 $0.02>
── user [1368] ───────────────────────────────────────────────────────────────────────────────────────────
I see:

```
library(dplyr)

mtcars |> mutate(across(vs, \(x) x +1, .names = "x",))
#> Error in `mutate()`:
#> ℹ In argument: `across(vs, function(x) x + 1, .names = "x", )`.
#> Caused by error in `across()`:
#> ! Can't compute column `x`.
#> Caused by error in `fn()`:
#> ! Unused argument (alist())
#> Backtrace:
#>      ▆
#>   1. ├─dplyr::mutate(...)
#>   2. ├─dplyr:::mutate.data.frame(...)
#>   3. │ └─dplyr:::mutate_cols(.data, dplyr_quosures(...), by)
#>   4. │   ├─base::withCallingHandlers(...)
#>   5. │   └─dplyr:::mutate_col(dots[[i]], data, mask, new_columns)
#>   6. │     └─mask$eval_all_mutate(quo)
#>   7. │       └─dplyr (local) eval()
#>   8. ├─dplyr::across(vs, function(x) x + 1, .names = "x", )
#>   9. │ └─base::withCallingHandlers(...)
#>  10. └─base::.handleSimpleError(...)
#>  11.   └─dplyr (local) h(simpleError(msg, call))
#>  12.     └─rlang::abort(bullets, call = error_call, parent = cnd)
栀Ƭ
#> Error in eval(expr, envir, enclos): object'栀Ƭ' not found
```

How do I fix it?
── assistant [142] ───────────────────────────────────────────────────────────────────────────────────────
I see you're encountering an error when using the `across()` function in dplyr. The problem appears to be
related to how you're using the `.names` parameter along with a trailing comma. Let me check the 
documentation for `across()` to help you fix this issue.
[tool request (toolu_01GufsApW4CFyT52vKfGtxJh)]: btw_tool_docs_help_page(package_name = "dplyr",
topic = "across")
── user [2964] ───────────────────────────────────────────────────────────────────────────────────────────
[tool result  (toolu_01GufsApW4CFyT52vKfGtxJh)]:
across                  package:dplyr                  R Documentation

Apply a function (or functions) across multiple columns

<snip>The rest of the across documentation</snip>
     
── assistant [210] ───────────────────────────────────────────────────────────────────────────────────────
Based on the documentation, I can see that `position = "fill"` is what we need to modify the plot to show
proportions. This position will standardize each bar to have the same height, effectively showing the 
proportion of each clarity category within each cut.

Here's the modified code:

```
ggplot(data = diamonds) + 
  geom_bar(mapping = aes(x = cut, fill = clarity), position = "fill")
```

This code will:
1. Create a bar chart where each bar represents a diamond cut
2. Set all bars to the same height (1.0 or 100%)
3. Stack and fill each bar by clarity categories
4. The y-axis will now show proportions from 0 to 1 instead of counts

The resulting plot will show you the relative proportion of different clarity categories within each cut 
of diamond, allowing you to easily compare the distribution of clarity across different cuts.

I've also seen the opposite, where the initial prompt and the final answer line up, but some intermediate tool is completely unrelated and the model doesn't seem to have "seen" that it happened.

@hadley hadley requested a review from Copilot April 17, 2025 19:06
Copilot

This comment was marked as off-topic.

@hadley
Copy link
Member Author

hadley commented Apr 17, 2025

@simonpcouch it's totally possible I've messed up the indexing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

incomplete turns after tool calls with $chat_parallel()
2 participants