Async VM #2155
Open
Async VM #2155
+204
−22
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Skipped Deployment
|
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Changes requested ❌
Reviewed everything up to 4809dfd in 2 minutes and 15 seconds. Click for details.
- Reviewed
383lines of code in6files - Skipped
0files when reviewing. - Skipped posting
1draft comments. View those below. - Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. engine/baml-vm/src/vm.rs:818
- Draft comment:
LLM function arguments are currently hardcoded to an empty map. Make sure to implement proper conversion of VM values to runtime values if needed. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% The comment is pointing out something that the code author already knows and has marked with a TODO comment. The comment essentially just restates what's already in the code as a TODO. It's not providing any new information or specific guidance on how to implement the conversion. The comment could be seen as reinforcing the importance of implementing proper value conversion. Maybe there are security or correctness implications that make this worth emphasizing. While proper value conversion is important, the author has already acknowledged this with their TODO comment. The review comment doesn't add any new information or specific guidance that would help implement the conversion. The comment should be deleted since it merely restates what's already marked as a TODO in the code without providing additional actionable guidance.
Workflow ID: wflow_qZSO2GUFVrkgRVJY
You can customize 
| // Either async LLM call or regular function call. | ||
| if self.llm_functions.contains(app.name.name()) { | ||
| self.emit(Instruction::DispatchFuture(app.args.len())); | ||
| self.emit(Instruction::Await(app.args.len())); |
There was a problem hiding this comment.
Instruction::Await is defined without any arguments. Remove the parameter (app.args.len()) when emitting Await.
Suggested change
| self.emit(Instruction::Await(app.args.len())); | |
| self.emit(Instruction::Await); |
| self.tx.send((future, function_name, value)).unwrap(); | ||
| } | ||
|
|
||
| pub fn recv(&self) -> (usize, baml_types::BamlValue) { |
There was a problem hiding this comment.
Avoid a busy spin loop in AsyncChannel::recv; consider yielding the thread or using a blocking recv to prevent high CPU usage.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Important
Adds async LLM function support to BAML VM with new instructions and async handling.
baml-compiler/src/lib.rsandbaml-vm/src/vm.rs.DispatchFutureandAwaitinstructions inbaml-vm/src/bytecode.rs.Vm::exec()inbaml-vm/src/vm.rs.llm_functionstoCompilerstruct inbaml-compiler/src/lib.rs.compile()to handle LLM functions.Futureenum toObjectinbaml-vm/src/vm.rs.AsyncChannelfor async communication inbaml-vm/src/vm.rs.Cargo.tomlto includetokioandbaml-typesdependencies.This description was created by
for 4809dfd. You can customize this summary. It will automatically update as commits are pushed.