Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

instruct the agent to comment less #4681

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

instruct the agent to comment less #4681

wants to merge 3 commits into from

Conversation

rbren
Copy link
Collaborator

@rbren rbren commented Nov 1, 2024

End-user friendly description of the problem this fixes or functionality that this introduces

  • Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below
    CHANGELOG: modified agent instructions to include fewer redundant comments in the code it generates

Give a summary of what the PR does, explaining any non-trivial design decisions

OpenHands constantly puts comments like this:

// greet the user with console.log
console.log('hello!');

These sorts of comments are almost always unhelpful, and they quickly fall out of sync with the code, creating confusion.

Even worse, it'll do stuff like this when editing existing code

// greet the user with console.log
console.log('greetings!');  // our new message instead of 'hello'

This PR instructs the agent to comment less. Not quite sure how to validate, other than manual testing. I'm also slightly concerned it might muck with the LLM-based editing strategy


Link of any specific issues this addresses


To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=ghcr.io/all-hands-ai/runtime:67c8ce9-nikolaik   --name openhands-app-67c8ce9   ghcr.io/all-hands-ai/runtime:67c8ce9

Copy link
Collaborator

@tofarr tofarr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - I wonder if we can comment to get the agent to produce code that has more shorter methods instead of fewer long ones?

@enyst
Copy link
Collaborator

enyst commented Nov 1, 2024

Is this intended with (a few) function calling LLMs we currently support, or with most LLMs? With function calling, the prompt in use is here.

@xingyaoww
Copy link
Contributor

^^ @enyst is correct - we should probably look into the function calling SYSTEM prompt

@rbren rbren marked this pull request as ready for review November 4, 2024 22:41
@rbren rbren marked this pull request as draft November 4, 2024 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants