Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add VSCode to OpenHands runtime and UI #4745

Merged
merged 48 commits into from
Nov 13, 2024
Merged

feat: add VSCode to OpenHands runtime and UI #4745

merged 48 commits into from
Nov 13, 2024

Conversation

xingyaoww
Copy link
Collaborator

@xingyaoww xingyaoww commented Nov 5, 2024

End-user friendly description of the problem this fixes or functionality that this introduces

  • Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

Give a summary of what the PR does, explaining any non-trivial design decisions

Add VSCode based on https://github.com/gitpod-io/openvscode-server, so we can quickly pop up an editor on UI to play around with files and download them properly (the current download zip in UI doesn't work too well)

openhands-vscode.mp4

Link of any specific issues this addresses


To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:6e60203-nikolaik   --name openhands-app-6e60203   docker.all-hands.dev/all-hands-ai/openhands:6e60203

@xingyaoww xingyaoww removed the fix-me Attempt to fix this issue with OpenHands label Nov 8, 2024
@rbren
Copy link
Collaborator

rbren commented Nov 8, 2024

Screenshot 2024-11-08 at 9 55 45 AM

Really rough mock up. Would like to get Paul on this once he's back

@xingyaoww
Copy link
Collaborator Author

Implemented! Ready for another look:

image

@xingyaoww xingyaoww requested a review from rbren November 8, 2024 18:25
@rbren
Copy link
Collaborator

rbren commented Nov 9, 2024

Screenshot 2024-11-09 at 9 47 31 AM

This button should be disabled or invisible until the runtime is ready

@rbren
Copy link
Collaborator

rbren commented Nov 9, 2024

Screenshot 2024-11-09 at 9 48 15 AM

The toast is a little frustrating. It's not enough time for me to read, and long enough that it feels like VS Code is taking a long time.

I'd suggest putting a message into the chat window saying:

You opened VS Code. Please inform the agent of any changes you made to the workspace or environment. To avoid conflicts, it's best to pause the agent before making any changes.

<button
type="button"
onClick={handleVSCodeClick}
className="mt-auto mb-2 w-full h-10 bg-[#0066B8] hover:bg-[#005BA4] text-white rounded flex items-center justify-center gap-2 transition-colors"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make this our normal purple, instead of VScode-blue. Or maybe even something subtler like the gray

Screenshot 2024-11-09 at 9 58 29 AM

- Remove short-lived toast when opening VS Code
- Add persistent message in chat instead
- Remove artificial delay when opening VS Code
- Keep error handling with toasts for failed cases
- Add disabled state when AgentState is INIT or LOADING
- Show grayed out button with not-allowed cursor when disabled
- Keep standard blue styling when enabled
@xingyaoww
Copy link
Collaborator Author

xingyaoww commented Nov 9, 2024

@rbren

Now the button is gray out & not working when the runtime is not initialized

image

And turn blue (our setting modal blue, not vscode blue) when the agent / runtime is ready:

image

And new messages will be added to VSCode (I've been thinking if we need to set a different color / style for this message so user know this is not from the agent)
image

@rbren
Copy link
Collaborator

rbren commented Nov 12, 2024

@xingyaoww sorry for the delay on review! But I'm getting this error when trying to run:

    raise LLMMalformedActionError(
openhands.core.exceptions.LLMMalformedActionError: action={'id': 5, 'timestamp': '2024-11-12T12:01:39.428213', 'source': 'user', 'message': 'do a flip', 'action': 'message', 'args': {'content': 'do a flip', 'image_urls': [], 'wait_for_response': False}} has the wrong arguments: MessageAction.__init__() got an unexpected keyword argument 'image_urls

maybe need to rebase?

@xingyaoww xingyaoww merged commit fd81670 into main Nov 13, 2024
13 checks passed
@xingyaoww xingyaoww deleted the xw/vscode branch November 13, 2024 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants