OpenHands icon indicating copy to clipboard operation
OpenHands copied to clipboard

feat: add VSCode to OpenHands runtime and UI

Open xingyaoww opened this issue 1 year ago • 7 comments

End-user friendly description of the problem this fixes or functionality that this introduces

  • [ ] Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

Give a summary of what the PR does, explaining any non-trivial design decisions

Add VSCode based on https://github.com/gitpod-io/openvscode-server, so we can quickly pop up an editor on UI to play around with files and download them properly (the current download zip in UI doesn't work too well)

https://github.com/user-attachments/assets/79cbd443-ff29-45e4-87db-dfa12d142dcd


Link of any specific issues this addresses


To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:6e60203-nikolaik   --name openhands-app-6e60203   docker.all-hands.dev/all-hands-ai/openhands:6e60203

xingyaoww avatar Nov 05 '24 03:11 xingyaoww

This is finally ready for review!

xingyaoww avatar Nov 08 '24 02:11 xingyaoww

Screenshot 2024-11-08 at 9 55 45 AM

Really rough mock up. Would like to get Paul on this once he's back

rbren avatar Nov 08 '24 14:11 rbren

Implemented! Ready for another look:

image

xingyaoww avatar Nov 08 '24 18:11 xingyaoww

Screenshot 2024-11-09 at 9 47 31 AM

This button should be disabled or invisible until the runtime is ready

rbren avatar Nov 09 '24 14:11 rbren

Screenshot 2024-11-09 at 9 48 15 AM

The toast is a little frustrating. It's not enough time for me to read, and long enough that it feels like VS Code is taking a long time.

I'd suggest putting a message into the chat window saying:

You opened VS Code. Please inform the agent of any changes you made to the workspace or environment. To avoid conflicts, it's best to pause the agent before making any changes.

rbren avatar Nov 09 '24 14:11 rbren

@rbren

Now the button is gray out & not working when the runtime is not initialized

image

And turn blue (our setting modal blue, not vscode blue) when the agent / runtime is ready:

image

And new messages will be added to VSCode (I've been thinking if we need to set a different color / style for this message so user know this is not from the agent) image

xingyaoww avatar Nov 09 '24 19:11 xingyaoww

@xingyaoww sorry for the delay on review! But I'm getting this error when trying to run:

    raise LLMMalformedActionError(
openhands.core.exceptions.LLMMalformedActionError: action={'id': 5, 'timestamp': '2024-11-12T12:01:39.428213', 'source': 'user', 'message': 'do a flip', 'action': 'message', 'args': {'content': 'do a flip', 'image_urls': [], 'wait_for_response': False}} has the wrong arguments: MessageAction.__init__() got an unexpected keyword argument 'image_urls

maybe need to rebase?

rbren avatar Nov 12 '24 17:11 rbren