OpenHands icon indicating copy to clipboard operation
OpenHands copied to clipboard

(feat) Add Jupyter Kernel for Interactive Python Interpreter for Sandbox

Open xingyaoww opened this issue 1 year ago • 1 comments

This PR adds Python code execution capability from CodeAct based on IPython to augment the bash command line interface for current agents.

Eventual expected outcome (see video below): Agents can automatically solve user's requests by writing and executing Python code (interactively) and running bash commands (based on our existing sandbox). This interactive Python interpreter supports displaying figures/charts in markdown format via base64, which we can later integrate into the front-end.

Demo of the expected capability - work-in-progress.

Right now, it only supports SSHBox, and I have tested and confirmed it is working in ubuntu with both RUN_AS_DEVIN='True' and RUN_AS_DEVIN='False'.

You can test it locally by first building the image via ./containers/build.sh sandbox (comment out the build architecture line if error), then run python3 opendevin/sandbox/ssh_box.py. Any command you type that starts with py: will be forwarded to the Python Interpreter for execution.

image

xingyaoww avatar Apr 18 '24 17:04 xingyaoww

This is very cool! The demo is neat

rbren avatar Apr 18 '24 19:04 rbren

@rbren, I will merge this so that I can get the new sandbox image built for further development -- but feel free to revert if you think something is wrong! ;)

xingyaoww avatar Apr 19 '24 08:04 xingyaoww