Feat: add stream output to exec_run

[iFurySt]

• Using command timeout to control the exec_box's timeout.

https://github.com/OpenDevin/OpenDevin/assets/16201837/cf3e6276-b1bf-44b8-9333-57f824a650de

i found the init is too long and haven't feedback, and IMO i think the init does not even need to control the timeout. i failed to init a few times yesterday because my network seems poor, it made me a bit confused ~

i added the stream output for feedback and kept the timeout but changed to use the timeout command.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 78.37838% with 32 lines in your changes are missing coverage. Please review.

:exclamation: No coverage uploaded for pull request base (main@eba5ef8). Click here to learn what that means.

Files	Patch %	Lines
opendevin/runtime/docker/exec_box.py	67.56%	12 Missing :warning:
opendevin/runtime/docker/ssh_box.py	84.12%	10 Missing :warning:
opendevin/runtime/plugins/mixin.py	75.00%	4 Missing :warning:
opendevin/core/schema/stream.py	84.21%	3 Missing :warning:
opendevin/runtime/e2b/sandbox.py	66.66%	1 Missing :warning:
opendevin/server/agent/agent.py	0.00%	1 Missing :warning:
opendevin/server/session/manager.py	0.00%	1 Missing :warning:

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1625   +/-   ##
=======================================
  Coverage        ?   64.73%           
=======================================
  Files           ?      100           
  Lines           ?     4143           
  Branches        ?        0           
=======================================
  Hits            ?     2682           
  Misses          ?     1461           
  Partials        ?        0           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[rbren]

This is very cool!

I'm a little worried about changing the footprint of execute, which hypothetically should be the same for every box. But I don't think it affects much here.

Do you think it'd be possible to add this to SSH box too?

And do you think it'd be possible to use this to stream output to the FE?

[iFurySt]

I'm a little worried about changing the footprint of execute, which hypothetically should be the same for every box.

i think every box should have a stream output as much as possible, they can easily implement the CancellableStream. (but i'm not so sure that will be the case for all in the future🤯)

Do you think it'd be possible to add this to the SSH box too?

yes, i wanna apply the stream to the SSH box. but i was too busy the last few days, i think i can work at it by the end of the week.

And do you think it'd be possible to use this to stream output to the FE?

i think so, but it may have a bit of trouble when the concurrent sends messages from the BE to FE.

[iFurySt]

https://github.com/OpenDevin/OpenDevin/assets/16201837/dad8b2f3-6747-4618-bb2e-9e9a7192c7cc

ignores the error for BrowserEnv, it has nothing to do with this pr. #350

TypeError: BrowserEnv.__init__() got an unexpected keyword argument 'start_url' was raised from the environment creator for browsergym/openended with kwargs ({'start_url': 'about:blank', 'wait_for_user_message': False, 'headless': True})

[iFurySt]

@rbren @xingyaoww feel free to review it whenever you have time.

[rbren]

This is looking good to me!

I'm mostly worried about the complicated logic for parsing the SSH stream--nothing we can leverage from pexpect or another SSH lib?

[iFurySt]

I'm mostly worried about the complicated logic for parsing the SSH stream--nothing we can leverage from pexpect or another SSH lib?

i can't found related lib to use. paramiko with paramiko-expect are similar with expect. the parse logic just buffer the output and then return line by line, and remove the PROMPT([PEXPECT]) finally.

[SmartManoj]

related lib

@iFurySt Paramiko with continuous stdout You were looking for this?