Create hosted demo

[neubig]

What problem or use case are you trying to solve?

Right now one major issue that we are experiencing is that people are having issues installing OpenDevin, we would like to have a hosted demo that people could use without having to set it up on their own computers.

Describe the UX of the solution you'd like

A user could sign up, log in, and start using OpenDevin.

Do you have thoughts on the technical implementation?

This will require lots of thought including.

• How to host the frontend/backend.
• Where to run agents.
• How to pay for it.

etc. etc.

[yimothysu]

Right now the frontend assumes it's run on the same machine as the backend. This is mostly due to the workspace directory setting; a user can currently set the workspace path to work with existing code on their machine.

If the backend isn't necessarily the user's machine, then the workspace directory is not relevant. We'll want to support uploading an existing project to the server instead. Since sessions are temporary, we'll also want to support downloading projects, version control, or some persistence method.

[foragerr]

I'm optimistic about https://github.com/OpenDevin/OpenDevin/pull/1023 fixing the installation woes.

The biggest barrier to hosting a demo is, how to pay for it. if there's cash, hosting options are a-plenty.

[zhonggegege]

When a project is infinitely close to maturity, your idea is reasonable, but if the purpose is to solve the problem of installing OpenDevin, I think it is a more valuable option to provide better installation environment adaptation and automated judgment scripts. For example, it seems more valuable and urgent to reduce the difficulty of project deployment by providing packaged docker images and automated conditional judgment script solutions as much as possible.

This's exactly what I have observed in a project. By integrating script automation with docker images, it has received more attention and positive experience in similar projects. Project address: QAnything

This's my opinion.

[neubig]

Thanks @zhonggegege ! We're also working on easier local install, as @foragerr mentioned. This issue is for even easier use than that, and it's on our roadmap for June, a somewhat longer way out. So basically, we agree!

[openartist]

I believe the best route is to just wrap it in electron, allow people to download the app.

[foragerr]

just wrap it in electron

Thoughts on VS Code plugin?

[rbren]

VSCode plugin and electron app are both great ideas! We should track those in separate issues

[rbren]

For the hosted demo, here's what I'm thinking:

• Users log in with GitHub
  • this gives us access to their code, which can be easily pulled in
  • it also lets us limit usage on a per-user basis to keep costs down
• Users must add an API key for a publicly available LLM
• Everything runs in Kubernetes
  • Helm chart for deploying
  • We create a KubernetesSandbox which creates Deployments in Kubernetes
    • These can be stood up and torn down quickly
    • We use ssh to connect into the sandbox
    • We can limit memory and CPU usage

And here are some features we'll need to support before we have a hosted demo:

• Add API keys via UI (stored in localStorage)
• Log in with GitHub (maybe just using a developer token in the self-hosted version)
• Create workspace from GitHub project
• Multi-tenancy
  • need ability to run multiple agents at once, each tied to a different session

We will probably need a database, but I'm not convinced of that yet. I'd love to get away with not having one until we really need it. We can probably do some basic persistence using S3 (and local files for self-hosted).

[foragerr]

Users must add an API key for a publicly available LLM

Would you trust a random website with your OpenAI keys? Probably yes because OpenAI makes managing budgets and limits easy - but it's worth pondering.

I wonder if OpenAI supports OIDC similar to "login with github".

[rbren]

I wonder if OpenAI supports OIDC similar to "login with github".

I don't think they do, but I'd be surprised if it's not on their roadmap. It'd be a very easy way to drive revenue.

Re: key trust, I plan to only store them in localStorage, and not on the backend. We can put a note about that in the app, and it's hypothetically something people could audit via the network tab

[tlpythonuse-colab]

I've made a quick hosted demo with Bring-Your-Own-Key (BYOK) taking into account the suggestions in this thread. Keys are purged when unused and strict budgeting is available. Here's a quick demonstration (audio on 🔉):

https://github.com/OpenDevin/OpenDevin/assets/134402079/0cb535db-647c-4674-ac30-b313eee5aad7

I could deploy gVisor for stronger isolation between users (on top of rootless Docker) and hope to get everything up soon. Compute costs are rather insignificant at this time with BYOK, so the demo can be free, and the demo's architecture is scalable across multiple hosts.

For now, MonologueAgent and CodeActAgent have been validated working (after some modification to entrypoint, plugins etc)! The work done for this demo could be useful in getting Electron / desktop app versions up as well, which I could explore.

Also, by Dockerising each OpenDevin instance, we could avoid needing to spend as much time on multi-tenancy and more time for agents and the like. Users can then also get completely separate environments.

Any feedback is appreciated, thanks! 😀

[rbren]

@tlpythonuse-colab this is rad! Is the code open source?

[rbren]

@tlpythonuse-colab I'd love to learn more about this if you want to get in touch with me on the Slack, or via the email in my GitHub profile

[PierrunoYT]

It looks cool :)

[AIFlowML]

This is really good. Where can we try it ?

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[assertion]

Any update on this issue @rbren ? I'm also thinking recently, about how to deploy opendevin to a distributed arch. If we deploy the whole opendevin as an instance, each instance can serve to many users/sessions ( with many agents running inside). But there maybe some problems we need to solve. For example: this is a stateful service, we need to route every request of a websocket connection to the same opendevin instance.

Or if we can seperate the agent execution process, so we can start a agent_runtime for each session on something like a container?

[mamoodi]

I believe this is something folks are working on but it will take some time to get up and running. Lots to consider.

[richardokonicha]

@rbren how is this going, would like to contribute to this effort.

[turbobuilt]

ok we could use jslinux to run an entire os in the browser on which opendevin runs, could we not?

https://bellard.org/jslinux/

Linux in browser, 64MB of downloads tho

https://bellard.org/jslinux/vm.html?cpu=riscv64&url=fedora33-riscv-xwin.cfg&graphic=1&mem=256

[turbobuilt]

Hm, just noticed it's doing risc v. I don't know if that's an issue