OpenHands icon indicating copy to clipboard operation
OpenHands copied to clipboard

[Docs]:Enhanced Installation & Getting Started Docs

Open autonomousit opened this issue 1 year ago • 5 comments

End-user friendly description of the problem this fixes or functionality that this introduces This PR provides a more detailed installation and getting started workflow designed to be user friendly for a range of user skill levels and to prevent/deflect requests for setup help for the maintainers. The intended result is a reduction in variability in the deployment state and configuration across users of the OpenHands app (development workflow will be addressed separately).

It's expected that this will require some rounds of feedback so is set to Draft. There are also further changes required to other docs that are part of this effort but I wanted feedback on the general approach ASAP before making too many changes.

  • [ ] Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

Give a summary of what the PR does, explaining any non-trivial design decisions

  • installation.mdx - most of the changes are to this page of the docs. The changes here are designed to move the user from zero to getting started as quickly and reliably as possible. There's a huge correlation between time-to-value and product adoption so this stage is critical for growing any project or product.
    • Most steps have been made more verbose to include potential background context that the reader doesn't posses
    • Pre-requisites have links to 3rd party docs
    • It focuses exclusively on deployment by Docker and using the Web UI
    • The installation is now explicitly via a bash script
    • 2 commonly required configuration options are included by default
      • Mounting a local folder as a Docker volume for OpenHands Session State data - this appears to be a common setting that is required for stability.
      • Setting a JWT Secret - this appears to be a setting required for stability,
    • Common Docker errors are addressed inline to prevent the reader having to break out to another troubleshooting page.
    • Readers are encouraged to the Discord #setup-help channel if they fail to complete these steps
  • getting-started.mdx - minor heading change
  • common-use-cases.mdx - this is a new page designed to highlight common use cases and requirements and then direct users to the correct advanced documentation sections. This is designed to be more or a routing page that doesn't have any unique content but helps new users find and navigate the technical sections of the documentation at a time when they may not know the OpenHands specific terminology or may need to combine features to complete a use case. At this stage it is pretty sparse but as the feature set and documentation grows I think this will become a very useful page. It is also a good place to catch common but more difficult configuration needs that would distract or complicate users during the installation phase. It also acts as a way to advertise the more advanced capabilities of the app early on.
  • troubleshooting.mdx - added additional Docker errors for consistency in case users end up here at a later time.
  • sidebars.ts - added the new Common Use Cases page to the ToC nav tree

Feedback & Open Questions

  • General feedback on approach and style would be very helpful. You can be direct and if you don't like something just say so, you don't need to worry about "hurt feelings" :-) I tend to be direct as well so things are clear, it is not meant to be disrespectful.
  • If I'm trying to change or debate something that is non-negotiable to you as the maintainers please just say so and I'll accept it immediately.
  • In the Installation section there are two TODO markers with questions regarding how to treat some content there. Specifically the case of Custom LLMs during Installation (I think we should skip it and address in the new Common Use Cases page) and another regarding cleanly exiting the Docker deployment when using a bash script. Feedback would be helpful.
  • I can't get the start_openhands.sh method working right now, the frontend times out waiting for the backed client. This shouldn't be merged until Ican resolve that.

Link of any specific issues this addresses #5878

autonomousit avatar Dec 29 '24 20:12 autonomousit

Thanks so much for putting this together! There's definitely some great additional info on here. The installation guide is changing quite a bit and becoming more configurable in this PR and I'm not sure it's exactly what we want (We want to keep this section super simple and basic). Though that's completely my opinion. Documentation is very personal.

Going to bring this up with folks and see what they say.

mamoodi avatar Dec 30 '24 15:12 mamoodi

Hey @mamoodi thanks for the update and context. I agree things are subjective which is why I wanted early feedback before committing too many changes :-) I'd just like to add a few talking points for your discussion with the team. I've run support and sales engineering for a lot of vendors and spent a lot of time working on documentation and knowledge base design and I think the current focus on "simple" misses some important issues. As with your comments these are opinionated and subjective but I think they're important to the discussion. My changes might not be right but I definitely think that some kind of changes are needed.

  1. Simple for who? - you're project is growing and there are lots of posts and vlogs on agentic projects right now. Your users will change from early adopters with high development and technical skills in AI to many other backgrounds. It might be a 15 yr old who just watched "How I built my app in 5 days with know programming skills" to a dev with 10+ years experience in Windows/.NET and is working with a linux system for the first time.
  2. What is the intended outcome? - The current docs leave you at "hello world" but don't set you up for something that is really working well for your project. You then need to find out about things like .openhands-intructions.md, custom sandboxes and other key items needed to get a stable and productive environment. This leaves a lot of room for error and makes it frustrating trying to go from the basic installation to something that does what you really need. For me the Installation and Getting Started should leave me in a production-ready deployment that covers 90% of the use cases I'm going to need. Perhaps Installation can stay simple but then you need to pack lots more details into the new Common Use Cases.
  3. Support Load Grows Fast - the current docs allow a lot of variation on the part of the reader. When they don't work you guys have a lot of work to find out why and try to correct the users state. I had things working for ~1.5 days and now I can't get any Docker instance to work and I can only get 0.16.1 to work via Development mode. However simple the current docs are, they don't work. As you grow, even if the failure rate for setup is only ~5%, that support workload is going to grow faster than you can cope with and distract from adding features.
  4. Commercial Testers Aren't Patient - I see you're also trying to commercialise, this brings other issues. I was originally testing OpenHands to introduce at work but on Day 2, when I tried to put together a demo for a real internal project (not just hello world) everything fell apart and despite a lot of energy from you guys to help me, I still don't have a working environment they way I want. Commercial testers have 2-3 days to test and demo 3-5 potential options. If they can't get things working in <60 mins and put together a reasonable demo and report on why they want to introduce it in under a day then they're going to walk away. The only reason I hung around is it's summer holidays here and I want to use OpenHands for a personal project. I also think with a little more time it could be really good but at this stage it's too unstable for me to introduce at work.

autonomousit avatar Dec 30 '24 19:12 autonomousit

Great points! Thanks for writing all that out!

My understanding was that the installation steps should have you in a working state for Ubuntu, MacOS, Windows with WSL. Unfortunately I can't confirm that myself other than on MacOS and it may not be 100% working on Ubuntu or Windows.

Lots of good points here so I'll defer this to @rbren and the rest of the maintainers on how they want this documentation to look like.

mamoodi avatar Dec 30 '24 20:12 mamoodi

Hi @autonomousit , thanks so much for doing this! I think there's some good stuff in here, like documenting the settings.

On the other hand, I think it'd be good for us to recommend only a single installation pathway at first (only Docker) and then we could cover the building from source one later.

One high-level suggestion -- there's a lot of stuff going on in this PR. I wonder if you could break it down a little bit so we can review and discuss each change separately?

neubig avatar Jan 14 '25 15:01 neubig

Agreed with Graham's comments :)

We should also minimize the number of places we put this docker run command since it's prone to change

And it looks like some troubleshooting info leaked into installation.mdx--we should just link to troubleshooting

rbren avatar Jan 14 '25 17:01 rbren

Hey @neubig and @rbren , thanks for taking the time to review this and provide feedback. Sorry for the delay, have been on holiday and also now back at work so my bandwidth is less than over the end of year holidays.

My key takeways and to-dos from this are:

  • Ignore "dev mode" for now, let's focus on docker deployment - since this is the primary path for a "user" this makes sense to me.
  • Keep error handling in the dedicated troubleshooting section and just use references to that from the main installation page. Again no comments or concerns, easy to do.
  • docker run is a concern due to lack of stability going forward. I also don't like this as it's difficult to read and understand in places. Should we look at moving to a Dockerfile that separates the execution of the container from the configuration variables? This may be more stable and easier to maintain as well.
  • I realise there's a lot here, sorry if it was a bit much. I was initially trying to explore how things interconnected and get a "holistic" approach to the getting started docs and also get a feeling from you guys on what you wanted. I will take a step back and work on smaller chunks at a time so we can go step-by-step.

I haven't deployed anything since 16.1 so I will make some time to start fresh with the latest builds and then circle back with a small snippet focused on getting the very first deployment up and running.

autonomousit avatar Jan 20 '25 02:01 autonomousit

@neubig and @rbren - so I'm trying to get 0.21 running via the docker run command and I'm still hitting the bug where the call to /alive on the runtime container times out (I can see the container is running). I'll look into a separate issue for that but in the mean time I'm looking at the latest docs. They are much neater with the expandable sections during install, etc but the call to docker run is difficult to copy paste due to the line breaks (my terminal doesn't like it). It also gets quite cumbersome as you add more configuration settings.

What I wanted to ask is you feelings on converting this to a docker compose.yml file. What I like about this is docker compose handles starting and stopping gracefully (as opposed to just killing a bash script) and it also allows us to create a compose.yml.template that contains examples of all possible settings and the user can comment/uncomment as needed. Let me know if you want to try this direction and I'll look to put together an example of what it looks like as a first step.

autonomousit avatar Jan 24 '25 20:01 autonomousit

the call to docker run is difficult to copy paste due to the line breaks (my terminal doesn't like it). It also gets quite cumbersome as you add more configuration settings.

What I wanted to ask is you feelings on converting this to a docker compose.yml file. What I like about this is docker compose handles starting and stopping gracefully (as opposed to just killing a bash script) and it also allows us to create a compose.yml.template that contains examples of all possible settings and the user can comment/uncomment as needed. Let me know if you want to try this direction and I'll look to put together an example of what it looks like as a first step.

I agree more focus on compose makes sense, and we already have a basic compose. Maybe we can enhance it to simplify configurations for common needs.

I like that the docker-compose approach is simple, it's just a declarative view of docker run. Same thing in a less cumbersome form, as you pointed out.

Ease of maintenance is a concern when adding alternative approaches, but this one is fine IMO. It's also well known and used out there.

However, I can see some trouble with the idea of a full compose.yml.template file, because... we already have one: config.template.toml with a similar purpose, as you can see. The difference would be that we'd list again all these variables (or almost all) with a different syntax. 😅 I'm trying to remember about its existence for every PR that adds/modifies configuration variables, and run around reminding folks that we need to update this little thing. 😂 Nevertheless, I bet that if we check it all now, we'll find a few misses!

I think a better solution is a script that updates such file. Maybe a github workflow? A periodic resolver task? WDYT?

enyst avatar Jan 25 '25 09:01 enyst

To clarify, I was talking about continuing to use compose and enhance it.

In my understanding, you're also suggesting to replace docker run ...etc with docker compose up.

Personally, I would go with that, as well, for the reasons you outlined, but that's just my opinion. 😄 That would be a fairly important change, it has more implications, and I'd love to know others' view on it.

enyst avatar Jan 25 '25 09:01 enyst

Thanks for the detailed feedback @enyst, I will try and roll up all the feedback from everyone into a new version sometime in the next day or two (have a few deadlines at the moment). For now I will exclude the idea of using docker compose and stick with the current docker run method until you can discuss the implications further.

autonomousit avatar Jan 26 '25 22:01 autonomousit

This PR is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

github-actions[bot] avatar Feb 26 '25 01:02 github-actions[bot]

Hey @enyst , sorry I expected to have a pretty good amount of bandwidth to dedicate to working with OpenHands but when I came back to work after the holidays things have been crazy. I still have this on my to-do list but I can't get to this right now. Might be best to close it and I'll track the project as best I can. When I get bandwidth I'll revisit the getting started docs and see if there's still something to add there.

Sorry again for the lack of follow through, wasn't part of the plan.

autonomousit avatar Feb 28 '25 01:02 autonomousit

Going to close the PR based on the comment. Please feel free to open a new PR when ready.

mamoodi avatar Mar 04 '25 14:03 mamoodi