Automate local developer setup

[bitplane]

As a developer
I want local setup automated
So I can start hacking with minimal effort

[bitplane]

In a comment rather than the main issue, for discussion's sake:

I tend to use a Makefile in my commercial + personal python projects for this, it's a bit of an abuse but it works pretty well:

https://github.com/bitplane/example-python-project

The idea is that the docs say "type make dev or make help" and they get a dev environment ready to go. The different targets are documented and just run scripts in the scripts dir, which are also used in CI.

What I like about using make for this:

• Onboard devs in one step
• Automation + build steps document the project layout
• Dependencies are automatically managed.
• It only builds steps you need when switching branches (like requirements or lockfile change)
• Add more steps as new components are added

What I don't like:

• You have to use make, it's old and cranky and it hates you being on its lawn.
• It introduces a dependency that isn't in Windows out of the box.

Think it's a reasonable idea? It works well for me and the last few places I've worked at liked it, and #python seems to like the idea too

[andreaskoepf]

We are committed to make the dev experience as sweet as possible. Currently we heavily rely on docker-compose. A challenge is that different parts of the prj need very different tools, e.g. backend, discord-bot, web-frontend, ml-models etc. It is hard to create a single solution that fits all, e.g. while frontends like discord-bot & web need a running backend/db this is not important for the ml guys who are developing training code and evaluating models. I saw makefile based solutions in other projects before, but it is a very custom approach, we have to discuss this internally. As a simple strategy I suggest to first improve the dev documentation and maybe add further scripts.

[bitplane]

Looks like #242 is a cleaner approach. If I'm reading it right then pre-commit is the only thing missing. Close in favour of that?

[andrewm4894]

If we could get something like #242 working then one (minimal effort all in browser) approach would be to just from the GitHub UI on any branch just create a github codespace.

Then from within that codespace you can get to port 3000 to play with the app and or port forward any ports locally too.

Could be nice and seamless if we could get it working - one option then would be for users to just spin up the containers with all the config in a temp vm somewhere in github codespaces. Could be handy for testing stuff or small sort of improvements.

Not an expert in this stuff myself at all so am sure might be others who might have more experience to see if would be feasible and useful addition. Going to see if can get anything working at least on that branch in my fork just out of curiosity.

[bitplane]

I think devcontainers don't have to be used remotely, they work locally in vscode and/or from the command line too (might need node for that? I'm not sure). So this could mean we don't need install instructions or to worry about conflicting versions of ML libraries, fully cross platform without resorting to Conda (building a whole docker container is still faster than conda!)

I'm pretty interested in getting this set up in vscode too if you want it for github, it'll kill two birds with one stone. Are you on Discord? Ping me a DM if you'd like to work on this together, or if you don't have time to collaborate I'd be happy to pick up the vscode part once you've got the github part working.

[andrewm4894]

@bitplane hey - am in the discords (also andrewm4894), only came here today after the live code session Yannic did on YT other day - so still getting up to speed a bit. I tried this: https://github.com/LAION-AI/Open-Assistant/issues/242#issuecomment-1368555982 but looks like maybe will be a bit more digging around to make sure we have whatever we need on whatever default image is being used by github codespaces.

Agree makes sense to try have a single devcontainer.json that could serve both local dev in vs code and via github codespaces. Am sure must be possible but probably just a bit of digging involved.

[fozziethebeat]

This hasn't gotten any comments in over a month and someone has added dev container configs. I'll be closing this as working well enough