Would a devcontainer set up be useful? eg make dev in github codespaces possible?

[andrewm4894]

trafficstars

I am wondering if it might be useful to create a devcontainer in this repo - this could help lower the entry barrier to users looking to maybe help or contribute or maybe even a devcontainer to run example apps via github codespaces in some way if that was to be possible.

Not sure if this is or is not a good idea given may be some more specific hardware requirements or constraints in this project but just wanted to throw it out there as an idea.

e.g. i think it could be possible to make a devcontainer.json such that the quickstart try it out could be done via a github codespace in browser if users wanted to go that way.

[andrewm4894]

I tried to just run as is in a github codespace to see if it would just work by seeing the docker compose file in root of the repo but does not seem to work for me. Not 100% sure why, made a fork and will try maybe play a bit with it to see if i can get anything that could be basis for a maybe useful PR together.

[yk]

Hey thanks for the suggestion, I think that's a great idea! I've assigned you for now to the issue, let us know if you find something useful :)

[andrewm4894]

Cool, will have a go at it and see if can get anything useful going.

[andrewm4894]

Adding for reference. Seems like below SO may be a similar set up to what I think we have in this repo.

https://stackoverflow.com/questions/67773973/how-to-run-docker-compose-inside-vs-code-devcontainer

[bitplane]

I've heard good things about these from devs I trust and respect but never used them myself. I take it this would remove the need to have Node, Python, ML libraries etc installed on local dev machines too?

If the entire dev setup steps are git clone [email protected]:LAION-AI/Open-Assistant.git && code Open-Assistant then that sounds like a massive win to me, the closer we can get to that the better :)

[andrewm4894]

I am getting the below error in the creation log on codespace.

Seems like this is main issue:

2023-01-01 21:18:19.961Z: The Compose file is invalid because:
2023-01-01 21:18:19.961Z: Service backend has neither an image nor a build context specified. At least one must be provided.

I am working in here, i am just starting by trying to add as minimal a devcontainer.json as i can get away with.

It seems like image: oasst-backend is maybe what codespaces perhaps does not know abount in whatever container registry it is pulling from perhaps. Where would i find the oasst-backend image?

=================================================================================
2023-01-01 21:18:17.677Z: Configuration starting...
2023-01-01 21:18:17.684Z: Cloning...

====================================== ERROR ====================================
2023-01-01 21:18:17.691Z: DockerComposeError: System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.CloudEnvironments.CLI.UnifiedContainers.UnifiedDevContainerUtils.WriteOverrideDockerComposeFile(TraceSource trace, IDiagnosticsLogger logger, FilePathProviderBase filePathProvider, JObject devContainerJson, String devContainerFolderPath, String containertype, Boolean gpu) in /__w/1/s/src/agent/CLI/UnifiedContainers/UnifiedDevContainerUtils.cs:line 859
   at Microsoft.CloudEnvironments.CLI.UnifiedContainers.UnifiedDevContainerUtils.MergeDevContainerJsonProperties(TraceSource trace, IDiagnosticsLogger logger, IProviderBundle providerBundle, IOutputWriter outputWriter, JObject orgDevContainer, JObject systemDevContainer, JObject repoDevContainer, JObject internalProperties, String devContainerFilePath, FlowTypes flowType) in /__w/1/s/src/agent/CLI/UnifiedContainers/UnifiedDevContainerUtils.cs:line 323
=================================================================================

=================================================================================
2023-01-01 21:18:17.965Z: Creating container...
2023-01-01 21:18:18.337Z: @microsoft/vscode-dev-containers-cli 0.78.0.
2023-01-01 21:18:18.361Z: Start: Resolving Remote
2023-01-01 21:18:18.361Z: $ docker ps -q -a --filter label=Type=codespaces
2023-01-01 21:18:18.389Z: Stop (46 ms): Run: docker ps -q -a --filter label=Type=codespaces
2023-01-01 21:18:18.389Z: $ /usr/bin/node /usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /var/lib/docker/codespacemount/.persistedshare --container-data-folder .vscode-remote/data/Machine --container-system-data-folder /var/vscode-remote --workspace-folder /var/lib/docker/codespacemount/workspace/Open-Assistant --id-label Type=codespaces --log-level info --log-format json --config /var/lib/docker/codespacemount/workspace/Open-Assistant/.devcontainer/devcontainer.json --override-config /root/.codespaces/shared/merged_devcontainer.json --default-user-env-probe loginInteractiveShell --remove-existing-container --mount type=bind,source=/.codespaces/agent/mount/cache,target=/vscode --skip-post-create --update-remote-user-uid-default never --mount-workspace-git-root false
2023-01-01 21:18:18.618Z: @devcontainers/cli 0.26.0. Node.js v14.21.1. linux 5.4.0-1094-azure x64.
2023-01-01 21:18:18.618Z: $ docker buildx version
2023-01-01 21:18:18.753Z: Stop (134 ms): Run: docker buildx version
2023-01-01 21:18:18.753Z: github.com/docker/buildx 0.9.1+azure-2 ed00243a0ce2a0aee75311b06e32d33b44729689
2023-01-01 21:18:18.753Z: 
2023-01-01 21:18:18.753Z: Start: Resolving Remote
2023-01-01 21:18:18.753Z: $ docker-compose version --short
2023-01-01 21:18:19.345Z: Stop (580 ms): Run: docker-compose version --short
2023-01-01 21:18:19.345Z: Docker Compose version: 1.29.2
2023-01-01 21:18:19.345Z: $ docker ps -q -a --filter label=com.docker.compose.project=open-assistant --filter label=com.docker.compose.service=undefined
2023-01-01 21:18:19.372Z: Stop (27 ms): Run: docker ps -q -a --filter label=com.docker.compose.project=open-assistant --filter label=com.docker.compose.service=undefined
2023-01-01 21:18:19.373Z: $ docker-compose -f /var/lib/docker/codespacemount/workspace/Open-Assistant/docker-compose.yaml config
2023-01-01 21:18:19.961Z: Stop (588 ms): Run: docker-compose -f /var/lib/docker/codespacemount/workspace/Open-Assistant/docker-compose.yaml config
2023-01-01 21:18:19.961Z: 
2023-01-01 21:18:19.961Z: The Compose file is invalid because:
2023-01-01 21:18:19.961Z: Service backend has neither an image nor a build context specified. At least one must be provided.
2023-01-01 21:18:19.961Z: 
2023-01-01 21:18:19.961Z: Exit code 1
2023-01-01 21:18:19.961Z: Error: Command failed: docker-compose -f /var/lib/docker/codespacemount/workspace/Open-Assistant/docker-compose.yaml config
2023-01-01 21:18:19.962Z:     at Fc (/usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js:1856:867)
2023-01-01 21:18:19.962Z:     at processTicksAndRejections (internal/process/task_queues.js:95:5)
2023-01-01 21:18:19.962Z:     at async Une (/usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js:1822:1370)
2023-01-01 21:18:19.962Z:     at async jne (/usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js:1804:2396)
2023-01-01 21:18:19.962Z:     at async oie (/usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js:1871:2266)
2023-01-01 21:18:19.962Z:     at async qf (/usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js:1871:3239)
2023-01-01 21:18:19.962Z:     at async Mse (/usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js:1995:16211)
2023-01-01 21:18:19.962Z:     at async Lse (/usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js:1995:15965)
2023-01-01 21:18:19.962Z: Stop (1588 ms): Run: /usr/bin/node /usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /var/lib/docker/codespacemount/.persistedshare --container-data-folder .vscode-remote/data/Machine --container-system-data-folder /var/vscode-remote --workspace-folder /var/lib/docker/codespacemount/workspace/Open-Assistant --id-label Type=codespaces --log-level info --log-format json --config /var/lib/docker/codespacemount/workspace/Open-Assistant/.devcontainer/devcontainer.json --override-config /root/.codespaces/shared/merged_devcontainer.json --default-user-env-probe loginInteractiveShell --remove-existing-container --mount type=bind,source=/.codespaces/agent/mount/cache,target=/vscode --skip-post-create --update-remote-user-uid-default never --mount-workspace-git-root false
2023-01-01 21:18:19.962Z: Exit code 1

====================================== ERROR ====================================
2023-01-01 21:18:19.962Z: Failed to create container.
=================================================================================
2023-01-01 21:18:19.964Z: Error: Command failed: /usr/bin/node /usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /var/lib/docker/codespacemount/.persistedshare --container-data-folder .vscode-remote/data/Machine --container-system-data-folder /var/vscode-remote --workspace-folder /var/lib/docker/codespacemount/workspace/Open-Assistant --id-label Type=codespaces --log-level info --log-format json --config /var/lib/docker/codespacemount/workspace/Open-Assistant/.devcontainer/devcontainer.json --override-config /root/.codespaces/shared/merged_devcontainer.json --default-user-env-probe loginInteractiveShell --remove-existing-container --mount type=bind, source=/.codespaces/agent/mount/cache,target=/vscode --skip-post-create --update-remote-user-uid-default never --mount-workspace-git-root false
2023-01-01 21:18:19.967Z: Error Code: 1302

====================================== ERROR ====================================
2023-01-01 21:18:19.970Z: Container creation failed.
=================================================================================
2023-01-01 21:18:19.980Z: 

===================================== WARNING ===================================
2023-01-01 21:18:19.980Z: Creating recovery container.
=================================================================================

=================================================================================
2023-01-01 21:18:20.230Z: Creating container...

=================================================================================
2023-01-01 21:18:27.969Z: Running blocking commands...

=================================================================================
2023-01-01 21:18:30.037Z: Configuring codespace...

=================================================================================
2023-01-01 21:18:30.092Z: Finished configuring codespace.

[bitplane]

I suspect this is due to version 3.7 features being used in docker-compose.yml that are incompatible with older docker-compose but work in recent docker compose (compose is integrated into docker nowadays). I have an offending version of docker-compose installed locally so can probably recreate this, it's probably worth hacking in a downgrade because a few people have hit this same problem.

[yk]

@andrewm4894 could you try the solution here: https://github.com/LAION-AI/Open-Assistant#problems-and-solutions ?

[andrewm4894]

will try

[andrewm4894]

I have tried by adding some initializeCommand options

"initializeCommand": "apt-get update -y && apt-get install docker-compose -y && apt-get install docker-compose-plugin -y"

However i get this error now:

2023-01-01 22:40:49.425Z: E: Unable to locate package docker-compose-plugin
2023-01-01 22:40:49.425Z: Stop (3234 ms): Run: /bin/sh -c apt-get update -y && apt-get install docker-compose -y && apt-get install docker-compose-plugin -y
2023-01-01 22:40:49.425Z: Error: Command failed: /bin/sh -c apt-get update -y && apt-get install docker-compose -y && apt-get install docker-compose-plugin -y
2023-01-01 22:40:49.425Z:     at D7 (/usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js:1865:15)
2023-01-01 22:40:49.425Z:     at processTicksAndRejections (internal/process/task_queues.js:95:5)
2023-01-01 22:40:49.425Z:     at async oie (/usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js:1871:1961)
2023-01-01 22:40:49.425Z:     at async qf (/usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js:1871:3239)
2023-01-01 22:40:49.425Z:     at async Mse (/usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js:1995:16211)
2023-01-01 22:40:49.425Z:     at async Lse (/usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js:1995:15965)
2023-01-01 22:40:49.425Z: Stop (3547 ms): Run: /usr/bin/node /usr/lib/node_modules/@microsoft/vscode-dev-containers-cli/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /var/lib/docker/codespacemount/.persistedshare --container-data-folder .vscode-remote/data/Machine --container-system-data-folder /var/vscode-remote --workspace-folder /var/lib/docker/codespacemount/workspace/Open-Assistant --id-label Type=codespaces --log-level info --log-format json --config /var/lib/docker/codespacemount/workspace/Open-Assistant/.devcontainer/devcontainer.json --override-config /root/.codespaces/shared/merged_devcontainer.json --default-user-env-probe loginInteractiveShell --remove-existing-container --mount type=bind,source=/.codespaces/agent/mount/cache,target=/vscode --skip-post-create --update-remote-user-uid-default never --mount-workspace-git-root false
2023-01-01 22:40:49.425Z: Exit code 1

So its like we'd need to do something different in whatever default image github is using for the codespaces or maybe customize it to be more specific as we might need.

[bitplane]

Found it. Looks like it just needs context: . adding to the build sections that use a Dockerfile. I'll test it and send a pull request.

[bitplane]

Added, #249 :)

[andrewm4894]

cool - will see if that helps for github codespace too

[andrewm4894]

Got the frontend-dev working. Will make a draft PR and we can see from there maybe.

[andrewm4894]

going to close this since initial stuff all done here. Will do any additions or improvements as fresh PR's or issues.