AutoGPT Architecture-agnostic dev-container patch, now with Redis 😍

Background

The old devcontainer.json was not architecture agnostic. It would fail to build on ARM devices.

Now, with one click, this should automatically start up an Auto-GPT and a Redis container, connect them, and put your VsCode into the development environment of the Auto-GPT container. Regardless of your base operating system or processor architecture.

Changes

Modified the devcontainer.json to point at the docker-compose.yml instead of the dockerfile, and modified the docker-compose.yml and dockerfile from PR #1843 to work while nested inside of the .devcontainer folder, as well as other fixes for dev work such as using a default entry-point and keep alive for TTY.

Documentation

In-code comments.

Test Plan

Open PR in VS-Code -> CMD + Shift + P -> Rebuild and Reopen in Dev Container -> Magic!

PR Quality Checklist

[X] My pull request is atomic and focuses on a single change.
[X] I have thoroughly tested my changes with multiple different prompts.
[X] I have considered potential risks and mitigations for my changes.
[X] I have documented my changes clearly and comprehensively.
[X] I have not snuck in any "extra" small tweaks changes

Apr 24 '23 08:04 johnseth97

Nice man, good work :)

Apr 24 '23 08:04 billysweird

Nice man, good work :)

Thanks! If anyone can test it and throw their errors my way I'd greatly appreciate it.

Apr 24 '23 08:04 johnseth97

Please mention in the PR that the new Dockerfile is an exact copy of the one proposed in #1843: https://github.com/Pwuts/Auto-GPT/blob/262c1edea2333a4a494ed5ff0ecce14e38011931/Dockerfile#L1-L26. Copying without any mention of the source is bad practice in OSS.

Same for the docker-compose.yml, looks heavily inspired.

Apr 24 '23 09:04 Pwuts

I've tested this as you mentioned, but rebuilding without cache and I having the following issue when I try to launch autogpt form the terminal:

vscode ➜ /workspace/Auto-GPT $ python -m autogpt
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/workspace/Auto-GPT/autogpt/__main__.py", line 5, in <module>
    autogpt.cli.main()
  File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 1635, in invoke
    rv = super().invoke(ctx)
  File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/vscode/.local/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/workspace/Auto-GPT/autogpt/cli.py", line 87, in main
    from autogpt.agent.agent import Agent
  File "/workspace/Auto-GPT/autogpt/agent/__init__.py", line 1, in <module>
    from autogpt.agent.agent import Agent
  File "/workspace/Auto-GPT/autogpt/agent/agent.py", line 3, in <module>
    from autogpt.app import execute_command, get_command
  File "/workspace/Auto-GPT/autogpt/app.py", line 5, in <module>
    from autogpt.agent.agent_manager import AgentManager
  File "/workspace/Auto-GPT/autogpt/agent/agent_manager.py", line 7, in <module>
    from autogpt.llm_utils import create_chat_completion
  File "/workspace/Auto-GPT/autogpt/llm_utils.py", line 10, in <module>
    from autogpt.api_manager import api_manager
  File "/workspace/Auto-GPT/autogpt/api_manager.py", line 6, in <module>
    from autogpt.logs import logger
  File "/workspace/Auto-GPT/autogpt/logs.py", line 202, in <module>
    logger = Logger()
  File "/workspace/Auto-GPT/autogpt/config/singleton.py", line 15, in __call__
    cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
  File "/workspace/Auto-GPT/autogpt/logs.py", line 49, in __init__
    self.file_handler = logging.FileHandler(
  File "/usr/local/lib/python3.10/logging/__init__.py", line 1169, in __init__
    StreamHandler.__init__(self, self._open())
  File "/usr/local/lib/python3.10/logging/__init__.py", line 1201, in _open
    return open_func(self.baseFilename, self.mode,
PermissionError: [Errno 13] Permission denied: '/workspace/Auto-GPT/logs/activity.log'

vscode ➜ /workspace/Auto-GPT $ id
uid=1001(vscode) gid=1001(vscode) groups=1001(vscode),998(pipx),999(nvm)

vscode ➜ /workspace/Auto-GPT $ ls -al  /workspace/Auto-GPT/logs/activity.log
-rw-r--r-- 1 appuser appuser 344300 Apr 24 07:07 /workspace/Auto-GPT/logs/activity.log

Apr 24 '23 10:04 hdkiller

Try doing it again, I just made a change to the devcontainer.json that should have fixed permissions, and was able to rebuild it without cache and from complete scratch without any permissions issues on my M1 Mac. Two other users on Intel and AMD windows machines were able to do the same as well :3

Apr 24 '23 10:04 johnseth97

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

Apr 24 '23 13:04 github-actions[bot]

For some reason I still get the permission issue:

vscode ➜ /workspace/Auto-GPT $ ls -al /workspace/Auto-GPT/logs/
total 356
drwxr-xr-x  2 appuser appuser   4096 Apr 15 06:28 .
drwxr-xr-x 13 appuser appuser   4096 Apr 24 15:03 ..
-rw-r--r--  1 appuser appuser 344300 Apr 24 07:07 activity.log
-rw-r--r--  1 appuser appuser    378 Apr 20 20:28 error.log
vscode ➜ /workspace/Auto-GPT $ id
uid=6942(vscode) gid=6942(vscode) groups=6942(vscode),998(pipx),999(nvm)
vscode ➜ /workspace/Auto-GPT $

Apr 24 '23 15:04 hdkiller

I've tested the master branch and that's working properly, I don't have the permission issues, but if I do the same with this branch checked out then I do.

Apr 24 '23 16:04 hdkiller

I've tested the master branch and that's working properly, I don't have the permission issues, but if I do the same with this branch checked out then I do.

And you have nuked your autogpt and redis images from orbit before reopening the PR in VSCode and choosing "Rebuild and reopen in container?"

Apr 24 '23 20:04 johnseth97

I believe that you might be having issues with not properly clearing all associated container files and docker images before building the dev container.

this is the output from me running the ID command.

vscode ➜ /workspace/Auto-GPT (devcontainer-fix) $ id uid=6942(vscode) gid=6942(vscode) groups=6942(vscode),998(pipx),999(nvm)

Notice that my uid/gid matches the declerations found in lines 10 and 11 of the devcontainer.json file, whereas yours match the older declarations from the master branch. Said gid/uid would be bumping up against the appuser profile, causing permissions errors.

My build process on my M1 MacBook Pro is as follows:

Loading PR #3102 in GitHub desktop -> Deleting all docker instances(Auto-GPT_devcontainer(redis-1 + auto-gpt-1)) -> Opening the PR in VSCode from Github Desktop(CMD/CTRL + SHIFT + ) -> Rebuilding without cache

Apr 24 '23 20:04 johnseth97

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

Apr 24 '23 23:04 github-actions[bot]

Codecov Report

Patch and project coverage have no change.

Comparison is base (cfc7817) 56.95% compared to head (85d6057) 56.95%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3102   +/-   ##
=======================================
  Coverage   56.95%   56.95%           
=======================================
  Files          67       67           
  Lines        3048     3048           
  Branches      509      509           
=======================================
  Hits         1736     1736           
  Misses       1174     1174           
  Partials      138      138

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.