devpod icon indicating copy to clipboard operation
devpod copied to clipboard
verified publisher icon loft-sh

devpod reset or fresh start works, but rebuild fails

Open ysaakpr opened this issue 1 year ago • 9 comments

  • Devpod Version: v0.5.18
  • Operating System: mac
  • ARCH of the OS: ARM64
  • AWS Provider - Ec2 - ubuntu ami

I am using devpod with docker compose for development setup. i am using preInstall hooks to initialize the env variables and creates .env file in the directory where the docker compose file and devcontainer.json stays, which is in the folder named .devcontainer in the root of git repo. The docker compose also have reference to the same .env file in the services which it references with env_file keyword.

I am able to do a fresh setup with the devpod(Create new workspace, as well as reset), it works and starts the vscode editor, successfully. Also if i have to reset, it does it well. The issue is while i want to rebuild. I get debug logs saying the expected env variables are not set, and because of that the containers are failing.

Devpod rebuild logs

[10:36:45] info Workspace discover-3 already exists
[10:36:46] info Starting machine 'devpod-shared-aws-7gl0j30c-1a521'...
[10:36:48] info Successfully started 'devpod-shared-aws-7gl0j30c-1a521'
[10:36:48] info Creating devcontainer...
[10:36:57] info Waiting for devpod agent to come up...
[10:37:04] info Rebuiling without resetting a git based workspace, keeping old content folder
[10:37:04] info Configuring docker daemon ...
[10:37:04] info Running initializeCommand from devcontainer.json: 'sh -c .devcontainer/gen-docker-compose-workspace-env.sh --container-workspace-folder '/workspace' --local-workspace-folder '/home/devpod/.devpod/agent/contexts/default/workspaces/discover-3/content''
[10:37:04] info .env file exists, skips regeneration
[10:37:04] info time="2024-12-24T05:07:04Z" level=warning msg="The \"TEMPORAL_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:04] info time="2024-12-24T05:07:04Z" level=warning msg="The \"TEMPORAL_ADMINTOOLS_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:04] info time="2024-12-24T05:07:04Z" level=warning msg="The \"TEMPORAL_UI_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:04] info time="2024-12-24T05:07:04Z" level=warning msg="The \"POSTGRESQL_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:05] info time="2024-12-24T05:07:05Z" level=warning msg="The \"POSTGRESQL_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:05] info time="2024-12-24T05:07:05Z" level=warning msg="The \"TEMPORAL_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:05] info time="2024-12-24T05:07:05Z" level=warning msg="The \"TEMPORAL_ADMINTOOLS_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:05] info time="2024-12-24T05:07:05Z" level=warning msg="The \"TEMPORAL_UI_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:09] info time="2024-12-24T05:07:09Z" level=warning msg="The \"TEMPORAL_UI_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:09] info time="2024-12-24T05:07:09Z" level=warning msg="The \"POSTGRESQL_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:09] info devcontainer up: start container: docker-compose run: exit status 1
[10:37:09] info error parsing workspace info: rerun as root: exit status 1
[10:37:09] info time="2024-12-24T05:07:09Z" level=warning msg="The \"TEMPORAL_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:10] error Try enabling Debug mode under Settings to see a more verbose output
[10:37:10] fatal run agent command: Process exited with status 1
[10:36:45] info Workspace discover-3 already exists
[10:36:46] info Starting machine 'devpod-shared-aws-7gl0j30c-1a521'...
[10:36:48] info Successfully started 'devpod-shared-aws-7gl0j30c-1a521'
[10:36:48] info Creating devcontainer...
[10:36:57] info Waiting for devpod agent to come up...
[10:37:04] info Rebuiling without resetting a git based workspace, keeping old content folder
[10:37:04] info Configuring docker daemon ...
[10:37:04] info Running initializeCommand from devcontainer.json: 'sh -c .devcontainer/gen-docker-compose-workspace-env.sh --container-workspace-folder '/workspace' --local-workspace-folder '/home/devpod/.devpod/agent/contexts/default/workspaces/discover-3/content''
[10:37:04] info .env file exists, skips regeneration
[10:37:04] info time="2024-12-24T05:07:04Z" level=warning msg="The \"TEMPORAL_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:04] info time="2024-12-24T05:07:04Z" level=warning msg="The \"TEMPORAL_ADMINTOOLS_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:04] info time="2024-12-24T05:07:04Z" level=warning msg="The \"TEMPORAL_UI_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:04] info time="2024-12-24T05:07:04Z" level=warning msg="The \"POSTGRESQL_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:05] info time="2024-12-24T05:07:05Z" level=warning msg="The \"POSTGRESQL_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:05] info time="2024-12-24T05:07:05Z" level=warning msg="The \"TEMPORAL_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:05] info time="2024-12-24T05:07:05Z" level=warning msg="The \"TEMPORAL_ADMINTOOLS_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:05] info time="2024-12-24T05:07:05Z" level=warning msg="The \"TEMPORAL_UI_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:09] info time="2024-12-24T05:07:09Z" level=warning msg="The \"TEMPORAL_UI_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:09] info time="2024-12-24T05:07:09Z" level=warning msg="The \"POSTGRESQL_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:09] info devcontainer up: start container: docker-compose run: exit status 1
[10:37:09] info error parsing workspace info: rerun as root: exit status 1
[10:37:09] info time="2024-12-24T05:07:09Z" level=warning msg="The \"TEMPORAL_VERSION\" variable is not set. Defaulting to a blank string."
[10:37:10] error Try enabling Debug mode under Settings to see a more verbose output
[10:37:10] fatal run agent command: Process exited with status 1
Screenshot 2024-12-24 at 10 40 10 AM

The .env file is in the same place as the docker-composeand the devcontainer.json

My devcontainer.json:

// For format details, see https://aka.ms/devcontainer.json.
// Refer https://github.com/h4l/dev-container-docker-compose-volume-or-bind
{
    "name": "Rev Backent Dev Container",
    "dockerComposeFile": [
        "docker-compose.workspace.yaml"
        // ,"docker-compose.temporal.yaml"
    ],
    "forwardPorts": ["mongodb:27017", "db:5432", "temporal-ui:8080","mongo-express:8081"],
    "service": "app",
    "workspaceFolder": "/workspace",
    // Generate .devcontainer/.env containing WORKSPACE_* envars for docker-compose.yml
    "initializeCommand": ".devcontainer/gen-docker-compose-workspace-env.sh --container-workspace-folder '${containerWorkspaceFolder}' --local-workspace-folder '${localWorkspaceFolder}'",

    // General properties unrelated to the example follow...
    "remoteUser": "vscode",
    "postStartCommand": "bash .devcontainer/setup.sh",
    "customizations":{
        "vscode":{
            "extensions": [
                "editorconfig.editorconfig",
                "streetsidesoftware.code-spell-checker",
                "timonwong.shellcheck",
                "ms-vscode-remote.remote-containers",
                "nguyenngoclong.terminal-keeper",
                "charliermarsh.ruff"
            ]
        }
    }
}

ysaakpr avatar Dec 24 '24 05:12 ysaakpr

May be similar : https://github.com/loft-sh/devpod/issues/1279

ysaakpr avatar Dec 24 '24 05:12 ysaakpr

Hi @ysaakpr thanks for raising your issue! Does this happen in the CLI as well as UI?

bkneis avatar Jan 06 '25 10:01 bkneis

I have tested this only on UI - let me know if you want me to try this on the cli as well

ysaakpr avatar Jan 06 '25 10:01 ysaakpr

yes please :pray: Just so we know if it is the same as #1279 or if it is fundamental to the rebuild pipeline

bkneis avatar Jan 06 '25 10:01 bkneis

@bkneis not sure what exactly the cli command equivalent to rebuild and reset option in the UI

ysaakpr avatar Jan 07 '25 05:01 ysaakpr

Ran devpod up discover --recreate - believe thats the equivalent command for desktop Rebuild - as per doc

11:16:18 info Workspace discover already exists
11:16:19 info Creating devcontainer...
11:16:28 info Rebuiling without resetting a git based workspace, keeping old content folder
11:16:28 info Running initializeCommand from devcontainer.json: 'sh -c .devcontainer/gen-docker-compose-workspace-env.sh --container-workspace-folder '/workspace' --local-workspace-folder '/home/devpod/.devpod/agent/contexts/default/workspaces/discover/content''
11:16:28 info .env file exists, skips regeneration
11:16:28 info time="2025-01-08T05:46:28Z" level=warning msg="The \"TEMPORAL_UI_VERSION\" variable is not set. Defaulting to a blank string."
11:16:28 info time="2025-01-08T05:46:28Z" level=warning msg="The \"POSTGRESQL_VERSION\" variable is not set. Defaulting to a blank string."
11:16:28 info time="2025-01-08T05:46:28Z" level=warning msg="The \"TEMPORAL_VERSION\" variable is not set. Defaulting to a blank string."
11:16:28 info time="2025-01-08T05:46:28Z" level=warning msg="The \"TEMPORAL_ADMINTOOLS_VERSION\" variable is not set. Defaulting to a blank string."
11:16:28 info time="2025-01-08T05:46:28Z" level=warning msg="The \"POSTGRESQL_VERSION\" variable is not set. Defaulting to a blank string."
11:16:28 info time="2025-01-08T05:46:28Z" level=warning msg="The \"TEMPORAL_VERSION\" variable is not set. Defaulting to a blank string."
11:16:28 info time="2025-01-08T05:46:28Z" level=warning msg="The \"TEMPORAL_ADMINTOOLS_VERSION\" variable is not set. Defaulting to a blank string."
11:16:28 info time="2025-01-08T05:46:28Z" level=warning msg="The \"TEMPORAL_UI_VERSION\" variable is not set. Defaulting to a blank string."
11:16:28 info time="2025-01-08T05:46:29Z" level=warning msg="The \"TEMPORAL_ADMINTOOLS_VERSION\" variable is not set. Defaulting to a blank string."
11:16:28 info time="2025-01-08T05:46:29Z" level=warning msg="The \"TEMPORAL_UI_VERSION\" variable is not set. Defaulting to a blank string."
11:16:29 info devcontainer up: start container: docker-compose run: exit status 1
11:16:29 info time="2025-01-08T05:46:29Z" level=warning msg="The \"POSTGRESQL_VERSION\" variable is not set. Defaulting to a blank string."
11:16:29 info error parsing workspace info: rerun as root: exit status 1
11:16:29 error Try using the --debug flag to see a more verbose output
11:16:29 fatal run agent command: Process exited with status 1

ysaakpr avatar Jan 08 '25 05:01 ysaakpr

This issue is stale because it has been open for 60 days with no activity.

github-actions[bot] avatar Mar 10 '25 01:03 github-actions[bot]

not stale, an OSS user reported this in slack

bkneis avatar Mar 31 '25 07:03 bkneis

Hi, I’m the user @bkneis mentioned - just adding my case here as well.

I’m using devpod with the Kubernetes provider and a .devcontainer/Dockerfile setup. Whenever I update the Dockerfile (e.g. adding packages or changing the base image, etc), the “Rebuild Workspace” does not pick up those changes. The only way to apply updates is to completely delete and recreate the workspace, which results in losing all workspace data – including large build artifacts.

ccemmanouilidis avatar Mar 31 '25 07:03 ccemmanouilidis

This issue is stale because it has been open for 60 days with no activity.

github-actions[bot] avatar Jun 01 '25 02:06 github-actions[bot]

This issue was closed because it has been inactive for 30 days since being marked as stale.

github-actions[bot] avatar Jul 02 '25 02:07 github-actions[bot]