initializeCommand not executed in envbuilder

[creeram]

I have the below configuration on the devcontainer.json file and I want to run some scripts before dockerbuild starts but the "initializeCommand" is not executed when the container is started.

{
	"name": "poc",
	"initializeCommand": "bash .devcontainer/update.sh",
	"build": {
		"dockerfile": "Dockerfile",
	},
	"service": "app",
	
	

	"remoteUser": "node",
	"features": {
		"ghcr.io/devcontainers/features/git:1": {
			"version": "os-provided"
		}
	}
}

There are not logs for this as well i can only see logs starting from building image

envbuilder - Build development environments from repositories in a container

#1: 📦 Cloning https://github.com/****/**** to /workspaces/poc

#1: 📦 The repository already exists! [4.421296ms]
#2: Deleting filesystem...
#2: 🏗️ Building image...

[johnstcn]

This looks like a functionality gap in Envbuilder. It currently supports the following lifecycle script hooks:

• onCreateCommand
• updateContentCommand
• postCreateCommand
• postStartCommand

We're currently missing support for:

• initializeCommand (Question: the "host machine" in this case will be the envbuilder container)
• postAttachCommand
• waitFor

ref: https://containers.dev/implementors/json_reference/#lifecycle-scripts

[mafredri]

before dockerbuild starts but the "initializeCommand" is not executed when the container is started.

@creeram could you expand on what you're expecting to happen? To me it sounds like before dockerbuild starts means this script should run before envbuilder. However, executed when the container is started sounds like it should run after envbuilder. If it's the latter, I believe onCreateCommand or updateContentCommand would better suit your needs.

@johnstcn I think finding a good fit for initializeCommand within envbuilder is hard. As I see it we can either run it before envbuilder builds the image or after but just before onCreateCommand. If we choose one method over the other, though, it will probably mismatch user expectations 50% of the time. If we decide to support it, we may have to make it configurable.

[creeram]

@mafredri I expect the script to be executed before the Docker build starts, That feature is already available in vs code dev container but not in Envbuilder.

[mafredri]

@creeram would you mind sharing what .devcontainer/update.sh does? The reason I ask because I want to ensure that if we do add support for what you're requesting, it will actually work.

Right now I don't think it will behave as you expect, if I'm understanding you correctly that is. Let me explain.

As an example let's say .devcontainer/update.sh is very simple:

#!/bin/sh
# Make sure repo is up-to-date.
git pull

Then we run envbuilder like this:

docker run -it --rm \
  -e ENVBUILDER_GIT_URL=https://github.com/my/repo-with-initialize-command.git
  -e ENVBUILDER_INIT_COMMAND=/bin/bash
  ghcr.io/coder/envbuilder:latest

What will happen is that Envbuilder will:

1. Clone the repository
2. Parse the devcontainer.json
3. Try to run initializeCommand .devcontainer/update.sh
4. (!) Build the Dockerfile

At step 3 we run into an error because neither /bin/sh nor git binaries exist on the ghcr.io/coder/envbuilder image. Those may become available after the build, depending on the contents of the Dockerfile.

[creeram]

@mafredri The envbuilder clones the repository the first time it runs, and on subsequent restarts, it skips the clone because the repo is already cloned in the persistent volume. Let’s assume I changed the Dockerfile in the remote repo and want to pull the updated content before the Docker build starts. However, ENVBUILDER_INIT_COMMAND=/bin/bash is currently executed after the Docker build. I want repoupdate.sh to run before the Docker build begins, so it pulls the latest changes in the Dockerfile and uses the updated version for the build.

The steps you mentioned:

What will happen is that Envbuilder will:

Clone the repository
Parse the devcontainer.json
Try to run initializeCommand .devcontainer/update.sh
(!) Build the Dockerfile

Envbuilder skips the Try to run initializeCommand .devcontainer/update.sh part. but when I try the same devcontainer.json file in vscode devcontainer the initializeCommand works as expected.

[mafredri]

I want repoupdate.sh to run before the Docker build begins, so it pulls the latest changes in the Dockerfile and uses the updated version for the build.

This tracks with what I thought you were looking for, thanks for clarifying.

Envbuilder skips the Try to run initializeCommand .devcontainer/update.sh part. but when I try the same devcontainer.json file in vscode devcontainer the initializeCommand works as expected.

Yes, that's what envbuilder does now. My example above was a hypothetical of what will happen if we support initializeCommand (with the behavior you're expecting), i.e. it won't work for.

It works in VS Code because there are two explicit environments, the host machine (runs initializeCommand) and a separate container (built afterwards). In envbuilder the host machine and the container are usually one and the same.

If envbuilder did support initializeCommand (behaving as discussed), the only way to actually execute that script is to create a custom image that has all the necessary tools (shell, git, etc). That's not a very good user experience.

Although I'm focusing on the problems here, it's not because we don't want to support initializeCommand, it's because we don't have a good idea for how to do that with envbuilder right now.

Side note: One way to achieve what you're looking for today is to use the ENVBUILDER_REMOTE_REPO_BUILD_MODE=true option. It will always use the latest commit from the repo to build the image.

We're also looking into implementing an option that will allow you to pull changes even if the repo is already cloned (e.g. #281). That's another way that the behavior you're looking for can be reproduced in envbuilder.

Unfortunately, neither is the same as initializeCommand and specific to envbuilder.

[creeram]

@mafredri Rather than pulling all changes in the repository, the goal is to pull only changes from the .devcontainer folder. This is what I'm trying to achieve using the update.sh script as follows:

#!/bin/bash

# Define the target directory
TARGET_DIR="/workspaces/demo"

# Check if the target directory exists
if [ -d "$TARGET_DIR" ]; then
  cd "$TARGET_DIR"
  
  # Configure safe directory
  git config --global --add safe.directory "$TARGET_DIR"
  
  # Initialize sparse checkout
  git sparse-checkout init --cone
  
  # Set sparse checkout for .devcontainer
  git sparse-checkout set .devcontainer
  
  # Pull changes from the specified branch
  git pull origin master
  
  # Disable sparse checkout
  git sparse-checkout disable
else
  echo "$TARGET_DIR folder does not exist"
fi