Buildpacks build fails in Cloud Build due to recent changes

[m11o]

Describe the bug

When using Cloud Build to build a project with Buildpacks, an error occurs during the build process.

Additional context How are you using GCP buildpacks?

• [x] pack and the gcr.io/buildpacks/builder
• [ ] Cloud Functions
• [ ] Cloud Run
• [x] Cloud Build
• [ ] App Engine Standard
• [ ] App Engine Flex
• [ ] Firebase App Hosting

Did this used to work? Yes Was this working before? When did you start noticing these errors?

What language is your project primarily written in? Go

Steps To Reproduce Steps to reproduce the behavior:

Bellow configures in cloudbuild.yml

steps:
  - name: docker.io/buildpacksio/pack
    env:
      - GOOGLE_BUILDABLE=github.com/xxx/yyy/backend/cmd/server
    args:
      - build
      - asia-northeast1-docker.pkg.dev/$PROJECT_ID/cloud-run-source-deploy/yyyy:$COMMIT_SHA
      - --builder
      - gcr.io/buildpacks/builder:v1
      - --network
      - cloudbuild
      - --buildpack
      - from=builder
      - --buildpack
      - datadog/serverless-buildpack:latest
      - --path
      - backend
      - --env
      - GOOGLE_BUILDABLE
      - --verbose
    id: Buildpack
    entrypoint: pack

Expected behavior The build process should complete successfully and produce a working application image.

Actual behavior What actually happened?

The build process fails with the following error:

Running "go build -o /layers/google.go.build/bin/main github.com/xxx/yyy/cmd/server (GOCACHE=/layers/google.go.build/gocache)"
Done "go build -o /layers/google.go.build/bin/main github.com/heyi..." (1m52.018711372s)
=== Utils - Label Image ([email protected]) ===
=== In Datadog Buildpack ===
=== Copying datadog-init ===
=== Detecting command and args ===
all files = [/layers/google.go.build/launch.toml]
[[processes]]
  type = "web"
  command = ["/layers/google.go.build/bin/main"]
  default = true
panic: could not find the command
goroutine 1 [running]:
main.main()
	/Users/maxime.david/dd/crun/buildpack/detect-launch/main.go:98 +0x205
Timer: Builder ran for 3m56.22082942s and ended at 2025-02-19T00:46:10Z
ERROR: failed to build: exit status 2
ERROR: failed to build: executing lifecycle: failed with status code: 51
Finished Step #0 - "Buildpack"
ERROR
ERROR: build step 0 "gcr.io/k8s-skaffold/pack" failed: step exited with non-zero status: 1

If applicable, add screenshots / logs / error messages

Further Investigation Based on our investigation, the build results differ between the following two commits:

361f23d349458eb052a3d6b43f6cf41a5f503037: The build succeeds with this commit.

f8f164dae7621c35acad655090204b9cc2d2f906: The build fails with this commit.

The differences between these commits can be found here: Compare changes

[YashBanka]

I was able to reproduce the error, using a sample app and the given cloudbuild.yaml.

This is using gcr.io/buildpacks/builder:v1 which uses ubuntu18 and is now a deprecated builder image.

[builder] === In Datadog Buildpack ===
[builder] === Copying datadog-init ===
[builder] === Detecting command and args ===
[builder] all files = [/layers/google.nodejs.npm/launch.toml]
[builder] [[processes]]
[builder]   type = "web"
[builder]   command = ["npm"]
[builder]   args = ["run", "start"]
[builder]   default = true
[builder] panic: could not find the command
[builder] 
[builder] goroutine 1 [running]:
[builder] main.main()
[builder] 	/Users/maxime.david/dd/crun/buildpack/detect-launch/main.go:98 +0x205
[builder] Timer: Builder ran for 1m4.220215378s and ended at 2025-02-19T08:29:00Z
[builder] ERROR: failed to build: exit status 2
ERROR: failed to build: executing lifecycle: failed with status code: 51
ERROR
ERROR: build step 0 "docker.io/buildpacksio/pack" failed: step exited with non-zero status: 1

• Got the same error on using gcr.io/buildpacks/builder:latest which is based on ubuntu22.

as you can notice the error comes in the Datadog Buildpack which is being executed as an additional buildpack post the buildpacks detected in gcr.io/buildpacks/builder

detector] 4 of 6 buildpacks participating
[detector] google.nodejs.runtime        1.0.0
[detector] google.nodejs.npm            1.1.0
[detector] google.utils.label-image     0.0.2
[detector] datadog.serverless.container 1.1.0

We have recently migrated our builder from libcnb v1 to libcnb v2

• Tried the same deployment by using a builder image before the libcnb v2 change - gcr.io/buildpacks/builder:deprecated-public-image-ebaa0e7e92ea5f5ef20206095706dad0d17749a9
  • This build passes successfully.

Difference in DataDog buildpack Output

• On failure build

[builder] === In Datadog Buildpack ===
[builder] === Copying datadog-init ===
[builder] === Detecting command and args ===
[builder] all files = [/layers/google.nodejs.npm/launch.toml]
[builder] [[processes]]
[builder]   type = "web"
[builder]   command = ["npm"]
[builder]   args = ["run", "start"]
[builder]   default = true
[builder] panic: could not find the command
[builder] 

• On successful build

[builder] === In Datadog Buildpack ===
[builder] === Copying datadog-init ===
[builder] === Detecting command and args ===
[builder] all files = [/layers/google.nodejs.npm/launch.toml]
[builder] [[processes]]
[builder]   type = "web"
[builder]   command = "npm"
[builder] command = npm
[builder] args = run start
[builder] === Content of datadog_layer ===
[builder] [types]
[builder] launch = true

We have changed the logic on how we set the process and args fields when adding a process. This is a change that is required when we migrate away from libcnb v1 to v2 as mentioned in their release notes (look for "Remove Shell-Specific Logic").

To further confirm this, I removed the step to call datadog/serverless-buildpack:latest and still used gcr.io/buildpacks/builder:latest and the build succeeded.

I think there is a possibility that datadog/serverless-buildpack:latest is not compatible with libcnb v2.