beam icon indicating copy to clipboard operation
beam copied to clipboard

[Task]: Migrate from archived kaniko for building SDK container images

Open RyuSA opened this issue 4 months ago • 0 comments

What needs to happen?

GoogleContainerTools/kaniko was recently publicly archived. https://github.com/GoogleContainerTools/kaniko/pull/3502

I remembered that Apache Beam uses kaniko for its SDK builds, so I checked the repository and confirmed it is still being used in one place for building the SDK container image: https://github.com/apache/beam/blob/v2.65.0/sdks/python/apache_beam/runners/portability/sdk_container_builder.py#L250-L252

While I haven't heard anything about the Kaniko container repository being deleted, it might be a good idea to migrate to a different tool sooner rather than later IF the pipeline is still in use.

For reference, the kaniko community has suggested podman, buildah, or buildKit as potential migration paths. Both are tools for building OCI images and seem like viable alternatives to kaniko. https://github.com/GoogleContainerTools/kaniko/issues/3348

BuildKit provides a build environment within a container image (Note: it requires privileged access). https://hub.docker.com/r/moby/buildkit

Next steps:

  • Confirm if this pipeline is still actively in use.
  • Outline the requirements for the replacement tool.

Issue Priority

Priority: 2 (default / most normal work should be filed as P2)

Issue Components

  • [ ] Component: Python SDK
  • [ ] Component: Java SDK
  • [ ] Component: Go SDK
  • [ ] Component: Typescript SDK
  • [ ] Component: IO connector
  • [ ] Component: Beam YAML
  • [ ] Component: Beam examples
  • [ ] Component: Beam playground
  • [ ] Component: Beam katas
  • [ ] Component: Website
  • [x] Component: Infrastructure
  • [ ] Component: Spark Runner
  • [ ] Component: Flink Runner
  • [ ] Component: Samza Runner
  • [ ] Component: Twister2 Runner
  • [ ] Component: Hazelcast Jet Runner
  • [ ] Component: Google Cloud Dataflow Runner

RyuSA avatar Jun 13 '25 02:06 RyuSA