flytekit icon indicating copy to clipboard operation
flytekit copied to clipboard

Add image builder user defined

Open arbaobao opened this issue 9 months ago • 9 comments

Tracking issue

Why are the changes needed?

In some situations, there isn't internet to build image from pulling uv and micromamba images from the net.

What changes were proposed in this pull request?

I add an ImageBuilderConfig to detect if there is user defined image_builder in config.yaml.

How was this patch tested?

This patch can be tested by adding image_builder to ~/.flyte/config.yaml

image_builder:
  uv_image: arbaobao/flytekit:uv
  micromamba_image: arbaobao/flyte-test-images:micromamba-amd64

if we don't specify the image, the default images are ghcr.io/astral-sh/uv:0.5.1 and mambaorg/micromamba:2.0.3-debian12-slim.

Setup process

  1. Run a flyte sandbox and add image_builder to the config.yaml. Screenshot 2025-03-07 at 3 09 38 PM
  2. if you are using image_spec to run the workflow, it will use the image that you specified to build the task image. Screenshot 2025-03-07 at 3 09 01 PM

Screenshots

Check all the applicable boxes

  • [ ] I updated the documentation accordingly.
  • [ ] All new and existing tests passed.
  • [X] All commits are signed-off.

Related PRs

Docs link

Summary by Bito

This PR adds support for user-defined image builder configuration to enhance flexibility in offline or air-gapped environments. It refactors configuration classes to better separate default from user-defined settings, replacing legacy configuration with new classes including ImageBuilderConfig and updated DataConfig. Docker file templates are revised to allow users to override default images through configuration files.

Unit tests added: False

Estimated effort to review (1-5, lower is better): 2

arbaobao avatar Mar 07 '25 07:03 arbaobao