flytekit icon indicating copy to clipboard operation
flytekit copied to clipboard

[feature/flytekit] Smarter copy handling with image spec

Open taieeuu opened this issue 11 months ago • 6 comments

Tracking issue

Related to issue#6094

Why are the changes needed?

What changes were proposed in this pull request?

If you use the source_root parameter in imagespec, the command execution will use pyflyte-fast-execute; otherwise, it will use pyflyte-execute.

image image

How was this patch tested?

the unit test pass.

Screenshots

Check all the applicable boxes

  • [ ] I updated the documentation accordingly.
  • [x] All new and existing tests passed.
  • [x] All commits are signed-off.

Summary by Bito

Implementation of enhanced container image handling in flytekit with improved validation logic and execution command determination. The system includes Kubernetes StatefulSet Data Service features, custom Python executable support, and smarter ImageSpec configurations. Changes focus on proper handling of container images, None value checks, and logic refinement for pyflyte-execute selection based on source_root parameters. The implementation includes improved error handling and CLI monitoring capabilities with configurable chunk sizes for data persistence.

Unit tests added: True

Estimated effort to review (1-5, lower is better): 5

taieeuu avatar Jan 18 '25 18:01 taieeuu