flytekit icon indicating copy to clipboard operation
flytekit copied to clipboard

Flytekit: Rename map_task to map, replace min_successes and min_success_ratio with tolerance, rename max_parallelism to concurrency

Open ChihTsungLu opened this issue 11 months ago • 15 comments

Tracking issue

Related to flyteorg/flyte#6139

Why are the changes needed?

The current Flytekit has several areas that could be improved for a better developer experience:

  1. The map_task name is unnecessarily verbose when imported via the recommended import flytekit as fl
  2. The failure tolerance parameters (min_successes and min_success_ratio) are powerful but overly verbose
  3. The max_parallelism parameter naming in workflow and LaunchPlan needs to be aligned with map_task's concurrency parameter

What changes were proposed in this pull request?

  1. Rename map_task to map

    • While this conflicts with Python's built-in map, it's acceptable since we recommend using import flytekit as fl
    • All changes will maintain backwards compatibility
  2. Simplify failure tolerance parameters

    • Deprecate min_successes and min_success_ratio
    • Introduce new tolerance parameter that accepts both float and int types
    • Maintain backwards compatibility with existing parameters
  3. Standardize parallelism parameter

    • Deprecate max_parallelism argument in workflow and LaunchPlan
    • Introduce new concurrency parameter to match map_task's parameter
    • Maintain backwards compatibility with existing parameter

How was this patch tested?

Ran tests with the command: make test

Setup process

Screenshots

Check all the applicable boxes

  • [ ] I updated the documentation accordingly.
  • [x] All new and existing tests passed.
  • [x] All commits are signed-off.

Related PRs

Docs link

Summary by Bito

This PR standardizes API naming by changing 'map_task' to 'map' and 'max_parallelism' to 'concurrency', while consolidating failure tolerance parameters. It transitions from legacy 'agent' nomenclature to a 'connector' paradigm while maintaining backward compatibility with deprecation warnings. The changes enhance execution metric retrieval, asynchronous task handling, resource validation, and file serialization for a more intuitive developer experience. It also improves exception handling and introduces geospatial support.

Unit tests added: False

Estimated effort to review (1-5, lower is better): 5

ChihTsungLu avatar Feb 04 '25 12:02 ChihTsungLu