flytekit
flytekit copied to clipboard
Flytekit: Rename map_task to map, replace min_successes and min_success_ratio with tolerance, rename max_parallelism to concurrency
Tracking issue
Related to flyteorg/flyte#6139
Why are the changes needed?
The current Flytekit has several areas that could be improved for a better developer experience:
- The
map_taskname is unnecessarily verbose when imported via the recommendedimport flytekit as fl - The failure tolerance parameters (
min_successesandmin_success_ratio) are powerful but overly verbose - The max_parallelism parameter naming in workflow and LaunchPlan needs to be aligned with
map_task'sconcurrencyparameter
What changes were proposed in this pull request?
-
Rename
map_tasktomap- While this conflicts with Python's built-in
map, it's acceptable since we recommend usingimport flytekit as fl - All changes will maintain backwards compatibility
- While this conflicts with Python's built-in
-
Simplify failure tolerance parameters
- Deprecate
min_successesandmin_success_ratio - Introduce new
toleranceparameter that accepts bothfloatandinttypes - Maintain backwards compatibility with existing parameters
- Deprecate
-
Standardize parallelism parameter
- Deprecate
max_parallelismargument in workflow and LaunchPlan - Introduce new
concurrencyparameter to matchmap_task's parameter - Maintain backwards compatibility with existing parameter
- Deprecate
How was this patch tested?
Ran tests with the command: make test
Setup process
Screenshots
Check all the applicable boxes
- [ ] I updated the documentation accordingly.
- [x] All new and existing tests passed.
- [x] All commits are signed-off.
Related PRs
Docs link
Summary by Bito
This PR standardizes API naming by changing 'map_task' to 'map' and 'max_parallelism' to 'concurrency', while consolidating failure tolerance parameters. It transitions from legacy 'agent' nomenclature to a 'connector' paradigm while maintaining backward compatibility with deprecation warnings. The changes enhance execution metric retrieval, asynchronous task handling, resource validation, and file serialization for a more intuitive developer experience. It also improves exception handling and introduces geospatial support.Unit tests added: False
Estimated effort to review (1-5, lower is better): 5