Feature/array node workflow parallelism
Tracking issue
https://github.com/flyteorg/flyte/issues/3924
Why are the changes needed?
Want to add support for array node map tasks to utilize threads under the parent workflow's set parallelism. Decrease confusion on default/expected map task parallelism
What changes were proposed in this pull request?
Set default parallelism to -1 to indicate
How was this patch tested?
Setup process
Screenshots
Check all the applicable boxes
- [x] I updated the documentation accordingly.
- [x] All new and existing tests passed.
- [x] All commits are signed-off.
Related PRs
Needs https://github.com/flyteorg/flyte/pull/5062 to be merged first
Docs link
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 53.57%. Comparing base (
f4d894a) to head (451fd03). Report is 22 commits behind head on master.
:exclamation: Current head 451fd03 differs from pull request most recent head 3f596c2. Consider uploading reports for the commit 3f596c2 to get more accurate results
Additional details and impacted files
@@ Coverage Diff @@
## master #2268 +/- ##
===========================================
- Coverage 83.09% 53.57% -29.52%
===========================================
Files 324 180 -144
Lines 24860 18133 -6727
Branches 3761 3547 -214
===========================================
- Hits 20657 9715 -10942
- Misses 3582 7991 +4409
+ Partials 621 427 -194
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Can you fix the unit test failure? The error:
FAILED tests/flytekit/unit/core/test_array_node_map_task.py::test_inputs_outputs_length - AssertionError: assert equals failed
'tests.flytekit.unit.core.test_ 'tests.flytekit.unit.core.test_
array_node_map_task.map_many_in array_node_map_task.map_many_in
puts_-6b^3bd0^353da5d^e6e8-4d79-82921 puts_b^f51001578d0^ae197a5+2c0af0a
-ead2b3-arraynode' +99d^d89-arraynode'
Can you fix the unit test failure? The error:
FAILED tests/flytekit/unit/core/test_array_node_map_task.py::test_inputs_outputs_length - AssertionError: assert equals failed 'tests.flytekit.unit.core.test_ 'tests.flytekit.unit.core.test_ array_node_map_task.map_many_in array_node_map_task.map_many_in puts_-6�b^3b�d0^353d�a5d^e6e�8-4d7�9-82921� puts_b^f51001578�d0^ae197�a5+2c0af0a� -ead2b3�-arraynode' +99�d^d�89-arraynode'
@eapolinario oof on my end. Just updated - will re-request your review after the builds pass