metaflow
metaflow copied to clipboard
fix: Deterministic foreach task id's for Argo Workflows
changes the Task ID generation for tasks in foreach branches to be deterministic. This is so that the following join steps do not require an enormous list of ID's to be passed in through the input-paths
input Parameter, which has limited the number of foreach tasks we can run on Argo Workflows until now.
Note: Still requires rigorous testing for different split cases to confirm this is ready for prime time.
Closes #1538 properly. Supersedes #1655
Testing[608] @ d7a320bc6c1d4549f950e574a5afbddaef02bc8a
Testing[608] @ d7a320bc6c1d4549f950e574a5afbddaef02bc8a had 1 FAILURE.
I'll check but failure is probably that card test that I need to increase timeout for. There is no change to common code.
Testing[608] @ bcc202639c392ee9aa5386cb1ef6b2c201a84c7f