Migrate ArrayJob to use TaskTemplate Config and deprecate ArrayJob proto

[pingsutw]

Signed-off-by: Kevin Su [email protected]

TL;DR

Migrate ArrayJob to use TaskTemplate Config and deprecate ArrayJob proto

Type

• [ ] Bug Fix
• [ ] Feature
• [ ] Plugin

Are all requirements met?

• [ ] Code completed
• [ ] Smoke tested
• [ ] Unit tests added
• [ ] Code documentation added
• [ ] Any pending items have an associated Issue

Complete description

How did you fix the bug, make the feature etc. Link to any design docs etc

Tracking Issue

https://github.com/flyteorg/flyte/issues/1791

Follow-up issue

NA

[codecov[bot]]

Codecov Report

Merging #784 (072b9ef) into master (0a6be27) will increase coverage by 0.00%. The diff coverage is 85.71%.

@@           Coverage Diff           @@
##           master     #784   +/-   ##
=======================================
  Coverage   86.63%   86.64%           
=======================================
  Files         243      242    -1     
  Lines       21452    21425   -27     
  Branches     2396     2397    +1     
=======================================
- Hits        18586    18564   -22     
+ Misses       2445     2440    -5     
  Partials      421      421           

Impacted Files	Coverage Δ
flytekit/core/map_task.py	81.30% <84.61%> (-0.33%)	:arrow_down:
tests/flytekit/unit/models/test_dynamic_job.py	100.00% <100.00%> (ø)
flytekit/models/array_job.py

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 0a6be27...072b9ef. Read the comment docs.

[pingsutw]

There was a spark test that failed - if it fails again, can you take a look?

I created a PR for it https://github.com/flyteorg/flytekit/pull/821

Approving, but this is not a backwards compatible change wrt propeller right? Users will have to update propeller (plugins) in order to use this flytekit code. Can we test everything and then release propeller/plugins first, and make sure to specify what the minimum version of propeller needed is?

yeah, okay