[WIP]: using cloudpickle to run a task

[madhur-tandon]

TL;DR

Implements a new CloudPickleResolver that should allow us to zip, pickle and ship a task for data persistence. This PR is a work in progress as of writing this comment.

Things that need to be figured out are:

1. How to upload / download (put / get) file from the data persistence store?
   • Hunch is that it is related to the DataPersistence class or perhaps the file_access object.
2. How to use a different task resolver i.e. what is the mechanism to plug in the new resolver and override the default one?

Unit tests are also remaining.

Type

• [ ] Bug Fix
• [x] Feature
• [ ] Plugin

Are all requirements met?

• [x] Code completed
• [ ] Smoke tested
• [ ] Unit tests added
• [ ] Code documentation added
• [ ] Any pending items have an associated Issue

Complete description

Documentation exploring the issue and the approach here: https://docs.google.com/document/d/19Y6mj_tPKRgzfkqQz1UoLHGavMb_rzqjtpeoLUlalNI/edit?usp=sharing

Tracking Issue

https://github.com/flyteorg/flyte/issues/2481

Follow-up issue

NA

[codecov[bot]]

Codecov Report

Merging #1089 (5eaeb73) into master (a968a5a) will increase coverage by 0.14%. The diff coverage is 31.25%.

@@            Coverage Diff             @@
##           master    #1089      +/-   ##
==========================================
+ Coverage   86.67%   86.82%   +0.14%     
==========================================
  Files         269      276       +7     
  Lines       25074    25505     +431     
  Branches     2826     2872      +46     
==========================================
+ Hits        21734    22144     +410     
- Misses       2871     2882      +11     
- Partials      469      479      +10     

Impacted Files	Coverage Δ
flytekit/remote/remote.py	40.77% <22.22%> (-0.35%)	:arrow_down:
flytekit/extras/cloud_pickle_resolver.py	45.83% <33.33%> (ø)
flytekit/types/numpy/__init__.py	100.00% <0.00%> (ø)
flytekit/types/structured/__init__.py	75.00% <0.00%> (ø)
tests/flytekit/unit/cli/pyflyte/test_register.py	100.00% <0.00%> (ø)
...ts/flytekit/unit/extras/pytorch/test_checkpoint.py	91.13% <0.00%> (ø)
flytekit/extras/pytorch/checkpoint.py	84.50% <0.00%> (ø)
tests/flytekit/unit/extras/pytorch/test_native.py	91.66% <0.00%> (ø)
flytekit/extras/pytorch/native.py	86.66% <0.00%> (ø)
... and 11 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update a968a5a...5eaeb73. Read the comment docs.