Make pyflyte run 4x faster

[pingsutw]

Tracking issue

NA

Why are the changes needed?

pyflyte run takes ~40 seconds to serialize and register a workflow.

What changes were proposed in this pull request?

Use multi-thread to register tasks, which can make pyflyte run 4x faster

Setup process

pyflyte run --remote integration_tests.py wf

Screenshots

Before: ~40s

After: <10s

Check all the applicable boxes

• [x] I updated the documentation accordingly.
• [x] All new and existing tests passed.
• [x] All commits are signed-off.

Related PRs

NA

Docs link

NA

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 88.88889% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 83.17%. Comparing base (d61e79e) to head (fe0037d). Report is 1 commits behind head on master.

Files	Patch %	Lines
flytekit/remote/remote.py	88.88%	1 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2267      +/-   ##
==========================================
- Coverage   83.31%   83.17%   -0.15%     
==========================================
  Files         309      303       -6     
  Lines       24050    23742     -308     
  Branches     3491     3491              
==========================================
- Hits        20038    19748     -290     
+ Misses       3386     3367      -19     
- Partials      626      627       +1     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[pingsutw]

fixed in https://github.com/flyteorg/flytekit/pull/2276