pyflyte run imperative workflows

[pingsutw]

TL;DR

support running imperative workflow by using pyflyte run. https://github.com/flyteorg/flyte/issues/2766

Type

• [x] Bug Fix
• [ ] Feature
• [ ] Plugin

Are all requirements met?

• [x] Code completed
• [ ] Smoke tested
• [x] Unit tests added
• [ ] Code documentation added
• [ ] Any pending items have an associated Issue

Complete description

How did you fix the bug, make the feature etc. Link to any design docs etc

Tracking Issue

https://github.com/flyteorg/flyte/issues/2706

Follow-up issue

NA

[codecov[bot]]

Codecov Report

Merging #1131 (7067721) into master (73eaad1) will increase coverage by 0.12%. The diff coverage is 80.85%.

@@            Coverage Diff             @@
##           master    #1131      +/-   ##
==========================================
+ Coverage   68.26%   68.38%   +0.12%     
==========================================
  Files         287      288       +1     
  Lines       25829    25963     +134     
  Branches     2885     2899      +14     
==========================================
+ Hits        17633    17756     +123     
- Misses       7719     7728       +9     
- Partials      477      479       +2     

Impacted Files	Coverage Δ
flytekit/remote/remote.py	41.11% <ø> (ø)
flytekit/tools/script_mode.py	49.25% <0.00%> (+0.72%)	:arrow_up:
flytekit/core/tracker.py	48.67% <60.00%> (+0.04%)	:arrow_up:
tests/flytekit/unit/cli/pyflyte/imperative_wf.py	86.36% <86.36%> (ø)
flytekit/clis/sdk_in_container/constants.py	100.00% <100.00%> (ø)
flytekit/clis/sdk_in_container/run.py	83.60% <100.00%> (ø)
tests/flytekit/unit/cli/pyflyte/test_run.py	98.87% <100.00%> (+0.06%)	:arrow_up:
flytekit/extras/tasks/shell.py	77.71% <0.00%> (-0.45%)	:arrow_down:
...ests/flytekit/unit/core/test_structured_dataset.py	98.88% <0.00%> (-0.31%)	:arrow_down:
flytekit/core/data_persistence.py	33.80% <0.00%> (-0.01%)	:arrow_down:
... and 13 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[wild-endeavor]

will take a look tomorrow

[pingsutw]

yes, It's working in the subfolder as well. I've tested it.

can we make it like pyflyte register and pyflyte package in the future?

Not sure what you mean. The reason why it only doesn't work in the pyflyte run is that mod_name in the compress_single_script is generated from extract_task_module, but extract_task_module doesn't work for the imperative workflow. Therefore, we have to use the module name passed from the pyflyte run command.

On the other hand, the pyflyte register uses fast_package, which doesn't require module name. That's why the pyflyte register works now.