flytekit icon indicating copy to clipboard operation
flytekit copied to clipboard

refactor(core): Improve task module extraction logic

Open pingsutw opened this issue 1 year ago • 1 comments

Tracking issue

NA

Why are the changes needed?

Cannot run a task registered by flytekit remote

What changes were proposed in this pull request?

  • Update task module extraction logic
  • Update remote.register task interface. make SerializationSettings optional

Before:

remote = FlyteRemote(...)
remote.register_task(
        entity=t1,
        serialization_settings=SerializationSettings(
            image_config=ImageConfig.auto_default_image(),
            source_root=Path(".").absolute().__str__(),
        ),
        version="v1",
    )

After:

remote = FlyteRemote(...)
remote.register_task(
        entity=t1,
        version="v1",
    )

How was this patch tested?

remote cluster

python functional_tests.py 

Setup process

from datetime import datetime

from flytekit import task, ImageSpec
from flytekit.configuration import Config, SerializationSettings, ImageConfig
from flytekit.remote import FlyteRemote
from flytekit.tools.script_mode import hash_file

image_spec = ImageSpec(
    name="flyte-conformance",
    registry="ghcr.io/unionai",
    packages=["pandas"],
)


@task(container_image=image_spec)
def t1(x: int) -> datetime:
    return datetime.now()


def test_cache_override():
    remote = FlyteRemote(
        config=Config.auto("/Users/kevin/.flyte/config-dogfood-gcp.yaml"),
        default_project="flytesnacks",
        default_domain="development",
    )
    _, digest, _ = hash_file(__file__)
    flyte_task = remote.register_task(
        entity=t1,
        version=digest,
    )

    exe = remote.execute(entity=flyte_task, inputs={"x": 3}, wait=False)

Screenshots

image
  • [x] I updated the documentation accordingly.
  • [x] All new and existing tests passed.
  • [x] All commits are signed-off.

Related PRs

NA

Docs link

NA

pingsutw avatar Mar 20 '24 23:03 pingsutw

Codecov Report

Attention: Patch coverage is 41.66667% with 7 lines in your changes are missing coverage. Please review.

Project coverage is 83.09%. Comparing base (6a383cd) to head (6d639c5). Report is 1 commits behind head on master.

Files Patch % Lines
flytekit/remote/remote.py 33.33% 4 Missing :warning:
flytekit/core/tracker.py 0.00% 3 Missing :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2290      +/-   ##
==========================================
- Coverage   83.12%   83.09%   -0.03%     
==========================================
  Files         324      324              
  Lines       24779    24789      +10     
  Branches     3523     3524       +1     
==========================================
+ Hits        20597    20599       +2     
- Misses       3558     3565       +7     
- Partials      624      625       +1     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Mar 20 '24 23:03 codecov[bot]