flyte icon indicating copy to clipboard operation
flyte copied to clipboard

[BUG] FlyteDirectory not copied to ContainerTask inputs

Open pryce-turner opened this issue 1 year ago • 14 comments

Describe the bug

When using a ContainerTask with a FlyteDirectory in the inputs, no inputs are actually copied to the running container. For example:

bt = ContainerTask(
    name="basic-test",
    input_data_dir="/var/inputs",
    output_data_dir="/var/outputs",
    inputs=kwtypes(indir=FlyteDirectory),
    outputs=kwtypes(),
    image="ghcr.io/flyteorg/rawcontainers-shell:v2",
    command=[
        "ls",
        "-la",
        "/var/inputs",
    ],
)

@task
def get_dir(dirpath: str) -> FlyteDirectory:
    fd = FlyteDirectory(path=dirpath)
    return fd

@workflow
def wf():
    fd = get_dir(dirpath='s3://my-s3-bucket/cv-in')
    bt(indir=fd)

This workflow returns an empty directory at /var/inputs according to the k8s logs despite there definitely being objects at the specified path.

Expected behavior

I would expect the above workflow to list either /var/inputs/indir with objects copied to indir or perhaps copy them directly i.e. /var/inputs/obj1, /var/inputs/obj2 as they are named in the object store.

Additional context to reproduce

This was run on a local sandbox via pyflyte run --remote basic_test.py wf

Screenshots

No response

Are you sure this issue hasn't been raised already?

  • [X] Yes

Have you read the Code of Conduct?

  • [X] Yes

pryce-turner avatar May 01 '23 21:05 pryce-turner