[frontend] Error when downloading Artifact that is Folder via UI

[rustam-ashurov-mcx]

Environment

• How did you deploy Kubeflow Pipelines (KFP)? AWS EKS k8s cluster + vanilla installation with multi-user auth
• KFP version: 1.7

Steps to reproduce

In Kubeflow pipelines V2 when an artifact is a folder it is not possible to download the Artifact via UI. Data is passed between components in a pipeline fine. When we are using V1 syntax folder (and non folders) are packed in tar archive and everything works in the same environment.

Expected result

Artifact is packed automatically in archive or just downloadable as a folder when it is a folder originally

Materials and Reference

There is closed issues on github, but I think it was a fix only for pipelines V1: https://github.com/kubeflow/pipelines/issues/3667 https://github.com/kubeflow/pipelines/issues/4799

Here is ho it looks on UI when we are trying to download an artifact that was a folder:

---

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

[Linchin]

Would you please share which code you are using? Thank you.

[rustam-ashurov-mcx]

For sure. We started with the declaration of pipelines via DSL syntax. Here I tried to minimize the number of steps, just creating a TensorFlow model and saving it as an Artifact:

import kfp
import kfp.dsl as dsl
from kfp.v2.dsl import (
    component,
    Input,
    Output,
    Dataset,
    Metrics,
    Model,
)

@component(base_image='tensorflow/tensorflow', packages_to_install=['pandas'])
def create_model(saved_model: Output[Model]):
    
    import pandas as pd
    import tensorflow as tf
    
    csv_file = tf.keras.utils.get_file('heart.csv', 'https://storage.googleapis.com/download.tensorflow.org/data/heart.csv')
    df = pd.read_csv(csv_file)
    
    train_features_data = df[['age', 'thalach', 'trestbps',  'chol', 'oldpeak']]
    
    normalizer = tf.keras.layers.Normalization(axis=-1)
    normalizer.adapt(train_features_data)
    
    model = tf.keras.Sequential([
        normalizer,
        tf.keras.layers.Dense(10, activation='relu'),
        tf.keras.layers.Dense(10, activation='relu'),
        tf.keras.layers.Dense(1)
    ])

    model.compile(optimizer='adam',
                loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
                metrics=['accuracy'])
    
    print(model.summary())
    
    model.save(saved_model.path)

@dsl.pipeline(name="artifacts-example", description="artifacts-example")
def example_pipeline():
    create_model_task = create_model()   
    
client = kfp.Client()
print(client.list_experiments())

client.create_experiment(name="client_test_1", description="client_test_desc", namespace="my-team")
client.create_run_from_pipeline_func(
    example_pipeline,
    None, run_name="test_run",
    experiment_name="client_test_1",
    namespace="my-team",
    enable_caching=False,
    mode=kfp.dsl.PipelineExecutionMode.V2_COMPATIBLE) 

As a result, the pipeline runs fine and green, but the artifact (that is a folder) is not available on UI:

In case of clicking on minio link just a blank page is opened in the browser

In case of clicking on View All button: 500 error http response is returned with a message: Failed to get object in bucket mlpipeline at path v2/artifacts/pipeline/artifacts-example/ea79a102-8eff-4289-ad3a-99af24960c3c/create-model/saved_model: S3Error: The specified key does not exist. (I'm using default vanilla build with minio under the hood)

When I use V1 DSL syntax all artifacts (single file or folders are packed as tar archives automatically), so I guess for me (or not only for me) V2 pipelines just store artifacts (files and folders) as it is. Probably it is fine and even cool, plus from what I see 'folder' artifacts are passed fine between pipeline staps.

But still it fails to download it on UI by some reason

[rustam-ashurov-mcx]

Another thing that maybe makes sense to mention, and maybe could help is that on the Artifacts UI page I see that all my Artifacts generated by all V2 syntax pipelines are under 'uknown' pipeline by some reason:

This is super confusing. The same time artifacts generated by V1 pipelines are there and look fine:

[rimolive]

Closing this issue. No activity for more than a year.

/close

[google-oss-prow[bot]]

@rimolive: Closing this issue.

In response to this:

Closing this issue. No activity for more than a year.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[zachgarner]

Hey everyone, this is still very much an issue and is reproducible with the instructions above

/reopen

[google-oss-prow[bot]]

@zachgarner: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

Hey everyone, this is still very much an issue and is reproducible with the instructions above

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[boarder7395]

Tested with latest master branch and this is still an issue. /reopen

[google-oss-prow[bot]]

@boarder7395: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

Tested with latest master branch and this is still an issue. /reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[HumairAK]

/reopen

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.

[hbelmiro]

/reopen

[google-oss-prow[bot]]

@hbelmiro: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[milinddethe15]

The problem here is that the folder artifacts are not getting stored in mino. After running the pipeline to create a folder artifact:

• check /v2/artifacts on the minio UI (Browser Access)
• It doesnt show any artifact files of the pipeline.

[zeidsolh]

@milinddethe15 Where are they being stored?

[zeidsolh]

Do you have any updates on this @zijianjoy ?