feat(frontend): replace variables in ARGO_ARCHIVE_PREFIX

[LEDfan]

Hi, this PR is meant to start a discussion on the feature, based on a possible implementation.

Description of your changes:

#2081 adds supports for retrieving the logs of a pipeline from the Argo artifact store. This is great, but one thing is missing for us: when storing artifacts/logs Argo allows to configure the path. This option allows to specify some variables, typically you use one of:

• workflow.creationTimestamp
• workflow.name
• workflow.namespace
• pod.name

but more variables are supported https://github.com/argoproj/argo-workflows/blob/516c54a7f04e6a0042028674c5b172fbc8efdd5b/docs/variables.md#global .

The workflow.namespace variable is very useful in multi-user deployments. Using this variable it is possible to store the artifacts of an user in their own directory (having one directory for every user). Unfortunately, these variables aren't supported by the Pipelines frontend. This PR adds a simple, but effective implementation. However, at the time the UI code fetches the logs from Argo, not all metadata of the workflow is known (in fact, I'm not sure it's possible to retrieve all this metadata again?). Therefore only a very limited amount of variables are supported:

• pod.name
• workflow.name
• workflow.namespace

Although, this is a very limited set of variables compared to what Argo supports, this covers the (IMO) must important use-case of isolating the artifacts of a user in their own directory.

What do you think? Is this a viable middle-way?

Example configuration:

           - name: ARGO_ARCHIVE_LOGS
              value: "true"
            - name: ARGO_ARCHIVE_ARTIFACTORY
              value: "minio"
            - name: ARGO_ARCHIVE_BUCKETNAME
              value: "mlpipeline-dev"
            - name: ARGO_ARCHIVE_PREFIX
              value: "artifacts/{{workflow.namespace}}/{{workflow.namespace}}/{{workflow.name}}/{{pod.name}}"

Checklist:

• [x] The title for your pull request (PR) should follow our title convention. Learn more about the pull request title convention used in this repository.

[google-oss-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: To complete the pull request process, please assign bobgy after the PR has been reviewed. You can assign the PR to them by writing /assign @bobgy in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• frontend/OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[google-cla[bot]]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

:memo: Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[google-oss-robot]

Hi @LEDfan. Thanks for your PR.

I'm waiting for a kubeflow member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[LEDfan]

@googlebot I signed it!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[zijianjoy]

Thank you for creating this PR! I have a couple questions about this:

1. Is pod.name a global variable? Is this field usable?
2. If we have pod.name, do we still need workflow.name?
3. Is there any backend change we need to merge before this PR? What happen if we cannot find pod log from this new prefix? Can we fallback to old prefix to get pod log?
4. Does this work for both full Kubeflow and KFP standalone scenario?