Bump minio client version to support `assume-web-identity-role` for AWS

[Jeffwan]

What steps did you take:

aws add IAM For Service Account feature and aws user doesn't have to inject AWS credentials as env variables.

This requires us to use high AWS SDK version which supports assume-web-identity-role. In this case, we need to add new resolver for minio client.

Here's the minio js and golang version.

• https://github.com/kubeflow/pipelines/blob/a92d52242524309a3e684f46d7f7bb570495362d/frontend/server/package.json#L13

• https://github.com/kubeflow/pipelines/blob/master/go.mod#L50

The minimum version to support this issue is

• https://github.com/minio/minio-go/releases/tag/v6.0.45
• minio-js side seems doesn't support this yet.

/kind feature /area frontend /area backend

[Jeffwan]

We also need to bump argo workflow version to make sure Argo sidecar use right SDK to persist artifacts to S3.

[discordianfish]

To be clear, this is different from assuming instance profiles via kube2iam / kiam

[Jeffwan]

Argo side, argo tracks awssdk upgrade in this issue. https://github.com/argoproj/argo/issues/1774. This has been included in https://github.com/argoproj/argo/releases/tag/v2.5.0. We need this version at least in KFP

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.

[stale[bot]]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.

[Jeffwan]

/reopen

[k8s-ci-robot]

@Jeffwan: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[PatrickXYS]

/assign

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Nagarajj]

@PatrickXYS minio-go module is still at 6.0.14, we would like to use IRSA on EKS, Is there a way this could be bumped up to 6.0.45 or latest. Thanks

Also any plans to do the same for minio-js ?

[PatrickXYS]

/assign @surajkota

Suraj will be working on AWS part

[PatrickXYS]

/unassign

[surajkota]

/unassign

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[LEDfan]

Hi, please re-consider to add support for this. In the meantime the minio-js client also got support for IRSA: https://github.com/minio/minio-js/pull/960 I don't think it's already part of a release, but hopefully the JS parts of Kubeflow can be updated as well somwhere soon. Thanks!

[goswamig]

@LEDfan yes, we are waiting for minio-js release 7.0.27 so that we can enable this feature.

[goswamig]

Minio-js 7.0.27 was released https://github.com/minio/minio-js/tree/7.0.27 with assume role support

[rawc0der]

@Jeffwan @surajkota @PatrickXYS any plans here?

[surajkota]

Yes, we do.

We had started on this and updated minio-go in #7946 but this is not sufficient for the UI to work. For the UI to work, we need minio-js support. Minio-js is also available but the corresponding typescript bindings are not. We have paused on this right now also to investigate user experience. Some of the service accounts are in Kubeflow namespace and some are in user profile namespace. Currently, the ml-pipeline-minio-artifact secret is copied to user namespace so using secrets works.

@rrrkharse feel free to post the latest on this, if I missed something

[rimolive]

Closing this issue. Looks like the PR is closed and it covered what this issue asked as feature.

/close

[google-oss-prow[bot]]

@rimolive: Closing this issue.

In response to this:

Closing this issue. Looks like the PR is closed and it covered what this issue asked as feature.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.